Column bit feature efficient identification method based on byte alignment

Abstract

The invention discloses an efficient column bit feature recognition method based on byte alignment, and belongs to the technical field of data processing. According to the method, different byte alignment processing modes are adopted according to whether the frame length in a data stream is integral multiple of 8 bits or not: when the frame length is integral multiple of 8 bits, a structured data group is constructed, and byte-level sliding scanning and jumping access are performed based on a processing window so as to extract and identify a feature sequence; and when the frame length is not an integer multiple of 8 bits, combining a plurality of frames into a multiframe to realize byte alignment, and identifying by remapping original column bit features into a plurality of parallel short sequences. Complex bit-level shifting and mask calculation are avoided through the byte alignment operation, the data processing efficiency and the cache hit rate are remarkably improved, the CPU load is reduced, meanwhile, the method can flexibly adapt to column bit feature recognition requirements of different frame lengths and intervals, and the real-time processing requirement of high-speed data streams is effectively met.

Description

Technical Field

[0001] This invention relates to the field of computer network data processing technology, and specifically to a method for efficient identification and extraction of network traffic based on bit features. Background Technology

[0002] In the field of digital communication and data processing, column bit feature recognition is fundamental to achieving key tasks such as frame synchronization, protocol identification, and data verification. A column bit feature refers to a pattern of bit sequences that repeat at fixed intervals in a data stream; these patterns typically have a clear location and synchronization function within the data frame structure.

[0003] Currently, common identification methods mainly include the following categories:

[0004] One type is the bit-by-bit scanning-based pattern matching method. This type of method uses an improved version of the classic Z-Bit string matching algorithm to perform precise matching by sliding a window bit by bit in the bit stream. However, this method has high computational complexity when processing high-speed, long bit streams, making it difficult to meet real-time requirements.

[0005] Another type is feature-based deep packet inspection technology. This method identifies protocols by matching predefined strings or bit sequences in the packet payload, but it suffers from problems such as feature redundancy and limited recognition accuracy when dealing with bitstream protocols, and its automation level is not high.

[0006] In addition, there are reverse analysis methods based on frequent sequence statistics and association rule mining. For example, multi-pattern matching algorithms such as Apriori are used to scan the bitstream, count frequent sequences, and mine the synchronization header structure to achieve data frame segmentation. These methods are effective in parsing unknown protocols, but the overall process is computationally cumbersome, and the clustering efficiency and accuracy still need improvement.

[0007] In terms of bit-level operation optimization, some research focuses on improving bitstream matching efficiency, such as the Z-Bit algorithm, which aims to achieve fast string search in bitstreams; other works improve the efficiency of leading zero skipping in Z-curve encoding and decoding by introducing lookup tables and bit detection mechanisms. However, these methods have poor versatility when dealing with non-byte aligned data and still have certain limitations.

[0008] In summary, existing column bit feature recognition methods have the following three significant drawbacks: First, they suffer from low processing efficiency, as bit-by-bit scanning involves numerous shifting and masking operations, resulting in discontinuous memory access and low cache hit rates. Second, they consume excessive system resources, with frequent bitstream conversion and matching operations leading to high CPU load, especially when the features are long, where resource consumption increases linearly. Finally, they lack algorithm flexibility, with most schemes failing to effectively adapt to scenarios where frame lengths are not multiples of 8 bits, and exhibiting weak adaptability to the recognition of features at different intervals.

[0009] Therefore, there is an urgent need for a column bit feature recognition scheme that can balance recognition accuracy, processing efficiency and structural flexibility to address the technical challenges of high-speed data stream processing and non-byte aligned scenarios. Summary of the Invention

[0010] In view of the above-mentioned shortcomings of the existing technology, the present invention provides an efficient column bit feature recognition method that can significantly improve processing efficiency while ensuring recognition accuracy, and can flexibly deal with different interval lengths and non-byte alignment of frame lengths.

[0011] To achieve the above objectives, the present invention provides the following technical solution:

[0012] An efficient method for identifying column bit features based on byte alignment includes the following steps:

[0013] S1. Receive the data stream and determine the frame length m of the frames in the data stream;

[0014] S2. Based on whether the frame length m is an integer multiple of 8 bits, column bit feature identification is performed using byte alignment processing.

[0015] When the frame length m is an integer multiple of 8 bits, the structured data processing flow is executed;

[0016] When the frame length m is not an integer multiple of 8 bits, the multiframe processing flow is executed.

[0017] Furthermore, when the frame length m is an integer multiple of 8 bits, the structured data processing flow includes:

[0018] Step a). Set initial parameters: column bit feature length is k, and the number of bytes corresponding to a frame in the data stream is n, where n = m / 8;

[0019] Step b). Establish structured data groups, each containing n bytes, and convert the data stream into structured data groups. The column bit features are distributed in the same positions in different structured data groups. The same positions are the positions in each structured data group where the byte number and bit number are the same.

[0020] Step c). Set the structured processing window, the length L of which is determined based on the column bit feature length k and the number of bytes n corresponding to the frame in the data stream;

[0021] Step d). Slide scan the structured data group, sliding in 1-byte increments within the structured processing window, for a maximum of n bytes, and then slide the entire structured processing window for k+1 structured data bytes.

[0022] Step e). Extract the feature sequence by skipping through the data byte by byte and reading the bits at the same position in the structured data in the structured processing window to form the feature sequence;

[0023] Step f). Feature sequence identification:

[0024] When the feature sequence matches the column bit feature, the matching result is output;

[0025] If the feature sequence does not match the column bit feature, determine whether the structured data reading in the current structured processing window has ended. If not, return to step e) to read the next feature sequence; if finished, return to step d).

[0026] Furthermore, the feature sequence is packaged into a first integer, and the column bit feature is defined as a second integer. The matching operation is completed by comparing whether the first integer and the second integer are equal.

[0027] Furthermore, when the feature sequence matches the column bit feature, a secondary verification is included before outputting the matching result. The secondary verification extracts one or more verification feature sequences from subsequent positions of the structured processing window and matches the verification feature sequences with the column bit feature.

[0028] When all verification feature sequences match successfully, the matching result is output;

[0029] If at least one verification sequence does not match, determine whether the reading of structured data in the current structured processing window has ended. If not, return to step e) to read the next feature sequence; if not, return to step d).

[0030] Furthermore, the starting point of the subsequent position of the structured processing window is separated from the starting point of the previous successfully matched feature sequence by k structured data units.

[0031] Furthermore, the length L of the structured processing window is k*n*2 structured data.

[0032] Furthermore, when the frame length m is not an integer multiple of 8 bits, 8 frames are combined into a multiframe, and the multiframe processing flow includes:

[0033] Step a). Set the initial parameters, the column bit feature length is k. Based on the column bit feature length k, and according to 8*p=k*q, calculate that p multiframes are needed to fill q complete column bit features, where p and q are both positive integers.

[0034] Step b). Convert the column bit features into column bit sequences. Fill the q*k bits contained in the q column bit features into a P-row 8-column matrix in row-major order. Each column in the matrix constitutes a column bit sequence, generating 8 column bit sequences of length p bits. The 8 column bit sequences are numbered sequentially for use as matching targets in the subsequent identification process.

[0035] Step c). Construct a data stream multiframe group by combining 8 frames in the data stream into a multiframe to form the data stream multiframe group. The total number of bits in the multiframe is an integer multiple of 8.

[0036] Step d). Set up a multiframe processing window, the length S of which is determined based on the number of multiframes p;

[0037] Step e). Slide scan the data stream multiframe group, slide scan in 1-byte increments within the multiframe processing window, slide a maximum of m bytes, and then slide the entire multiframe processing window for p+1 multiframes;

[0038] Step f). Extract the feature sequence by skipping through the target bytes at byte intervals in the multiframe processing window, and extract the specified bits from the target bytes to form the feature sequence.

[0039] Step g). Feature sequence identification:

[0040] When the feature sequence matches the first column bit sequence, proceed to the next step;

[0041] If the feature sequence does not match the first column bit sequence, determine whether the current multiframe processing window data reading has ended. If it has not ended, return to step f) to read the next feature sequence; if it has ended, return to step e).

[0042] Step h). Extract seven consecutive feature sequences from subsequent positions within the multiframe processing window and sequentially verify the other seven column bit sequences:

[0043] When all 8 column bit sequences match, the matching result is output;

[0044] If one of the column bit sequences does not match, determine whether the current multiframe processing window data reading has ended. If not, return to step f) to read the next feature sequence; if it has ended, return to step e).

[0045] Furthermore, when the column bit feature length k is 2, 4 or 8, the number of multiframes p is set to 3.

[0046] Furthermore, the length S of the multiframe processing window is 2*p multiframes.

[0047] Furthermore, the starting point of the subsequent position of the multiframe processing window is separated from the starting point of the previous successfully matched feature sequence by a full column bit feature with a length of k.

[0048] The technical solution provided by this invention has the following advantages compared with the known prior art:

[0049] This invention innovatively provides two efficient paths—structured data processing and multiframe processing—for two scenarios: whether the frame length is a multiple of 8 bits. This allows the invention to not only efficiently handle the ideal case of byte alignment but also flexibly address complex scenarios where the frame length is not byte aligned. It can adapt to the needs of column bit feature recognition with various interval lengths in different communication protocols and data formats, demonstrating excellent versatility.

[0050] The byte alignment and integer comparison (packing bit sequences into integers for comparison) techniques employed in this invention transform the traditional, complex bit-by-bit scanning and matching operations into efficient byte-level skip access. This avoids numerous bit shifting and masking operations, resulting in a more continuous memory access pattern and significantly improving CPU cache hit rate, thereby meeting the real-time processing requirements of high-speed data streams.

[0051] The method employs a multi-factor identification mechanism that combines initial matching with continuous verification. After a successful initial matching, multiple feature sequences are extracted from subsequent positions for verification, effectively avoiding misidentification caused by accidental similar patterns in the data stream, thus greatly improving the accuracy of identification and the robustness of the system. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0053] Figure 1 This is a flowchart illustrating an efficient method for identifying column bit features based on byte alignment.

[0054] Figure 2 This is a schematic diagram of the structured data processing workflow.

[0055] Figure 3 This is a schematic diagram of a structured data processing flow that includes secondary verification.

[0056] Figure 4 This is a diagram illustrating the byte alignment of a data stream multiframe group.

[0057] Figure 5 This is a schematic diagram of the multiframe processing flow. Detailed Implementation

[0058] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0059] The core of this invention lies in employing different efficient processing paths based on whether the frame length is 8-bit aligned. In implementation, the system first receives the data stream and determines its frame length m. Then, based on whether m is an integer multiple of 8, it automatically selects to execute either a structured data processing flow or a multiframe processing flow.

[0060] like Figure 1 As shown, this embodiment provides an efficient method for identifying column bit features based on byte alignment, including the following steps:

[0061] An efficient method for identifying column bit features based on byte alignment includes the following steps:

[0062] S1. Receive the data stream and determine the frame length m of the frames in the data stream;

[0063] S2. Based on whether the frame length m is an integer multiple of 8 bits, column bit feature identification is performed using byte alignment processing methods respectively;

[0064] When the frame length m is an integer multiple of 8 bits, the structured data processing flow is executed;

[0065] When the frame length m is not an integer multiple of 8 bits, the multiframe processing flow is executed.

[0066] This embodiment innovatively provides two efficient paths for structured data processing and multiframe processing, respectively, for the two cases where the frame length is not a multiple of 8 bits. This enables the invention not only to efficiently handle the ideal case of byte alignment, but also to flexibly cope with complex scenarios where the frame length is not byte aligned. It can adapt to the needs of column bit feature recognition with various interval lengths in different communication protocols and data formats, and has excellent versatility.

[0067] like Figure 2 As shown, when the frame length m is an integer multiple of 8 bits, the structured data processing flow includes:

[0068] Step a). Set initial parameters: column bit feature length is k, and the number of bytes corresponding to a frame in the data stream is n, n=m / 8; in this embodiment, column bit feature length k is 3, frame length m=16 (an integer multiple of 8 bits), and frame length m is 2 bytes.

[0069] Step b). Establish structured data groups, each containing n bytes, and convert the data stream into structured data groups. The column bit features are distributed in the same positions in different structured data groups. The same positions are the positions in each structured data group where the byte number and bit number are the same.

[0070] This embodiment establishes a structured data structure group, organizing the data stream into multiple 16-bit (2-byte) long structured data, converting the byte stream into a structured data group:

[0071] Structured data 0: [byte0: 11001111, byte1: 11001111],

[0072] Structured data 1: [byte0: 11001111, byte1: 11001111],

[0073] Structured data 2: [byte0: 11001111, byte1: 11001111].

[0074] When the frame length is a multiple of 8 bits, the data stream is processed into structured data. If column bit features exist at the same position in different structured data, and the 0th byte bit in each structured data is 5, then the column bit features are extracted.

[0075] Extract the 5th bit from the 0th byte of structured data 0, the value is 1 (11001111); extract the 5th bit from the 0th byte of structured data 1, the value is 1 (11001111); extract the 5th bit from the 0th byte of structured data 2, the value is 1 (11001111); thus obtaining the column bit feature [1,1,1] of length k=3.

[0076] By pre-structuring the data, byte-level "jumping" memory access can be performed directly. To obtain the entire feature sequence, it is only necessary to access the same fixed bit of the same fixed byte of data[0], data[1], and data[2] in sequence. This greatly reduces the computational overhead and improves the locality and efficiency of memory access.

[0077] Step c). Set the structured processing window. The length L of the structured processing window is determined based on the column bit feature length k and the number of bytes n corresponding to the frame in the data stream. In this embodiment, L=k*n.

[0078] Setting the processing window length L based on k (column bit feature length) and n (number of bytes corresponding to the frame) is the optimal solution after a precise trade-off between reliability, efficiency, and feasibility. It ensures that at any point within the processing window, once an initial match is found, there must be enough data within the window to complete at least one full verification process to prevent false identification.

[0079] Step d). Slide scan the structured data group, slide scan in steps of 1 byte within the structured processing window, slide a maximum of n bytes, and then slide the entire structured processing window for k+1 structured data; in this embodiment, slide scan in steps of 1 byte within the structured processing window, slide a maximum of 2 bytes, and then slide the entire structured processing window for 4 structured data.

[0080] This embodiment employs a two-layer nested sliding window mechanism, aiming to balance detection accuracy and scanning efficiency. The inner 1-byte step ensures accurate positioning of the feature's starting offset, while the outer k+1 large-step sliding avoids the extremely low efficiency of single-byte sliding across the entire scope, greatly reducing the number of "processing windows" that need to be established and scanned. The k+1 step size and the sufficiently large window size L together guarantee a 100% detection rate.

[0081] Step e). Extract the feature sequence by skipping through the data byte by byte and reading the bits at the same position in the structured data in the structured processing window to form the feature sequence.

[0082] Because the data stream is preprocessed into a neat two-dimensional array (each structured data has 2 bytes), the column bit features are located at the same position in each structured data. Within each target byte, non-target positions can be skipped directly, and the target bits can be extracted. The target bit values ​​at the same position in the structured data within the processing window are read, i.e., the j-th bit of the i-th byte of structured data 0, the j-th bit of the i-th byte of structured data 1, and so on. The structured data in the window is read sequentially according to this pattern, and the read bit values ​​are combined into a bit sequence.

[0083] This embodiment transforms the traditional, complex bit-by-bit scanning and matching operations into efficient byte-level skip access. This avoids numerous bit shifting and masking operations, resulting in a more continuous memory access pattern and significantly improving CPU cache hit rate, thereby meeting the real-time processing requirements of high-speed data streams.

[0084] Step f). Feature sequence identification:

[0085] When the feature sequence matches the column bit feature, the matching result is output;

[0086] If the feature sequence does not match the column bit feature, determine whether the reading of structured data in the current structured processing window has ended. If not, return to step e) to read the next feature sequence; if it has ended, return to step d).

[0087] Step f) compares the extracted feature sequence with the pre-known column bit features we are looking for to determine if they match. To achieve fast feature matching, the feature sequence and column bit features can be compared bit by bit. To improve matching efficiency, in another embodiment, the feature sequence is packaged into a first integer, and the column bit features are defined as a second integer. The matching operation is completed by comparing whether the first integer and the second integer are equal.

[0088] If a system determines successful identification based on only one match, it will generate a large number of false positives, causing subsequent processing steps (such as frame parsing and protocol analysis) to be performed based on incorrect locations, thus rendering them completely ineffective. Therefore, in another embodiment of the present invention, as... Figure 3 As shown, a schematic diagram of a structured data processing flow including secondary verification is provided. When a feature sequence matches a column bit feature, a secondary verification is also included before outputting the matching result. The secondary verification extracts one or more verification feature sequences from the subsequent position of the structured processing window and matches the verification feature sequences with the column bit features.

[0089] When all verification feature sequences match successfully, the matching result is output;

[0090] If at least one verification sequence does not match, determine whether the reading of structured data in the current structured processing window has ended. If not, return to step e) to read the next feature sequence; if not, return to step d).

[0091] In this embodiment, the starting point of the subsequent position of the structured processing window is separated from the starting point of the previous successfully matched feature sequence by k structured data units.

[0092] After a successful match at the j-th bit of the i-th byte of the N-th structured data, it doesn't immediately output the result. It jumps to position N+k and checks if the sequence there is also [1,1,1]. If it is, it might continue jumping to position N+2k for a third verification. Only when all these consecutive positions match successfully does it finally confirm. This eliminates the vast majority of false positives caused by data randomness.

[0093] To ensure that the "preliminary matching + continuous verification" identification mechanism has complete data support and that the verification process is not interrupted due to insufficient data, the length L of the structured processing window is k*n*2 structured data points.

[0094] Initial matching requires consecutive access to k pieces of structured data (each contributing 1 bit to the feature sequence). Verification to prevent false identification also requires consecutive access to k more pieces of structured data. Therefore, a complete identification cycle requires 2k consecutive pieces of structured data, where the parameter n (the number of bytes per piece of structured data) acts as a unit conversion factor. It converts the abstract "number of structured data" into a specific memory byte length, thereby achieving precise control over the physical memory window size.

[0095] The structured processing window size in this embodiment ensures that once a preliminary match is found at any starting position within the processing window, there will be at least k data units available for continuous verification. This fundamentally avoids missed detections caused by window boundary cutting. This size is the minimum size that meets the above reliability requirements, avoiding increased memory usage and internal scanning computation caused by an excessively large window, achieving a perfect balance between efficiency and reliability.

[0096] When the frame length m is not an integer multiple of 8 bits, 8 frames are combined into a multiframe. The multiframe processing flow is as follows: Figure 5 As shown, it includes:

[0097] Step a). Set the initial parameters, the column bit feature length is k. Based on the column bit feature length k, and according to 8*p=k*q, calculate that p multiframes are needed to fill q complete column bit features, where p and q are both positive integers.

[0098] In this embodiment, the column bit feature length is k=12. Based on 8*p=12*q, we get p=3 and q=2. Therefore, as follows... Figure 4 As shown, a multiframe consists of 8 frames. Therefore, for any "same byte position, same bit position" within a multiframe, we can extract 8 bits from left to right. We call these 8 positions "columns". When there are p = 3 multiframes, for any "column" defined above, we can extract 3 bits from top to bottom. In this embodiment, the original 24-bit sequence (2 column bit features) is filled into this matrix.

[0099] Step b). Convert the column bit features into column bit sequences. Take the q*k bits contained in the q column bit features and fill them into a P-row 8-column matrix in row-major order. Each column in the matrix constitutes a column bit sequence, generating 8 column bit sequences of length p bits. The 8 column bit sequences are numbered sequentially and used as matching targets in the subsequent recognition process.

[0100] Step b) transforms the column bit features to be identified into a new form that is more suitable for efficient identification under the "multiframe" structure, as shown in the table below. The column bit features are transformed into 8 parallel column bit feature sequences of length 3.

[0101] E1 E2 E3 E4 E5 E6 E7 E8 E9 E10 E11 E12 E1 E2 E3 E4 E5 E6 E7 E8 E9 E10 E11 E12

[0102] Including sequence 1: [E1, E9, E5], sequence 2: [E2, E10, E6], sequence 3: [E3, E11, E7], sequence 4: [E4, E12, E8]... sequence 8: [E8, E4, E12].

[0103] In the original bitstream, tracing a long (k) non-byte aligned sequence requires a large number of bit-level operations, which is inefficient. In this embodiment, eight shorter (n) fixed-position sequences are found in the byte-aligned multiframe to improve processing efficiency.

[0104] Step c). Construct a data stream multiframe group by combining 8 frames in the data stream into a multiframe, forming a data stream multiframe group. The total number of bits in the multiframe is an integer multiple of 8.

[0105] Step d). Set up the multiframe processing window. The length S of the multiframe processing window is determined based on the number of multiframes p.

[0106] Step e). Slide scan the data stream multiframe group, sliding in 1-byte increments within the multiframe processing window, sliding up to m bytes, and then the entire multiframe processing window slides p+1 multiframes. In this embodiment, the sliding scan is performed in 1-byte increments within the processing window, sliding up to a maximum multiframe length of 193 bytes, and then the entire processing window slides 4 multiframes.

[0107] Step f). Extract the feature sequence by skipping through the target bytes at byte intervals in the multiframe processing window and extract the specified bits from the target bytes to form the feature sequence.

[0108] Step g). Feature sequence identification:

[0109] When the feature sequence matches the first column bit sequence, proceed to the next step;

[0110] If the feature sequence does not match the first column bit sequence, return to step f) to read the next feature sequence, then return to step e).

[0111] In this embodiment, the column bit sequence [E1, E9, E5] is first matched. If the matching fails, it is determined whether the data reading of the current multiframe processing window has ended. If it has not ended, step f) is executed to read the next bit sequence. If the multiframe reading has ended, step e) is executed. If the recognition is successful, step h) is executed.

[0112] Step h). Extract 7 consecutive feature sequences from subsequent positions in the multiframe processing window and sequentially verify the other 7 column bit sequences:

[0113] Output the matching result when all 8 column bit sequences match;

[0114] If a column bit sequence does not match, determine whether the current multiframe processing window data reading has ended. If not, return to step f) to read the next feature sequence; otherwise, return to step e).

[0115] Specifically, when the column bit feature length k is equal to 2, 4 or 8, according to the formula 8*p=k*q, one multiframe can fill q complete column bit features. In order to enhance the recognition features, the number of multiframes is specifically set to 3. When the column bit feature length k is 2, 4 or 8, the number of multiframes p is set to 3.

[0116] In another implementation, the frame length m is 193 bits and the column bit feature length k is 4. The specific implementation steps are as follows:

[0117] Step a). Set the initial parameters. The column bit feature length is 4. Based on the column bit feature length of 4, it takes 3 multiframes to fill 6 complete column bit features.

[0118] Step b). Convert the column bit features into column bit sequences. Obtain 8 column bit features, as shown in the table below:

[0119] E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4 E1 E2 E3 E4

[0120] The characteristic of each column bit feature is that it takes the same value, such as [E1,E1,E1] and [E2,E2,E2].

[0121] Step c). Construct a data stream multiframe group by combining 8 frames in the data stream into a multiframe, forming a data stream multiframe group. The total number of bits in the multiframe is an integer multiple of 8.

[0122] Step d). Set the processing window. The length S of the multiframe processing window is 2*p multiframes, that is, a processing window for 6 multiframes.

[0123] Step e). Slide scan the data stream multiframe group. By default, the sliding scan is performed in the processing window with a step of 1 byte. The maximum sliding length is 193 bytes. Then the processing window slides for 4 multiframes.

[0124] Step f). Extract the feature sequence by skipping bytes. Read the target byte at the same position in each multiframe of the processing window, and extract the specified bits from the target byte to form the feature sequence;

[0125] Step g). Identify the feature sequence. Match the column bit sequence {E1,E1,E1}. If the match fails, determine whether the multiframe reading has ended. If not, proceed to step f) to read the next bit sequence. If the reading has ended, proceed to step e). If the identification is successful, proceed to step h).

[0126] Step h): Verify the other 7 bit sequences. Access the sequences in a byte-interval skipping manner, starting from the currently matched position, extracting multiple bit sequences and identifying them sequentially. If any bit sequence fails to be identified, it indicates that the identification in step g) was a misidentification. Further determine if the multiframe data reading has ended. If not, execute step f) to retrieve the next bit sequence. If the multiframe reading has ended, execute step e). If all are successfully identified, it indicates that the column feature identification was successful, and the matching result is output.

[0127] The starting point of the subsequent position in the multiframe processing window is separated from the starting point of the previously successfully matched feature sequence by a full column bit feature with a length of k.

[0128] The byte alignment and integer comparison techniques employed in this embodiment transform the traditional, complex bit-by-bit scanning and matching operations into efficient byte-level skip access. This avoids numerous bit shifting and masking operations, resulting in a more continuous memory access pattern and significantly improving CPU cache hit rate, thereby meeting the real-time processing requirements of high-speed data streams.

[0129] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0130] The above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for efficient identification of column bit features based on byte alignment, comprising the following steps: S1. Receive the data stream and determine the frame length m of the frames in the data stream; S2. Based on whether the frame length m is an integer multiple of 8 bits, column bit feature identification is performed using byte alignment processing. When the frame length m is an integer multiple of 8 bits, the structured data processing flow is executed; When the frame length m is not an integer multiple of 8 bits, the multiframe processing flow is executed.

2. The efficient identification method for column bit features based on byte alignment according to claim 1, characterized in that, When the frame length m is an integer multiple of 8 bits, the structured data processing flow includes: Step a). Set initial parameters: column bit feature length is k, and the number of bytes corresponding to a frame in the data stream is n, where n = m / 8; Step b). Establish structured data groups, each containing n bytes, and convert the data stream into structured data groups. The column bit features are distributed in the same positions in different structured data groups. The same positions are the positions in each structured data group where the byte number and bit number are the same. Step c). Set the structured processing window, the length L of which is determined based on the column bit feature length k and the number of bytes n corresponding to the frame in the data stream; Step d). Slide scan the structured data group, sliding in 1-byte increments within the structured processing window, for a maximum of n bytes, and then slide the entire structured processing window for k+1 structured data bytes. Step e). Extract the feature sequence by skipping through the data byte by byte and reading the bits at the same position in the structured data in the structured processing window to form the feature sequence; Step f). Feature sequence identification: When the feature sequence matches the column bit feature, the matching result is output; If the feature sequence does not match the column bit feature, determine whether the structured data in the current structured processing window has been read. If not, return to step e) to read the next feature sequence; if not, return to step d).

3. The efficient identification method for column bit features based on byte alignment according to claim 2, characterized in that, The feature sequence is packaged into a first integer, and the column bit feature is defined as a second integer. The matching operation is completed by comparing whether the first integer and the second integer are equal.

4. The efficient identification method for column bit features based on byte alignment according to claim 2, characterized in that, When the feature sequence matches the column bit feature, before outputting the matching result, a secondary verification is also included. The secondary verification extracts one or more verification feature sequences from the subsequent position of the structured processing window and matches the verification feature sequences with the column bit feature. When all verification feature sequences match successfully, the matching result is output; If at least one verification sequence does not match, determine whether the structured data in the current structured processing window has been read. If not, return to step e) to read the next feature sequence; if not, return to step d).

5. The efficient identification method for column bit features based on byte alignment according to claim 4, characterized in that, The starting point of the subsequent position of the structured processing window is k structured data units away from the starting point of the previous successfully matched feature sequence.

6. The efficient identification method for column bit features based on byte alignment according to claim 4, characterized in that, The length L of the structured processing window is k*n*2 structured data.

7. The efficient identification method for column bit features based on byte alignment according to claim 1, characterized in that, When the frame length m is not an integer multiple of 8 bits, 8 frames are combined into a multiframe, and the multiframe processing flow includes: Step a). Set the initial parameters, the column bit feature length is k. Based on the column bit feature length k, and according to 8*p=k*q, calculate that p multiframes are needed to fill q complete column bit features, where p and q are both positive integers. Step b). Convert the column bit features into column bit sequences. Fill the q*k bits contained in the q column bit features into a P-row 8-column matrix in row-major order. Each column in the matrix constitutes a column bit sequence, generating 8 column bit sequences of length p bits. The 8 column bit sequences are numbered sequentially for use as matching targets in the subsequent identification process. Step c). Construct a data stream multiframe group by combining 8 frames in the data stream into a multiframe to form the data stream multiframe group. The total number of bits in the multiframe is an integer multiple of 8. Step d). Set up a multiframe processing window, the length S of which is determined based on the number of multiframes p; Step e). Slide scan the data stream multiframe group, slide scan in 1-byte increments within the multiframe processing window, slide a maximum of m bytes, and then slide the entire multiframe processing window for p+1 multiframes; Step f). Extract the feature sequence by skipping through the target bytes at byte intervals in the multiframe processing window, and extract the specified bits from the target bytes to form the feature sequence. Step g). Feature sequence identification: When the feature sequence matches the first column bit sequence, proceed to the next step; If the feature sequence does not match the first column bit sequence, determine whether the current multiframe processing window data reading has ended. If it has not ended, return to step f) to read the next feature sequence; if it has ended, return to step e). Step h). Extract seven consecutive feature sequences from subsequent positions within the multiframe processing window and sequentially verify the other seven column bit sequences: When all 8 column bit sequences match, the matching result is output; If one of the column bit sequences does not match, determine whether the current multiframe processing window data reading has ended. If not, return to step f) to read the next feature sequence; if it has ended, return to step e).

8. The efficient identification method for column bit features based on byte alignment according to claim 7, characterized in that, When the column bit feature length k is 2, 4 or 8, the number of multiframes p is set to 3.

9. The efficient identification method for column bit features based on byte alignment according to claim 7, characterized in that, The length S of the multiframe processing window is 2*p multiframes.

10. The efficient identification method for column bit features based on byte alignment according to claim 7, characterized in that, The starting point of the subsequent position of the multiframe processing window is separated from the starting point of the previous successfully matched feature sequence by a full column bit feature with a length of k.

Images