Protocol automatic identification method used in Internet of Things environment

Abstract

The invention relates to the technical field of security communication protocols of the Internet of Things, in particular to an automatic protocol identification method used in an environment of the Internet of Things, which comprises the following steps of: acquiring a difference value between a message index and frequency, aggregating a field section to generate a group number, matching a structure offset to calculate a fluctuation quantity, and identifying a period hopping field to output an identification result. According to the method, the distribution characteristics and the structure change trend of byte sections can be effectively described by performing difference analysis on repetition frequency and density change of each byte position in a communication message and establishing a field aggregation relationship, and the structure stability and drift fields in the message are extracted and the offset amplitude of the fields is quantized, so that the communication efficiency is improved. A message structure consistency judgment mechanism is established, a comprehensive evaluation mode of periodic change and structure stability is formed in combination with analysis of a periodic field byte change range and frequency hopping frequency, and field position mapping and periodic characteristics of a newly received message are subjected to matching judgment, so that a message structure consistency judgment result is obtained. Accurate classification and efficient identification of the dynamic change communication data are realized.

Description

Technical Field

[0001] This invention relates to the field of IoT secure communication protocol technology, and in particular to an automatic protocol identification method for IoT environments. Background Technology

[0002] The field of IoT secure communication protocol technology involves achieving secure and reliable data transmission within the IoT ecosystem. This includes secure design of communication protocols, security authentication mechanisms, data encryption and decryption, authentication, intrusion detection and prevention strategies, etc. By constructing communication mechanisms suitable for the resource-constrained characteristics of IoT devices and combining them with methods such as network protocol identification, key management, and access control, the security of data interaction between IoT devices in open and heterogeneous network environments is ensured. Among these methods, automatic IoT protocol identification refers to the way to identify the protocol type used in IoT network communication. This mainly involves determining the protocol category corresponding to communication traffic by setting static rules, feature library matching, or port-based protocol classification. Typically, it relies on the known message structure characteristics of protocols, comparing key fields in network data packets layer by layer, or judging based on the correspondence between port numbers and protocols. There are also methods that use shallow machine learning based on statistical features to model communication features and classify them. However, these methods largely depend on prior protocol knowledge, resulting in limited accuracy and adaptability.

[0003] Existing IoT protocol identification technologies mostly rely on static rules, feature library matching, or port number association for protocol classification. Limited by the coverage of pre-set rules and the completeness of feature libraries, they struggle to accurately locate protocol fields in IoT communication environments with complex protocol formats or dynamically changing field distributions. Comparison modes based on known protocol structures cannot adapt to communication data with unknown protocols or significant structural drift, leading to decreased identification accuracy. Furthermore, they lack in-depth modeling capabilities for periodic communication characteristics, resulting in identification biases when dealing with periodic jumps or field variations. In addition, shallow statistical feature modeling methods are not sensitive enough to changes in the internal structure of messages, and are prone to misjudging protocol types when facing data segments with unstable structures or significant density changes. Overall, their adaptability and robustness are weak, making it difficult to support the intelligent protocol identification needs in complex IoT scenarios. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an automatic protocol identification method for use in the Internet of Things (IoT) environment.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: an automatic protocol identification method for an Internet of Things (IoT) environment, comprising the following steps: S1: Obtain each communication protocol message collected by the communication interface of the IoT edge node, number it sequentially, identify the byte position index and repetition frequency of each byte in the message, perform difference calculation on all byte positions, and generate the field position repetition frequency difference value. S2: Based on the difference in the repetition frequency of the field positions, calculate the density value of consecutive byte positions in the IoT message, extract consecutive byte segments for byte position aggregation, and generate the number of consecutive aggregation groups for field partitions; S3: Based on the number of consecutive aggregation groups of the field partition, perform field structure matching on different message instances in the communication cycle, calculate the maximum starting position offset of each group of field segments, mark structurally stable fields and structurally drifting fields, and generate data on the maximum fluctuation of structural offset. S4: Based on the maximum fluctuation data of the structural offset, obtain the byte value change frequency of each field segment in the periodic message, calculate the difference between the maximum and minimum values ​​of the same field segment in each period, identify the periodic jump field segment and the periodic stable field segment, and generate the periodic jump field segment. S5: Based on the periodic transition field segment, compare the matching relationship of the currently newly received IoT messages and output the protocol automatic identification result.

[0006] As a further aspect of the present invention, the structural stability field specifically refers to the field segment whose offset amplitude is less than a set structural consistency limit threshold. The structural drift field specifically refers to the field segment where the offset amplitude is not less than the set structural consistency limit threshold; The periodic transition field segment specifically refers to the field segment where the difference value is greater than the preset periodic transition judgment threshold; The periodic stable field segment specifically refers to the field segment where the difference value is not greater than the preset periodic jump judgment threshold.

[0007] As a further aspect of the present invention, the field position repetition frequency difference includes the repetition frequency difference value and the difference calculation interval; the number of consecutive aggregation groups of the field partition includes the starting position of the continuous field segment, the length of the continuous field segment, and the gradient change direction feature; the maximum fluctuation data of the structural offset includes the field starting position offset value, the field structure matching ratio, and the structural consistency evaluation index; the set of periodic jump field segments includes the field byte value change frequency, the field period range, and the periodic jump identification threshold; and the protocol automatic identification result includes field position mapping data, the structural and periodic characteristic matching relationship, and field similarity judgment records.

[0008] As a further aspect of the present invention, the specific steps for obtaining the difference in the repetition frequency of the field position are as follows: S111: Acquire each communication protocol message collected by the edge node communication interface, sequentially number each message data frame, extract the complete byte sequence, mark the position of each byte with a position index, establish a set of number and index relationship, and generate a byte position mapping sequence; S112: Based on the byte position mapping sequence, monitor the number of times the corresponding byte at each position index appears repeatedly in the message, perform aggregation and statistical operations on all byte values ​​at the same index position, calculate the total number of occurrences, establish the correspondence between index position and the number of occurrences, and generate an index position repetition frequency table; S113: Based on the index position repetition frequency table, perform pairwise difference operations on the repetition frequency values ​​between each index position to obtain the set of frequency difference values ​​between adjacent interval positions, and perform position mapping reconstruction based on the numerical distribution of the frequency difference values ​​to obtain the field position repetition frequency difference value.

[0009] As a further aspect of the present invention, the specific steps for obtaining the number of consecutive aggregation groups for field partitioning are as follows: S211: Based on the difference in the repetition frequency of the field position, extract the continuous byte position index sequence in all messages, perform a ratio calculation on the cumulative number of adjacent bytes and the position span, obtain the number of bytes per unit length in each continuous position segment, and generate a continuous position density value group. S212: Based on the continuous position density value group, perform difference calculation on the density values ​​between adjacent position segments, calculate and obtain the interval density change metric value, filter the continuous segment intervals that are less than the preset density difference limit threshold, and obtain the low difference continuous density segment index set. S213: Based on the low-difference continuous density segment index set, perform directional consistency verification on the byte index sequence within the index interval, filter the interval sequences with the same density change direction, perform aggregation processing on the index position set that satisfies directional consistency, obtain the byte position set to which the field belongs in the continuous region, and establish the number of continuous aggregation groups for field partitioning.

[0010] As a further aspect of the present invention, the formula for calculating the interval density change metric is as follows: ; in, , Represents byte position index , The corresponding density value, Indicates the position within the interval density value, For interval to The average value of internal density, It is a measure of density variation within a range, reflecting the degree of change at the endpoints and the volatility of the range.

[0011] As a further aspect of the present invention, the specific steps for obtaining the maximum structural offset fluctuation data are as follows: S311: Based on the number of consecutive aggregation groups of the field partition, perform field structure matching on each message instance within the communication period, extract the starting position index and byte length of each group of fields in the corresponding message, aggregate the starting position indexes of the same group of fields recorded in different messages, and generate field position aggregation groups. S312: Based on the field position aggregation group, calculate the difference of the starting offset value of each group of fields in all message instances, extract the span of the change interval by calculating the difference between the maximum and minimum values ​​of the starting offset value, perform the corresponding operation on each field group, obtain the corresponding maximum offset value of the starting position, and establish a set of field offset amplitudes. S313: Based on the set of field offset amplitudes, if the offset amplitude is less than the set structural consistency limit threshold, the corresponding field group is marked as a structurally stable field; otherwise, it is marked as a structural drift field. Statistical analysis is performed on the marking status of all field groups to obtain the maximum fluctuation data of structural offset.

[0012] As a further aspect of the present invention, the specific steps for obtaining the periodic transition field segment are as follows: S411: Based on the maximum fluctuation data of the structural offset, monitor the byte value distribution of each field segment in the periodic message within the communication cycle, extract the byte value set of each field segment in each cycle, count the number of occurrences of non-repeating byte values, summarize the change frequency of each field segment in the whole cycle, and generate a field byte change frequency matrix. S412: Based on the field byte change frequency matrix, perform difference calculation on the maximum and minimum byte values ​​of each field segment in the periodic sequence. When the difference value is greater than the set periodic jump judgment threshold, it is marked as a periodic jump field segment, and the other segments are marked as periodic stable field segments, thus obtaining the field periodic jump mark sequence. S413: Based on the field periodic transition marker sequence, retrieve the number of field segments identified as transition fields, perform a deduplication counting operation, output the total number of all field segments that meet the transition conditions in the current communication cycle, and obtain the set of periodic transition field segments.

[0013] As a further aspect of the present invention, the specific steps for obtaining the protocol automatic identification result are as follows: S511: Based on the set of periodic transition field segments, obtain the content of the newly received message, extract the starting index position and mapping structure information of each field segment, compare the structure information with the periodic position distribution of the transition field at the field level, perform position calibration according to the field index alignment method, and establish a field position mapping matrix. S512: Based on the field position mapping matrix, the structural attributes of each field segment in the current message are mapped one-to-one with the periodic jump distribution of the corresponding field in the periodic message, and a joint feature sequence of the current field segment in the structural position and periodic jump state is constructed. The overlap of matching states between all field segments is calculated to obtain the field structure matching similarity vector. S513: Based on the field structure matching similarity vector, perform a structure equivalence measurement judgment on the current message and the historical periodic structure feature set, and output the identification label for the message structure type that meets the structure matching condition to obtain the protocol automatic identification result.

[0014] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, by performing differential analysis on the repetition frequency and density changes of each byte position in the communication message and establishing field aggregation relationships, the distribution characteristics and structural change trends of byte segments can be effectively characterized. By extracting structurally stable and drifting fields in the message and quantifying their offset amplitude, a message structure consistency discrimination mechanism is established. Combined with the analysis of the range of changes and jump frequency of periodic field bytes, a comprehensive evaluation method for periodic changes and structural stability is formed. By matching and judging the field position mapping and periodic characteristics of newly received messages, the adaptability to unknown protocol structures and field fluctuations is improved, enabling accurate classification and efficient identification of dynamically changing communication data, and enhancing the reliability of automatic identification of IoT communication protocols in open and heterogeneous environments. Attached Figure Description

[0015] Figure 1 This is a flowchart of the main steps of the present invention; Figure 2 This is a flowchart of the process for obtaining the difference in the repetition frequency of field positions in this invention; Figure 3 This is a flowchart of the process for obtaining the number of consecutive aggregation groups for field partitioning in this invention; Figure 4 This is a flowchart of the process for obtaining the maximum fluctuation data of structural offset in this invention; Figure 5 This is a flowchart of the process for obtaining the periodic jump field segment in this invention; Figure 6 This is a flowchart of the process for obtaining the automatic identification results of the protocol in this invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0017] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0018] Please see Figure 1 An automatic protocol identification method for use in an Internet of Things (IoT) environment includes the following steps: S1: Obtain each communication protocol message collected by the edge node communication interface in the IoT environment, number it sequentially, record the complete byte sequence and the receiving timestamp, identify the byte position index and repetition frequency of each byte in the message, perform difference calculation on all byte positions based on the repetition frequency, and generate the field position repetition frequency difference value. S2: Based on the difference in the repetition frequency of field positions, count the density value of consecutive byte positions in the IoT message, calculate the density difference of consecutive byte positions and determine whether it is less than the density difference limit threshold, extract consecutive byte segments that conform to the gradient change direction, and aggregate the corresponding byte positions into consecutive field segments to generate the number of consecutive aggregation groups of field partitions. S3: Based on the number of consecutive aggregation groups of field partitions, perform field structure matching on different message instances in the communication cycle, extract the field start offset value and length, calculate the maximum start position offset of each group of field segments, and mark field segments with offset amplitude less than the set structural consistency limit threshold as structurally stable fields, and mark the remaining segments as structurally drifting fields, generating the maximum structural offset fluctuation data. S4: Based on the maximum fluctuation data of structural offset, obtain the byte value change frequency of each field segment in the periodic message, and perform difference calculation on the maximum and minimum values ​​of the same field segment in each period. When the difference is greater than the preset period jump judgment threshold, it is marked as a period jump field segment, and the other segments are marked as period stable field segments, generating a set of period jump field segments. S5: Based on the set of periodic jump field segments, analyze the field position mapping data and periodic variation distribution in the newly received messages of the Internet of Things, compare the matching relationship of each field segment in terms of structure and periodic characteristics, and output the protocol automatic identification results.

[0019] The field position repetition frequency difference includes the repetition frequency difference value and the difference calculation interval; the number of consecutive aggregation groups of field partitions includes the starting position of the consecutive field segment, the length of the consecutive field segment, and the gradient change direction characteristics; the maximum fluctuation data of structural offset includes the field starting position offset value, the field structure matching ratio, and the structural consistency evaluation index; the set of periodic jump field segments includes the frequency of field byte value changes, the field period range, and the periodic jump identification threshold; the protocol automatic identification results include field position mapping data, the matching relationship between structure and periodic characteristics, and field similarity judgment records.

[0020] Please see Figure 2 The specific steps of S1 are as follows: S111: Acquire each communication protocol message collected by the edge node communication interface, sequentially number each message data frame, extract the complete byte sequence, mark the position of each byte with a position index, establish a set of number and index relationship, and generate a byte position mapping sequence; For each communication protocol message collected from the edge node communication interface, the first step is to lock the binary bit stream input from the physical layer and convert it into a hexadecimal byte sequence in groups of 8 bits. For 1000 Modbus-TCP protocol message instances within the collection window, the processor reads the receive timestamp in the message header, accurate to the microsecond level. Then, a linear scan is performed on the byte data frame of each message, starting with an initial index value of 0 from the first byte and incrementing sequentially until the last byte of the message. For example, for a message with a length of 64 bytes, an index sequence is generated. The processor binds each index position to the specific hexadecimal byte value at that position (such as 0xFF, 0x01) and the corresponding microsecond-level timestamp, constructs a structure containing four-dimensional data of "message ID-byte index-byte value-timestamp", and stores the structure data of all messages in the cache queue to form a byte position mapping sequence.

[0021] S112: Based on the byte position mapping sequence, monitor the number of times the corresponding byte at each position index appears repeatedly in the message, perform aggregation and statistical operations on all byte values ​​at the same index position, calculate the total number of occurrences, establish the correspondence between index position and the number of occurrences, and generate an index position repetition frequency table; Based on the byte position mapping sequence, the statistics unit traverses all message instances in the buffer queue, targeting each fixed position index in the sequence. (For example This involves retrieving the set of byte values ​​corresponding to the index within 1000 packets. If the byte value at index 5 is 0x06 980 times and 0x05 20 times in the 1000 packets, an aggregate count is performed on all byte values ​​appearing at that position. The recurrence intensity of the fixed value at that index position is calculated, i.e., the total number of occurrences of the dominant byte values ​​at that position. In this example, if the focus is on key protocol features, the cumulative count of major high-frequency bytes is calculated, or the count of all byte values ​​remaining constant at that position is calculated. If a certain index position... If the value is always 0x00 in all messages, the total number of occurrences at that position is recorded as 1000. If a position is a random check bit with frequently changing values, the number of times the same value is repeated is extremely low. The processor uses the maximum number of repetitions of the maximum single-byte value corresponding to each index position as the frequency characteristic value of that position. For example, the frequency value of indexes 0 to 3 is 1000 (constant header), the frequency value of index 4 is 1000, and the frequency value of index 5 is 50 (high-frequency changing data field). The processor then traverses the full-length byte index in this way to generate an index position repetition frequency table.

[0022] S113: Based on the index position repetition frequency table, perform pairwise difference operations on the repetition frequency values ​​between each index position to obtain the set of frequency difference values ​​between adjacent interval positions. Perform position mapping reconstruction based on the numerical distribution of the frequency difference values ​​to obtain the field position repetition frequency difference value. According to the repetition frequency table at the index position, the processor reads the repetition frequency value at each position in index order. Perform adjacent position difference operation For example, if the frequency value of index 3 is 1000 and the frequency value of index 4 is 1000, the difference is 0, indicating a continuous structure. If the frequency value of index 7 is 1000 and the frequency value of index 8 is 50, the difference is 950, indicating a field boundary or structural abrupt change. The processor traverses the entire frequency table to obtain the result. The frequency difference set is formed, and then the position points where the difference is non-zero and greater than the preset noise reference value (e.g., set to 10) are identified according to the numerical distribution of the difference. These points are marked as potential field segmentation points. The original linear byte index is logically segmented and reconstructed based on these segmentation points. The continuous high-frequency steady-state region and the low-frequency dynamic region are quantized and distinguished by the difference amplitude to obtain the field position repetition frequency difference.

[0023] Please see Figure 3 The specific steps of S2 are as follows: S211: Based on the difference in the repetition frequency of field positions, extract the sequence of consecutive byte position indices in all messages, perform a ratio calculation between the cumulative number of adjacent bytes and the position span, obtain the number of bytes per unit length in each consecutive position segment, and generate a group of consecutive position density values. Based on the difference in the frequency of repetition at field positions, the processor extracts the sequence of consecutive byte positions marked as the same logical segment from all messages. For each consecutive position segment (e.g., the index interval [12, 18]), the total number of bytes in the segment (i.e., 7 bytes) is counted, and the position span of the segment in the original message is calculated, and a ratio operation is performed. In fixed-length protocols, this value usually approaches 1, but it fluctuates in variable-length or padded regions. Here, we introduce a practical example: If a continuous segment covers indices 10 to 14, contains 5 valid bytes, and the difference distribution shows that the frequency consistency within the segment is extremely high, then we calculate the byte aggregation density per unit length. Specifically, the processor calculates the density index based on the frequency consistency of each position. If the frequency difference in a certain region is all 0, then the density index is set as the ratio of the average frequency to the maximum frequency in that region, and the density characteristics of each continuous segment are established. For example, the density of segment A is 0.98, the density of segment B is 0.21, and a group of density values ​​for continuous positions is generated.

[0024] S212: Based on the continuous set of density values, perform difference calculations on the density values ​​between adjacent location segments using the following formula: ; The algorithm calculates and obtains interval density change metrics, then filters out continuous intervals with density differences less than a preset threshold, resulting in a low-difference continuous density segment index set; where, , Represents byte position index , The corresponding density value, Indicates the position within the interval density value, For interval to The average value of internal density, It serves as a measure of density variation within a given interval, reflecting the degree of change at the endpoints and the variability within the interval. Based on a continuous set of density values, the processor performs a difference calculation on the density values ​​between adjacent location segments, using the following formula: (1); In the formula, Indicates from position index arrive The interval density variation metric is used to measure the stability and endpoint variability of the density distribution within a continuous interval. and These represent the starting index of the interval. and end index The density values ​​at each location, and the absolute value of their difference. This reflects the macroscopic density difference between the two ends of the interval; For interval Any index variable within; Indicates the first term in the interval Density values ​​at each location; This is the arithmetic mean of all density values ​​within this interval; This represents the sum of absolute deviations of the density at each point within the interval from its average value; this part reflects the micro-fluctuations within the interval. This is the normalization coefficient, eliminating the influence of interval length on the cumulative volatility value. The advantage of this formula is that, by combining macroscopic differences at the endpoints with internal microscopic fluctuations, it can accurately identify pseudo-stable intervals that have similar endpoint densities but violent internal fluctuations, thus more accurately screening out truly low-difference continuous segments.

[0025] Here, a practical example is introduced to assign values ​​to the parameters and perform calculations: Set the interval index of the continuous position segment to be calculated as follows: to There are a total of 5 location points.

[0026] Density values ​​at each location obtained and calculated through previous steps As shown in Table 1 below: Table 1. Distribution of density values ​​in continuous location segments; As shown in Table 1, the data source is the density analysis results of a certain industrial control protocol message data segment.

[0027] The calculation process is as follows: Calculate the absolute value of the endpoint difference: ; Calculate the interval average : ; Calculate the absolute deviation of each point from the average value. : ; ; ; ; ; Summation of absolute deviation: ; Calculate the normalized fluctuation value: Interval length ; ; Calculate the final interval density change measure : ; The density difference threshold was set to 0.15. This threshold was set with reference to the density fluctuation characteristics of a large number of stable industrial protocol fields (such as TCPHeader and ModbusFunctionCode). Experiments show that the internal density fluctuation of stable fields is usually less than 0.10 and the endpoint difference is less than 0.12, so 0.15 was taken as the tolerance.

[0028] Comparison results: Calculation results The interval is determined to meet the low difference condition.

[0029] The results indicate that the data density structure within the interval from index 3 to 7 is highly stable, and the probability of these data belonging to consecutive fields of the same attribute is extremely high. Therefore, the processor indexes this interval... After adding to the set and filtering, a set of low-difference continuous density segment indexes is obtained.

[0030] S213: Based on the low-difference continuous density segment index set, perform directional consistency verification on the byte index sequence within the index interval, filter the interval sequences with the same density change direction, perform aggregation processing on the index position set that satisfies directional consistency, obtain the byte position set to which the field belongs in the continuous region, and establish the number of continuous aggregation groups for field partitioning; Based on the low-discrepancy continuous density segment index set, the processor processes the set as follows: and Perform directional verification on multiple interval byte index sequences, calculate the sign of the first derivative of the density value within the interval, and if the density in interval A shows a monotonically increasing or stable trend with increasing index (derivative... ), while interval B exhibits a monotonically decreasing trend (derivative). If the direction is inconsistent, the processor only filters interval sequences with the same density change direction (such as both being stable or both fluctuating in the positive direction). For example, it filters all intervals with stable density and minimal fluctuation as static field candidates. It then performs a union operation on the set of adjacent index positions that satisfy the direction consistency. and merged into This allows us to determine the full byte position range of a field within a continuous region and establish the number of continuous aggregation groups for field partitioning.

[0031] Please see Figure 4 The specific steps of S3 are as follows: S311: Based on the number of consecutive aggregation groups of field partitions, perform field structure matching on each message instance within the communication period, extract the starting position index and byte length of each group of fields in the corresponding message, aggregate the starting position indexes of the same group of fields recorded in different messages, and generate field position aggregation groups. Based on the number of consecutive aggregation groups for field partitioning, the processor extracts the field definitions for each aggregated group (e.g., the logical index range corresponding to field group A). Then, reverse matching is performed on the 1000 message instances captured during the communication cycle. For each message instance, the physical starting byte position of the logical field group A in the specific message is identified (e.g., starting at byte 3 in message 1 and starting at byte 3 in message 2), and its byte length (8 bytes) is recorded. The processor summarizes the starting position index of the same group of fields in all 1000 messages to generate a dataset containing "field group ID - message ID - starting position". If the starting position of a certain group of fields is found to be different in different messages (e.g., some messages have padding bytes causing offset), all variant positions are recorded in accordance with the facts to generate a field position aggregation group.

[0032] S312: Based on the field position aggregation group, calculate the difference of the starting offset value of each group of fields in all message instances, extract the span of the change range by calculating the difference between the maximum and minimum values ​​of the starting offset value, perform the corresponding operation on each field group, obtain the corresponding maximum offset value of the starting position, and establish a set of field offset amplitudes; Based on the field position aggregation group, the processor extracts all the starting offset values ​​of each field group (such as field group A) in 1000 message instances. Assuming that the offset of field group A is 3 in 900 messages and 4 in 100 messages, the processor calculates the difference between the maximum offset value (4) and the minimum offset value (3), that is... This difference represents the span of the change range. Perform this operation on all field groups (assuming there are M groups in total) to obtain M corresponding maximum offset values ​​of the starting position. For example, the offset of field group B is 0 (completely fixed), and the offset of field group C is 10 (drastic drift). Organize these offset values ​​into a list to establish a set of field offset values.

[0033] S313: Based on the set of field offset amplitudes, if the offset amplitude is less than the set structural consistency limit threshold, the corresponding field group is marked as a structurally stable field; otherwise, it is marked as a structurally drifting field. Statistical analysis is performed on the marking status of all field groups to obtain the maximum fluctuation data of structural offset. Based on the set of field offset amplitudes, the processor sets the structural consistency limit threshold to 0. This threshold is based on the characteristic of industrial control protocols (such as Profinet and EtherCAT) that require strict alignment of frame structures during the cyclic data exchange phase. Any non-zero offset is considered a structural drift. If the offset amplitude of a field group is equal to 0, it is marked as a "structurally stable field". If the offset amplitude is greater than 0, it is marked as a "structural drift field". The processor counts the number of fields marked as drift fields and their respective offsets in all field groups, finds the maximum offset (e.g., offset of 10 for field group C), and uses it as an indicator to measure the uncertainty of the protocol structure throughout the entire communication cycle, obtaining the maximum fluctuation data of structural offset.

[0034] Please see Figure 5 The specific steps of S4 are as follows: S411: Based on the maximum fluctuation data of structural offset, monitor the distribution of byte values ​​of each field segment in the periodic message within the communication cycle, extract the set of byte values ​​of each field segment in each cycle, count the number of occurrences of non-repeating byte values, summarize the change frequency of each field segment in the whole cycle, and generate a field byte change frequency matrix. Based on the maximum fluctuation data of structural offset, the key segments to be analyzed are determined. The processor monitors the byte value distribution of each field segment in the periodic messages within the communication cycle. For each field segment (e.g., field segment D, 2 bytes in length), its specific values ​​in 1000 periodic messages are extracted to form a set of values ​​of size 1000. The number of unique byte values ​​in this set is counted. For example, if field segment D only has two values ​​{0x0A, 0x0B} in 1000 samples, the unique value is 2. If field segment E has 1000 different values ​​(e.g., a counter), the unique value is 1000. The processor summarizes the change frequency data of all field segments and constructs a matrix with fields as rows and change frequency features as columns, generating a field byte change frequency matrix.

[0035] S412: Based on the field byte change frequency matrix, perform difference calculation on the maximum and minimum byte values ​​of each field segment in the periodic sequence. When the difference value is greater than the set periodic jump judgment threshold, it is marked as a periodic jump field segment, and the other segments are marked as periodic stable field segments, thus obtaining the field periodic jump mark sequence. Based on the field byte change frequency matrix, the processor performs extreme value analysis on the byte values ​​recorded in the periodic sequence for each field segment and extracts the maximum byte value. With the smallest byte value Calculate the difference value A threshold for judging periodic jumps is set. This threshold is obtained by measuring the noise range of sensor data under normal operating conditions. For example, the temperature sensor reading will not change by more than 5 degrees Celsius within a short period (corresponding to the numerical change), so the threshold is set to 50 (assuming range mapping). If the difference value of a certain field segment... If the value is greater than 50 and the field does not have the characteristics of an accumulator counter, it is determined that a non-continuous periodic jump has occurred and is marked as a periodic jump field segment. Otherwise, it is marked as a periodic stable field segment. Traverse all fields to obtain the field periodic jump marker sequence.

[0036] S413: Based on the field periodic transition marker sequence, retrieve the number of field segments identified as transition fields, perform a deduplication counting operation, output the total number of all field segments that meet the transition conditions in the current communication cycle, and obtain the set of periodic transition field segments; Based on the field periodic transition marker sequence, the processor traverses the marker sequence, retrieves the field segment index whose status is set to "transition", performs a deduplication counting operation, and if there are multiple consecutive transition markers in the sequence belonging to the same logical field, they are merged and counted. Finally, the total number of field segments that meet the abnormal transition conditions in the current communication cycle is output (for example, a total of 3 independent segments are detected to have transitioned). This number reflects the degree of non-stationarity in the current communication process, and the periodic transition field segments are obtained.

[0037] Please see Figure 6 The specific steps of S5 are as follows: S511: Based on the set of periodic jump field segments, obtain the content of the newly received message, extract the starting index position and mapping structure information of each field segment, compare the structure information with the periodic position distribution of the jump field at the field level, perform position calibration according to the field index alignment method, and establish a field position mapping matrix. Based on the set of periodic transition field segments, when a new frame of data is received, the processor no longer performs full blind parsing. Instead, it directly extracts the pre-determined starting index position and mapping structure information of each field segment. It compares the known transition field structure information with the content of the corresponding position in the new data. If the previous steps have determined three transition segments, the processor focuses on locking the position of these three segments in the current new data. Using the structural offset rules established in the previous steps, it dynamically aligns and adjusts the field index. For example, if the expected position offset is +1, it reads the data at index +1 to ensure accurate field reading. It then establishes the final two-dimensional correspondence between the aligned field index and the specific physical address, creating a field position mapping matrix.

[0038] S512: Based on the field position mapping matrix, map the structural attributes of each field segment in the current message to the periodic jump distribution of the corresponding field in the periodic message, construct the joint feature sequence of the current field segment in the structural position and periodic jump state, and calculate the overlap of the matching states between all field segments to obtain the field structure matching similarity vector. Based on the field position mapping matrix, the processor first extracts the values ​​defined in the matrix. The processor analyzes the structural attributes of key field segments, including the starting byte index, byte length, and data type marker for each field. It also retrieves periodic jump distribution data generated during the periodic message analysis phase. This data defines the legal numerical fluctuation range or discrete value set for each field within the historical communication cycle. The processor compares the actual parsed field values ​​in the currently received message with the corresponding periodic jump distribution range one by one, constructing a joint feature sequence that includes both position matching status and content value status. For example, for field A at index position 12, its structural attribute is defined as a 2-byte length, and historical distribution shows it as a constant value of 0x00FF. If the value at this position in the current message is 0x00FF, then both position and content match, and it is marked as a double match. For field B at index position 20, historical data shows it to be a dynamic counter, and its value range should be within... The value increases within the interval. If the current value is 50, it is considered to meet the jump rule and is marked as such. If the current value is 255, it is considered abnormal and marked as such. The processor traverses all Each field generates a result from [number of fields]. The joint feature sequence consists of tuples. The overlap of matching states in this sequence is then calculated. A weight coefficient is assigned to each field (core control words are weighted high, reserved bits are weighted low), and the weighted matching score is calculated. A practical example is introduced here: assuming the message contains 5 key fields, the weight vector is... The matching state vector of the current message after comparison is: (If the 4th field does not match), then the overlap is calculated as follows: The processor then vectorizes and encapsulates the overlap value and the matching results of each sub-item to generate a field structure matching similarity vector.

[0039] S513: Based on the field structure matching similarity vector, perform a structure equivalence measurement judgment on the current message and the historical periodic structure feature set, and output the identification label for the message structure type that meets the structure matching condition to obtain the protocol automatic identification result; Based on the field structure matching similarity vector, the processor will use this vector The structure is compared with the historical periodic structural feature sets of various standard industrial protocols pre-stored in the database to determine structural equivalence. A weighted Euclidean distance or cosine similarity algorithm is used to calculate the degree of fit between the current message structure and each standard template. Here, a weighted feature matching formula is used: (2); In the formula, This represents the final structural equivalence metric score, with a range of values. ; This represents the total number of feature fields. For the first The confidence weight of each feature field is determined based on the uniqueness of the field in the protocol; for example, the function code field has a higher weight than the data field. The first element in the current message similarity vector The value of each component (1 or 0); This is a feature validity indicator function; it is 1 if the feature exists in the database template, and 0 otherwise. For the target protocol template, the first The standard eigenvalues ​​at each position (usually 1); The term represents the root mean square error between the current vector and the template vector, used to penalize structural bias. The advantage of this formula is that the first part measures the weighted feature hit rate, while the second part, by subtracting the root mean square error, further reduces the scores of candidate protocols that, although hitting high-weight fields, have multiple minor deviations in the overall structure, thereby improving the specificity of the recognition.

[0040] Suppose we are identifying a packet that is suspected to be Modbus-TCP. How many feature fields are there? .

[0041] The set structure matching threshold is 0.85. This threshold was determined through backtesting analysis of 100,000 known protocol messages. At a threshold of 0.85, the false alarm rate is less than 0.1%.

[0042] The values ​​of each parameter and the calculation process are shown in Table 2: Table 2. Protocol Structure Feature Matching Calculation Parameters; The calculation process is as follows: Calculate the weighted sum of the denominators: ; Calculate the numerator-weighted hit value: ; Basic score in the first half: ; Calculate the root mean square error term: Sum of squared differences: ; Mean square value: ; Root mean square error: ; Correction factor: ; Calculate the final score : ; Comparison results: Calculation results The system determines that the current message does not conform to the structural characteristics of the protocol template.

[0043] The processor filters out protocol types that score above the structure matching threshold (such as "Modbus-TCPReadHoldingRegisters"), uses them as identification tags, outputs the final protocol classification information, and obtains the automatic protocol identification results.

[0044] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for automatic protocol identification in an Internet of Things (IoT) environment, characterized in that, Includes the following steps: S1: Obtain each communication protocol message collected by the communication interface of the IoT edge node, number it sequentially, identify the byte position index and repetition frequency of each byte in the message, perform difference calculation on the positions of adjacent bytes, and generate the field position repetition frequency difference value. S2: Based on the difference in the repetition frequency of the field positions, calculate the density value of consecutive byte positions in the IoT message, extract consecutive byte segments for byte position aggregation, and generate the number of consecutive aggregation groups for field partitions; S3: Based on the number of consecutive aggregation groups of the field partition, perform field structure matching on different message instances in the communication cycle, calculate the maximum starting position offset of each group of field segments, mark structurally stable fields and structurally drifting fields, and generate data on the maximum fluctuation of structural offset. S4: Based on the maximum fluctuation data of the structural offset, obtain the byte value change frequency of each field segment in the periodic message, calculate the difference between the maximum and minimum values ​​of the same field segment in each period, identify the periodic jump field segment and the periodic stable field segment, and generate a set of periodic jump field segments; S5: Based on the periodic transition field segment, compare the matching relationship of the currently newly received IoT messages and output the protocol automatic identification result.

2. The method for automatic protocol identification in an Internet of Things environment according to claim 1, characterized in that: The structural stability field specifically refers to the field segment whose offset amplitude is less than the set structural consistency limit threshold; The structural drift field specifically refers to the field segment where the offset amplitude is not less than the set structural consistency limit threshold; The periodic transition field segment specifically refers to the field segment where the difference value is greater than the preset periodic transition judgment threshold; The periodic stable field segment specifically refers to the field segment where the difference value is not greater than the preset periodic jump judgment threshold.

3. The method for automatic protocol identification in an Internet of Things environment according to claim 1, characterized in that: The field position repetition frequency difference includes the repetition frequency difference value and the difference calculation interval; the number of consecutive aggregation groups of the field partition includes the starting position of the continuous field segment, the length of the continuous field segment, and the gradient change direction feature; the maximum fluctuation data of structural offset includes the field starting position offset value, the field structure matching ratio, and the structural consistency evaluation index; the set of periodic jump field segments includes the field byte value change frequency, the field period range, and the periodic jump identification threshold; and the protocol automatic identification result includes field position mapping data, the matching relationship between structure and periodic characteristics, and field similarity judgment records.

4. The method for automatic protocol identification in an Internet of Things environment according to claim 1, characterized in that, The specific steps for obtaining the difference in the frequency of repetition at the field position are as follows: S111: Acquire each communication protocol message collected by the edge node communication interface, sequentially number each message data frame, extract the complete byte sequence, mark the position of each byte with a position index, establish a set of number and index relationship, and generate a byte position mapping sequence; S112: Based on the byte position mapping sequence, monitor the number of times the corresponding byte at each position index appears repeatedly in the message, perform aggregation and statistical operations on all byte values ​​at the same index position, calculate the total number of occurrences, establish the correspondence between index position and the number of occurrences, and generate an index position repetition frequency table; S113: Based on the index position repetition frequency table, perform adjacent position difference calculation on the repetition frequency values ​​between each index position to obtain the frequency difference set between adjacent interval positions, and perform position mapping reconstruction based on the numerical distribution of the frequency difference to obtain the field position repetition frequency difference.

5. The method for automatic protocol identification in an Internet of Things environment according to claim 1, characterized in that, The specific steps for obtaining the number of consecutive aggregation groups for the field partition are as follows: S211: Based on the difference in the repetition frequency of the field position, extract the continuous byte position index sequence in all messages, perform a ratio calculation on the cumulative number of adjacent bytes and the position span, obtain the number of bytes per unit length in each continuous position segment, and generate a continuous position density value group. S212: Based on the continuous position density value group, perform difference calculation on the density values ​​between adjacent position segments, calculate and obtain the interval density change metric value, filter the continuous segment intervals that are less than the preset density difference limit threshold, and obtain the low difference continuous density segment index set. S213: Based on the low-difference continuous density segment index set, perform directional consistency verification on the byte index sequence within the index interval, filter the interval sequences with the same density change direction, perform aggregation processing on the index position set that satisfies directional consistency, obtain the byte position set to which the field belongs in the continuous region, and establish the number of continuous aggregation groups for field partitioning.

6. The method for automatic protocol identification in an Internet of Things environment according to claim 5, characterized in that, The formula for calculating the interval density change metric is as follows: ； in, , Represents byte position index , The corresponding density value, Indicates the position within the interval density value, For interval to The average value of internal density, This is a measure of density variation within a given interval.

7. The method for automatic protocol identification in an Internet of Things environment according to claim 1, characterized in that, The specific steps for obtaining the maximum fluctuation data of the structural offset are as follows: S311: Based on the number of consecutive aggregation groups of the field partition, perform field structure matching on each message instance within the communication period, extract the starting position index and byte length of each group of fields in the corresponding message, aggregate the starting position indexes of the same group of fields recorded in different messages, and generate field position aggregation groups. S312: Based on the field position aggregation group, calculate the difference of the starting offset value of each group of fields in all message instances, extract the span of the change interval by calculating the difference between the maximum and minimum values ​​of the starting offset value, perform the corresponding operation on each field group, obtain the corresponding maximum offset value of the starting position, and establish a set of field offset amplitudes. S313: Based on the set of field offset amplitudes, if the offset amplitude is less than the set structural consistency limit threshold, the corresponding field group is marked as a structurally stable field; otherwise, it is marked as a structural drift field. Statistical analysis is performed on the marking status of all field groups to obtain the maximum fluctuation data of structural offset.

8. The method for automatic protocol identification in an Internet of Things environment according to claim 1, characterized in that, The specific steps for obtaining the periodic jump field segment are as follows: S411: Based on the maximum fluctuation data of the structural offset, monitor the byte value distribution of each field segment in the periodic message within the communication cycle, extract the byte value set of each field segment in each cycle, count the number of occurrences of non-repeating byte values, summarize the change frequency of each field segment in the whole cycle, and generate a field byte change frequency matrix. S412: Based on the field byte change frequency matrix, perform difference calculation on the maximum and minimum byte values ​​of each field segment in the periodic sequence. When the difference value is greater than the set periodic jump judgment threshold, it is marked as a periodic jump field segment, and the other segments are marked as periodic stable field segments, thus obtaining the field periodic jump mark sequence. S413: Based on the field periodic transition marker sequence, retrieve the number of field segments identified as transition fields, perform a deduplication counting operation, output the total number of all field segments that meet the transition conditions in the current communication cycle, and obtain the set of periodic transition field segments.

9. The method for automatic protocol identification in an Internet of Things environment according to claim 1, characterized in that, The specific steps for obtaining the automatic recognition result of the protocol are as follows: S511: Based on the set of periodic transition field segments, obtain the content of the newly received message, extract the starting index position and mapping structure information of each field segment, compare the structure information with the periodic position distribution of the transition field at the field level, perform position calibration according to the field index alignment method, and establish a field position mapping matrix. S512: Based on the field position mapping matrix, the structural attributes of each field segment in the current message are mapped one-to-one with the periodic jump distribution of the corresponding field in the periodic message, and a joint feature sequence of the current field segment in the structural position and periodic jump state is constructed. The overlap of matching states between all field segments is calculated to obtain the field structure matching similarity vector. S513: Based on the field structure matching similarity vector, perform a structure equivalence measurement judgment on the current message and the historical periodic structure feature set, and output the identification label for the message structure type that meets the structure matching condition to obtain the protocol automatic identification result.

Images