A data stream analysis and detection method and device for a gateway isolation component private protocol

Abstract

The application discloses a data stream analysis and detection method and device for a private protocol of a network gate isolation component, belongs to the technical field of network security, collects bidirectional original data streams in a communication process of the network gate isolation component, integrates the bidirectional original data streams into a structured set containing complete frame sequences, byte contents of each frame and transmission intervals between frames according to transmission time sequences; based on dynamic time window division, sessions are extracted, three types of behavior characteristics are extracted and compared with standard public protocol standard modes to obtain behavior similarity quantitative values; then, frame payloads are scanned byte by byte, network semantic feature fragments are identified, an associated network is constructed and a cross-frame semantic logic chain is formed; finally, the similarity quantitative values and the semantic chain completeness are comprehensively judged, when any index exceeds a preset standard, it is judged that the private protocol data stream has not completed complete protocol conversion, and a detection report containing specific abnormal behavior characteristics and illegal semantic association sequences is generated.

Description

Technical Field

[0001] This invention relates to the field of network security technology, specifically to a method and apparatus for parsing and detecting data streams of proprietary protocols in a network gateway isolation component. Background Technology

[0002] A network gateway (security isolation and information exchange system) is an important network boundary security device that achieves physical isolation and data exchange between different security domains through dedicated isolation components. This component typically uses a dedicated isolation chip or board to establish a unique and trusted physical channel between the two security domains. Its core mechanism lies in stripping public network protocols (such as TCP / IP) and encapsulating application layer data into proprietary security protocols for transmission, thereby achieving secure data transfer. This protocol conversion process is crucial to ensuring the security function of the network gateway, ensuring that no public protocol information that could pose a security risk is carried during communication.

[0003] However, current testing of dedicated isolation components for network gateways suffers from varying proprietary protocol encapsulation mechanisms employed by different manufacturers, coupled with a lack of unified interface specifications and communication standards. Existing technologies struggle to effectively parse and verify the actual communication content within these components. On one hand, it's impossible to confirm whether proprietary protocols have completely stripped away public network protocol stack information; on the other hand, the diversity of interface types makes it difficult to adapt testing equipment. Currently, there is no established testing method or device capable of in-depth data flow analysis, adapting to multiple interface types, and automatically identifying protocol remnants. This results in blind spots in the compliance testing and security assessment of network gateway products, making it difficult to meet the needs of high-level network security protection. Summary of the Invention

[0004] The purpose of this invention is to provide a method and apparatus for parsing and detecting the data stream of a private protocol of a network gateway isolation component, so as to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a method for parsing and detecting the data stream of a private protocol in a network gateway isolation component, comprising:

[0006] Step S1: Collect the bidirectional raw data stream during the communication process of the network gateway isolation component, and integrate it according to the transmission time sequence to form a structured data set containing the complete frame sequence, the byte content of each frame, and the inter-frame transmission interval;

[0007] Step S2: Divide the data stream into sessions based on a dynamic time window, and extract three types of behavioral features from each session, including the difference distribution and standard deviation of consecutive frame lengths, the variance and frequency distribution of frame interval values, the time difference range between request frames and corresponding response frames, and the correlation of their order of occurrence; compare these features with the standard behavioral patterns of standard public protocols item by item and calculate the overlap of distribution patterns to obtain a quantitative value of the behavioral similarity between the current session and the public protocol; the standard public protocol is a pre-stored protocol template that conforms to public network specifications;

[0008] Step S3: Scan the payload content of all frames in the session byte by byte to identify network semantic feature fragments, and construct an association network of feature fragments with session time as the axis. Network nodes are feature fragments, and the connections between nodes mark the time sequence relationship. By statistically analyzing the recurrence frequency of the same feature fragment and the order dependency relationship of different feature fragments, construct a semantic logic chain across frames.

[0009] Step S4: Determine the integrity of the comprehensive behavioral similarity quantification value and semantic logic chain. If any indicator exceeds the preset standard, it is determined that the private protocol data stream has not completed a complete protocol conversion, and a detection report containing specific abnormal behavior characteristics and illegal semantic association sequences is generated.

[0010] Furthermore, step S1 includes:

[0011] Step S1-1: Connect the TX transmitter and RX receiver of the network gateway isolation component through the physical interface, synchronously capture the original electrical signal of bidirectional transmission, convert the electrical signal into a binary byte sequence, and record the transmission timestamp of each byte; continuously monitor the captured byte sequence, identify the frame start mark and frame end mark, remove invalid padding bytes between the marks, extract the complete frame data, and generate the original data sequence with timestamps.

[0012] Step S1-2: Arrange the extracted complete frames sequentially according to the transmission timestamps to form a frame sequence; associate each frame with all the bytes it contains to form a correspondence between frames and byte contents; calculate the timestamp difference between two adjacent complete frames as the transmission interval value between the two frames; integrate the frame sequence, the byte contents of each frame, and the inter-frame transmission interval value into a unified structured data set, in which each record contains the frame sequence number, the corresponding byte contents, and the transmission interval with the previous frame.

[0013] Furthermore, step S2 includes:

[0014] Step S2-1: First, perform statistical analysis on the first g inter-frame transmission intervals in the structured data set, calculate the historical frame interval mean μ (the sum of all historical frame interval values ​​divided by the number of historical frame intervals) and the historical frame interval standard deviation σ (the square root of the average of the sum of squares of the differences between each historical frame interval and μ), set the silence threshold θ to μ+2σ, and establish the session boundary.

[0015] The transmission interval ΔTi of consecutive frames in the monitoring data stream is used. When the interval ΔTi of a certain frame is greater than the silent threshold θ, the frame sequence before and after the interval is divided into two independent session units. The preceding frame sequence is marked as a session, and the subsequent frame sequence is used as the start of a new session. All frame sequences are traversed according to this rule to complete the division of all sessions and assign a unique number to each session.

[0016] For each segmented conversation unit, a three-dimensional behavioral feature vector F=[f1,f2,f3] is extracted, where:

[0017] f1 is the coefficient of variation (CV) of the frame length sequence, calculated as the standard deviation of the frame length (σ). l (The square root of the average of the sum of squares of the differences between all frame lengths and the mean frame length within the session) and the mean frame length μ l The ratio of (the sum of all frame lengths in a session divided by the number of frames), i.e., CV = σ l / μ l ;

[0018] f2 is the Shannon entropy H of the frame interval, which is calculated by dividing the values ​​of all frame intervals in the session into intervals, calculating the probability p(x) of the occurrence of a frame interval in each interval (the ratio of the number of frame intervals in that interval to the total number of frame intervals in the session), and then applying the formula H=-Σ[p(x)×log p (x)] is calculated;

[0019] f3 is the Pearson correlation coefficient ρ for request-response delay. It is calculated by identifying the transmission time difference between the request frame and the corresponding response frame within the session (response frame timestamp minus request frame timestamp) and the linear correlation between the time difference sequence and the request frame sending order sequence. The value range is [-1, 1], and the closer the value is to 1, the stronger the correlation.

[0020] Step S2-2: For each session, extract the length sequence L={l1,l2,...,l} of consecutive frames in the session. n}, where l1, l2, l n These represent the lengths of frame 1, frame 2, and frame n, respectively, where n is the total number of frames in the session; The pre-stored standard common protocol standard frame length sequence L is invoked. s L s Includes the frame length distribution characteristics of this public protocol in standard scenarios; calculates L and Ls The dynamic time warped distance DTW(L,L) between s ): Constructing L and L s The frame length distance matrix, where the element in the u-th row and j-th column is the distance between the u-th frame and L. s The sum of the squared differences in frame length of the j-th frame is used; a recursive cumulative distance relation is defined, where the cumulative distance at the current position (u,j) is equal to the squared difference in frame length at that position plus the minimum cumulative distance among the positions at its top left, top, or left; finally, backtracking from the end of the two sequences to the beginning of the two sequences, the backtracking path is the optimal alignment path that minimizes the cumulative distance, and the sum of the squared differences in frame length along this path is accumulated to obtain DTW(L,L). s Simultaneously calculate the mean μ of the current session frame length sequence. l and standard deviation σ l The corresponding parameter μ of the standard public protocol s σ s Calculate the relative deviation δ l δ l =|μ l -μ s | / μ s +|σ l -σ s | / σ s ;

[0021] Step S2-3: Extract the transmission intervals of all adjacent frames in the current session, and divide the interval range into k equal-width intervals, where k is a preset positive integer based on the interval value range; count the number of times the interval appears in each interval, and the ratio of this number to the total number of intervals is the probability density p of that interval. i , i=1,2……k, all p i The frame interval distribution P = {p1, p2, ..., p} that constitutes the current session k};

[0022] The frame interval distribution of the standard common protocol is Q={q1,q2,...,q}. k}, where q k Let represent the probability density of the k-th interval in the standard common protocol frame interval distribution; calculate the divergence JSD(P||Q) between P and Q. The calculation process is as follows: first, obtain the average distribution M of P and Q, where the probability density of each interval in M ​​is: (probability density of the corresponding interval of P + probability density of the corresponding interval of Q) / 2; then calculate the KL divergence D(P||M) of P with respect to M and the KL divergence D(Q||M) of Q with respect to M respectively; the KL divergence is expressed by the formula Σ[p i ×ln(p i / m i )]Calculate, where m iLet JSD be the probability density of the i-th interval in M; finally, the divergence value is calculated as: JSD(P||Q)=[D(P||M)+D(Q||M)] / 2. The smaller this value is, the closer the current session is to the frame interval distribution pattern of the standard common protocol.

[0023] Step S2-4: Based on the physical transmission direction of the frame and the transaction identifier field in the payload, identify the request frame sequence R={r1,r2,...,r...} in the session. e} and the corresponding response frame sequence A={a1,a2,...,a e}; where r1, r2, r e These represent the timestamps of the 1st, 2nd, and eth request frames, respectively, where e is the total number of request and response pairs in the session; a1, a2, ..., a e These represent the timestamps of the response frames corresponding to each request frame in R;

[0024] Construct the request frame sequence number sequence X={1,2,...,e} and the delay sequence Y={d1,d2,...,d} e}, where d e =a e -r e Calculate the Pearson correlation coefficient ρ between the two. XY This is used to quantify the linear correlation between the order of request sending and the corresponding latency; it extracts the correlation coefficient ρ between the request-response latency and the sequence number of standard public protocols. s Through the formula δ ρ =|ρ XY -ρ s |Calculate the pattern difference δρ to quantify the difference between the two in their association patterns;

[0025] Weights w1, w2, and w3 are assigned to three dimensions, with w1 + w2 + w3 = 1. The weight allocation is based on the importance of the core behavioral characteristics of the common protocol and is configured differently according to the protocol type. A preset maximum normalized distance factor, maxDTW, is introduced, based on the maximum DTW distance obtained from a large number of samples, and is used to normalize DTW(L,L) distances. s Normalize to the [0,1] interval;

[0026] The weights are allocated based on the importance of the core behavioral characteristics of common protocols, and are differentiated according to protocol type (connection-oriented / connectionless). For connection-oriented protocols such as TCP, the frame length sequence shape (w1) directly reflects its segmentation and reassembly rules, and the request-response association pattern (w3) reflects its confirmation mechanism. Both are more identifiable of protocol behavior, so w1 and w3 are assigned higher weights (e.g., w1=0.4, w3=0.3). The frame interval distribution (w2) reflects the influence of the network environment more and is relatively less identifiable, so it is assigned a lower weight (e.g., w2=0.3). For connectionless protocols such as UDP, the randomness of the frame interval distribution (w2) is its core characteristic, so the weight of w2 is increased (e.g., w2=0.4), and w1 and w3 are adjusted accordingly (e.g., w1=0.3, w3=0.3). The total weight is always kept at 1.

[0027] Collect at least 1000 known sample data streams conforming to the standard common protocol standard, covering different network loads, device models, and communication scenarios. Extract the frame length sequence L from each sample and calculate its relationship with the frame length sequence L of the standard common protocol. s The DTW distance is calculated; the numerical distribution of the DTW distance of all samples is statistically analyzed, and the 99.9th percentile value is taken as the maxDTW. This value can cover the frame length sequence fluctuations in almost all normal communication scenarios, ensuring that the actual calculated DTW(L,Ls) falls within the [0,1] interval after normalization, and avoiding interference from extreme abnormal samples on the normalization results.

[0028] By weighted fusion of the difference indicators from the above three dimensions, a comprehensive behavioral similarity quantification value is generated: S=w1×[1-DTW(L,L)] s ) / maxDTW]+w2×(1-JSD)+w3×(1-δ ρ The closer the similarity quantification value S is to 1, the more similar the current session is to the behavior pattern of the public protocol.

[0029] Furthermore, step S3 includes:

[0030] Step S3-1: Perform a byte-by-byte sliding scan on the payload content of all frames in the session, and identify feature fragments based on a predefined network semantic feature rule base, including continuous byte sequences that conform to the standard network address format, numeric fields that conform to the value range of the transmission service port, and feature byte combinations that match public protocol control instructions; record the frame number, byte offset within the frame, and feature type for each identified feature fragment;

[0031] Step S3-2: Construct an association network of feature segments based on the session timeline. Each node in the network corresponds to a feature segment, and the node attributes include segment content, type, and appearance time; the nodes are connected by directed lines, and the line direction represents the chronological relationship of segment appearances, and the line weight is the ratio of the interval between the appearances of two segments to the average frame interval of the session; calculate three types of association parameters by traversing the network: the ratio of the total number of times the same feature segment appears repeatedly in different frames to the total number of frames in the session, the sequence length of different types of feature segments that appear continuously in a fixed order, and the number of links formed by feature segments distributed across frames to constitute a complete protocol interaction logic; construct a cross-frame semantic logic chain based on these parameters, and the integrity of the chain is the ratio of the number of types of feature segments actually included to the number of types included in the corresponding common protocol standard semantic chain.

[0032] Further, step S4 includes:

[0033] Step S4-1: Set the behavior similarity determination threshold T1 and the semantic logic chain integrity standard T2. Among them, T1 is determined based on the similarity quantization value distribution of the standard common protocol samples, and the lowest similarity value of normal protocol interactions in the sample set is taken; T2 is the ratio threshold of the number of types of feature segments that the corresponding common protocol standard semantic chain should include, that is, when the number of feature types actually included in the semantic chain reaches 90% or more of the standard type number, it is determined to meet the integrity requirement;

[0034] Step S4-2: Compare the comprehensive behavior similarity quantization value S calculated in step S2 with the threshold T1, and at the same time compare the integrity of the semantic logic chain obtained in step S3 with the standard T2; if S < T1, it is determined that there is a significant difference between the current session behavior pattern and the common protocol; if the integrity of the semantic logic chain < T2, it is determined that there is a lack or abnormality in the cross-frame semantic association; when any one of the determination results is true, output the conclusion that the private protocol data stream has not completed a complete protocol conversion, generate a detection report containing specific abnormal items, and in the report, details are recorded about the feature dimensions with behavior similarity lower than the threshold, the types of feature segments missing in the semantic chain, and the abnormal association order, and the corresponding frame numbers and timestamps are marked.

[0035] A data stream parsing and detection device for the private protocol of a network isolation component, including a data acquisition module, a behavior analysis module, a semantic parsing module, a judgment output module, a storage module, and a human-computer interaction module;

[0036] The data acquisition module connects to the communication interfaces of different models of network isolation components through a configurable physical interface adapter, captures the original signals during the communication process, and converts them into a structured data set including frame sequences, byte contents, and transmission intervals;

[0037] The behavior analysis module divides the data stream into sessions, extracts frame length variation features, temporal distribution features, and interaction pattern features, and calculates the behavior similarity quantification value by comparing it with the standard public protocol behavior patterns pre-stored in the storage module.

[0038] The semantic parsing module identifies feature segments of the frame payload content, constructs a time-axis-based association network, and generates cross-frame semantic logic chains by analyzing the spatiotemporal relationships between feature segments.

[0039] The judgment output module generates a protocol compliance detection report based on the determination results of the behavioral similarity quantification value and the completeness of the semantic logic chain;

[0040] The storage module is used to store the interface protocol rule base, the standard public protocol behavior feature base, the network semantic feature rule base, and various types of data during the parsing process;

[0041] The human-computer interaction module provides functions for configuring detection parameters, visualizing the detection process, and outputting detection reports.

[0042] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0043] This invention addresses the issues of incomplete behavior matching and missing semantic analysis in existing private protocol detection methods for network gateways by employing a dual-dimensional detection scheme that combines behavioral feature quantification and semantic logic association. First, it synchronously captures bidirectional raw data streams and integrates them into a structured set containing frame sequences, byte content, and transmission intervals, avoiding feature extraction biases caused by data fragmentation. Second, it divides sessions based on dynamic time windows, extracting three types of features: frame length difference distribution, frame interval variance, and request-response temporal correlation. Through quantification and comparison with standard public protocol patterns, it overcomes the limitations of traditional single-feature detection, improving the accuracy of behavior recognition. Next, it scans frame payloads byte-by-byte to construct a semantic association network, clarifying cross-frame semantic logic chains and filling gaps in semantic layer analysis. Finally, it comprehensively judges behavioral similarity and semantic chain completeness to ensure coverage of key dimensions of protocol behavior and semantics. Ultimately, it accurately identifies incompletely converted public protocol remnants, providing reliable protection for network gateway isolation security and reducing network security risks. Attached Figure Description

[0044] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0045] Figure 1 This is a flowchart of a method and device for parsing and detecting the data stream of a private protocol of a network gateway isolation component. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Please see Figure 1 This invention provides a technical solution: a method for parsing and detecting the data stream of a private protocol in a network gateway isolation component, comprising:

[0048] Step S1: Collect the bidirectional raw data stream during the communication process of the network gateway isolation component, and integrate it according to the transmission time sequence to form a structured data set containing the complete frame sequence, the byte content of each frame, and the inter-frame transmission interval;

[0049] Step S2: Divide the data stream into sessions based on a dynamic time window, and extract three types of behavioral features from each session, including the difference distribution and standard deviation of consecutive frame lengths, the variance and frequency distribution of frame interval values, the time difference range between request frames and corresponding response frames, and the correlation of their order of occurrence; compare these features with the standard behavioral patterns of standard public protocols item by item and calculate the overlap of distribution patterns to obtain a quantitative value of the behavioral similarity between the current session and the public protocol; the standard public protocol is a pre-stored protocol template that conforms to public network specifications;

[0050] Step S3: Scan the payload content of all frames in the session byte by byte to identify network semantic feature fragments, and construct an association network of feature fragments with session time as the axis. Network nodes are feature fragments, and the connections between nodes mark the time sequence relationship. By statistically analyzing the recurrence frequency of the same feature fragment and the order dependency relationship of different feature fragments, construct a semantic logic chain across frames.

[0051] Step S4: Determine the integrity of the comprehensive behavioral similarity quantification value and semantic logic chain. If any indicator exceeds the preset standard, it is determined that the private protocol data stream has not completed a complete protocol conversion, and a detection report containing specific abnormal behavior characteristics and illegal semantic association sequences is generated.

[0052] Step S1 includes:

[0053] Step S1-1: Connect the TX transmitter and RX receiver of the network gateway isolation component through the physical interface, synchronously capture the original electrical signal of bidirectional transmission, convert the electrical signal into a binary byte sequence, and record the transmission timestamp of each byte; continuously monitor the captured byte sequence, identify the frame start mark and frame end mark, remove invalid padding bytes between the marks, extract the complete frame data, and generate the original data sequence with timestamps.

[0054] Step S1-2: Arrange the extracted complete frames sequentially according to the transmission timestamps to form a frame sequence; associate each frame with all the bytes it contains to form a correspondence between frames and byte contents; calculate the timestamp difference between two adjacent complete frames as the transmission interval value between the two frames; integrate the frame sequence, the byte contents of each frame, and the inter-frame transmission interval value into a unified structured data set, in which each record contains the frame sequence number, the corresponding byte contents, and the transmission interval with the previous frame.

[0055] Example 1: Connect the TX transmitter and RX receiver of the network gateway through the RJ45 physical interface to synchronously capture bidirectional raw electrical signals, convert them into binary byte sequences and record the timestamp of each byte; monitor the byte sequence, identify the frame start and end markers, remove invalid padding bytes in the middle, and extract 200 complete frames; sort the frames by timestamp to form a frame sequence, associate the byte content of each frame, calculate the timestamp difference between adjacent frames, for example, the interval between frames 1 and 2 is 12ms, the interval between frames 2 and 3 is 8ms, and integrate them into a structured set of frame sequence number + byte content + transmission interval.

[0056] Step S2 includes:

[0057] Step S2-1: First, perform statistical analysis on the first g inter-frame transmission intervals in the structured data set, calculate the historical frame interval mean μ (the sum of all historical frame interval values ​​divided by the number of historical frame intervals) and the historical frame interval standard deviation σ (the square root of the average of the sum of squares of the differences between each historical frame interval and μ), set the silence threshold θ to μ+2σ, and establish the session boundary.

[0058] The transmission interval ΔTi of consecutive frames in the monitoring data stream is used. When the interval ΔTi of a certain frame is greater than the silent threshold θ, the frame sequence before and after the interval is divided into two independent session units. The preceding frame sequence is marked as a session, and the subsequent frame sequence is used as the start of a new session. All frame sequences are traversed according to this rule to complete the division of all sessions and assign a unique number to each session.

[0059] For each segmented conversation unit, a three-dimensional behavioral feature vector F=[f1,f2,f3] is extracted, where:

[0060] f1 is the coefficient of variation (CV) of the frame length sequence, calculated as the standard deviation of the frame length (σ). l (The square root of the average of the sum of squares of the differences between all frame lengths and the mean frame length within the session) and the mean frame length μ l The ratio of (the sum of all frame lengths in a session divided by the number of frames), i.e., CV = σ l / μ l ;

[0061] f2 is the Shannon entropy H of the frame interval, which is calculated by dividing the values ​​of all frame intervals in the session into intervals, calculating the probability p(x) of the occurrence of a frame interval in each interval (the ratio of the number of frame intervals in that interval to the total number of frame intervals in the session), and then applying the formula H=-Σ[p(x)×log p (x)] is calculated;

[0062] f3 is the Pearson correlation coefficient ρ for request-response delay. It is calculated by identifying the transmission time difference between the request frame and the corresponding response frame within the session (response frame timestamp minus request frame timestamp) and the linear correlation between the time difference sequence and the request frame sending order sequence. The value range is [-1, 1], and the closer the value is to 1, the stronger the correlation.

[0063] Step S2-2: For each session, extract the length sequence L={l1,l2,...,l} of consecutive frames in the session. n}, where l1, l2, l n These represent the lengths of frame 1, frame 2, and frame n, respectively, where n is the total number of frames in the session; The pre-stored standard common protocol standard frame length sequence L is invoked. s L s Includes the frame length distribution characteristics of this public protocol in standard scenarios; calculates L and L s The dynamic time warped distance DTW(L,L) between s ): Constructing L and L s The frame length distance matrix, where the element in the u-th row and j-th column is the distance between the u-th frame and L. s The sum of the squared differences in frame length of the j-th frame is used; a recursive cumulative distance relation is defined, where the cumulative distance at the current position (u,j) is equal to the squared difference in frame length at that position plus the minimum cumulative distance among the positions at its top left, top, or left; finally, backtracking from the end of the two sequences to the beginning of the two sequences, the backtracking path is the optimal alignment path that minimizes the cumulative distance, and the sum of the squared differences in frame length along this path is accumulated to obtain DTW(L,L). s Simultaneously calculate the mean μ of the current session frame length sequence. l and standard deviation σ l The corresponding parameter μ of the standard public protocol s σ s Calculate the relative deviation δ l δ l =|μ l -μ s | / μ s +|σ l -σ s | / σ s ;

[0064] Step S2-3: Extract the transmission intervals of all adjacent frames in the current session, and divide the interval range into k equal-width intervals, where k is a preset positive integer based on the interval value range; count the number of times the interval appears in each interval, and the ratio of this number to the total number of intervals is the probability density p of that interval. i , i=1,2……k, all p i The frame interval distribution P = {p1, p2, ..., p} that constitutes the current session k};

[0065] The frame interval distribution of the standard common protocol is Q={q1,q2,...,q}. k}, where q k Let represent the probability density of the k-th interval in the standard common protocol frame interval distribution; calculate the divergence JSD(P||Q) between P and Q. The calculation process is as follows: first, obtain the average distribution M of P and Q, where the probability density of each interval in M ​​is: (probability density of the corresponding interval of P + probability density of the corresponding interval of Q) / 2; then calculate the KL divergence D(P||M) of P with respect to M and the KL divergence D(Q||M) of Q with respect to M respectively; the KL divergence is expressed by the formula Σ[p i ×ln(p i / m i )]Calculate, where m i Let JSD be the probability density of the i-th interval in M; finally, the divergence value is calculated as: JSD(P||Q)=[D(P||M)+D(Q||M)] / 2. The smaller this value is, the closer the current session is to the frame interval distribution pattern of the standard common protocol.

[0066] Step S2-4: Based on the physical transmission direction of the frame and the transaction identifier field in the payload, identify the request frame sequence R={r1,r2,...,r...} in the session. e} and the corresponding response frame sequence A={a1,a2,...,a e}; where r1, r2, r e These represent the timestamps of the 1st, 2nd, and eth request frames, respectively, where e is the total number of request and response pairs in the session; a1, a2, ..., a e These represent the timestamps of the response frames corresponding to each request frame in R;

[0067] Construct the request frame sequence number sequence X={1,2,...,e} and the delay sequence Y={d1,d2,...,d} e}, where d e =a e -r e Calculate the Pearson correlation coefficient ρ between the two. XYThis is used to quantify the linear correlation between the order of request sending and the corresponding latency; it extracts the correlation coefficient ρ between the request-response latency and the sequence number of standard public protocols. s Through the formula δ ρ =|ρ XY -ρ s |Calculate the pattern difference δρ to quantify the difference between the two in their association patterns;

[0068] Weights w1, w2, and w3 are assigned to three dimensions, with w1 + w2 + w3 = 1. The weight allocation is based on the importance of the core behavioral characteristics of the common protocol and is configured differently according to the protocol type. A preset maximum normalized distance factor, maxDTW, is introduced, based on the maximum DTW distance obtained from a large number of samples, and is used to normalize DTW(L,L) distances. s Normalize to the [0,1] interval;

[0069] The weights are allocated based on the importance of the core behavioral characteristics of common protocols, and are differentiated according to protocol type (connection-oriented / connectionless). For connection-oriented protocols such as TCP, the frame length sequence shape (w1) directly reflects its segmentation and reassembly rules, and the request-response association pattern (w3) reflects its confirmation mechanism. Both are more identifiable of protocol behavior, so w1 and w3 are assigned higher weights (e.g., w1=0.4, w3=0.3). The frame interval distribution (w2) reflects the influence of the network environment more and is relatively less identifiable, so it is assigned a lower weight (e.g., w2=0.3). For connectionless protocols such as UDP, the randomness of the frame interval distribution (w2) is its core characteristic, so the weight of w2 is increased (e.g., w2=0.4), and w1 and w3 are adjusted accordingly (e.g., w1=0.3, w3=0.3). The total weight is always kept at 1.

[0070] Collect at least 1000 known sample data streams conforming to the standard common protocol standard, covering different network loads, device models, and communication scenarios. Extract the frame length sequence L from each sample and calculate its relationship with the frame length sequence L of the standard common protocol. s The DTW distance is calculated; the numerical distribution of the DTW distance of all samples is statistically analyzed, and the 99.9th percentile value is taken as the maxDTW. This value can cover the frame length sequence fluctuations in almost all normal communication scenarios, ensuring that the actual calculated DTW(L,Ls) falls within the [0,1] interval after normalization, and avoiding interference from extreme abnormal samples on the normalization results.

[0071] By weighted fusion of the difference indicators from the above three dimensions, a comprehensive behavioral similarity quantification value is generated: S=w1×[1-DTW(L,L)] s ) / maxDTW]+w2×(1-JSD)+w3×(1-δ ρThe closer the similarity quantification value S is to 1, the more similar the current session is to the behavior pattern of the public protocol.

[0072] Example 2: Statistical analysis of the first 20 frame intervals yielded a mean μ = 10ms and a standard deviation σ = 3ms. A silence threshold θ = 16ms was set. Detection showed that the interval between frames 50 and 51 was 20ms > 16ms, dividing the session into two segments: frames 1-50 and frames 51-200. For the first segment: the frame length sequence L was extracted and compared with the pre-stored TCP standard frame length sequence L. s DTW distance = 80, maxDTW is taken from 1000 TCP samples, 99.9% quantile = 200, after normalization, the frame length dimension score is 0.6; the frame interval is divided into 4 intervals, and JSD = 0.15 is calculated, corresponding to a score of 0.85; 30 request-response frame pairs are identified, and the correlation coefficient ρ between the delay sequence and the sequence number is calculated. XY =0.92, compared to the TCP standard ρ s =0.95 δ ρ =0.03, score 0.97; weighted by w1=0.4, w2=0.3, w3=0.3, the behavioral similarity quantification value is S=0.4×0.6+0.3×0.85+0.3×0.97=0.78.

[0073] Step S3 includes:

[0074] Step S3-1: Perform a byte-by-byte sliding scan on the payload content of all frames in the session, and identify feature fragments based on a predefined network semantic feature rule base, including continuous byte sequences that conform to the standard network address format, numeric fields that conform to the value range of the transmission service port, and feature byte combinations that match public protocol control instructions; record the frame number, byte offset within the frame, and feature type for each identified feature fragment;

[0075] Step S3-2: Construct an association network of feature fragments based on the session timeline. Each node in the network corresponds to a feature fragment, and the node attributes include fragment content, type, and occurrence time. Nodes are connected by directed lines, with the direction of the line indicating the chronological order of fragment occurrences. The weight of the line is the ratio of the interval between two fragment occurrences to the average frame interval of the session. Three types of association parameters are calculated by traversing the network: the ratio of the total number of times the same feature fragment appears repeatedly in different frames to the total number of frames in the session, the sequence length of different types of feature fragments appearing consecutively in a fixed order, and the number of links in the complete protocol interaction logic formed by feature fragments distributed across frames. Based on these parameters, construct a cross-frame semantic logic chain. The completeness of the chain is the ratio of the number of feature fragment types actually included to the number of types included in the corresponding public protocol standard semantic chain.

[0076] Embodiment 3: Scan the frame payload byte by byte to identify the network layer address, transport layer port, and application layer instruction; construct an association network based on the time axis, and label the node with the segment type and time. For example, the 10th frame "port 0x0050" → the 11th frame "address 0xC0A80101" → the 12th frame "instruction 0x02" form a link; it is statistically obtained that the semantic chain contains 3 types of TCP feature segments, and the integrity = 3 / 3 = 1.0.

[0077] Step S4 includes:

[0078] Step S4-1: Set the behavior similarity determination threshold T1 and the semantic logic chain integrity standard T2. Among them, T1 is determined based on the similarity quantization value distribution of the standard common protocol samples, and the lowest similarity value of the normal protocol interaction in the sample set is taken; T2 is the proportion threshold of the number of feature segment types that the corresponding common protocol standard semantic chain should contain, that is, when the number of feature types actually contained in the semantic chain reaches 90% or more of the standard type number, it is determined to meet the integrity requirement;

[0079] Step S4-2: Compare the comprehensive behavior similarity quantization value S calculated in step S2 with the threshold T1, and at the same time compare the integrity of the semantic logic chain obtained in step S3 with the standard T2; if S < T1, it is determined that the current session behavior mode has a significant difference from the common protocol; if the integrity of the semantic logic chain < T2, it is determined that there is a lack or abnormality in the cross-frame semantic association; when any one of the determination results is true, output the conclusion that the private protocol data stream has not completed a complete protocol conversion, generate a detection report containing specific abnormal items, and in the report, details such as the feature dimension with behavior similarity lower than the threshold, the type of feature segments missing in the semantic chain, and the abnormal association order are recorded, and the corresponding frame numbers and timestamps are marked.

[0080] Embodiment 4: Set the behavior similarity threshold T1 = 0.8 and the semantic integrity threshold T2 = 0.9; because S = 0.78 < T1, it is determined that there is a TCP protocol residue in this private protocol, and a detection report is generated: mark that the behavior similarity does not meet the standard (the score of the frame length dimension is low), list the abnormal frame numbers 1-50, and record the illegal semantic sequence "port 0x0050 → address 0xC0A80101 → instruction 0x02".

[0081] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, in any aspect, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.

Claims

1. A method for parsing and detecting the data stream of a private protocol in a network gateway isolation component, characterized in that: The method includes: Step S1: Collect the bidirectional raw data stream during the communication process of the network gateway isolation component, and integrate it according to the transmission time sequence to form a structured data set containing the complete frame sequence, the byte content of each frame, and the inter-frame transmission interval; Step S2: Divide the data stream into sessions based on a dynamic time window, and extract three types of behavioral features from each session, including the difference distribution and standard deviation of consecutive frame lengths, the variance and frequency distribution of frame interval values, the time difference range between request frames and corresponding response frames, and the correlation of their order of occurrence; compare these features with the standard behavioral patterns of standard public protocols item by item and calculate the overlap of distribution patterns to obtain a quantitative value of the behavioral similarity between the current session and the public protocol; the standard public protocol is a pre-stored protocol template that conforms to public network specifications; Step S3: Scan the payload content of all frames in the session byte by byte to identify network semantic feature fragments, and construct an association network of feature fragments with session time as the axis. Network nodes are feature fragments, and the connections between nodes mark the time sequence relationship. By statistically analyzing the recurrence frequency of the same feature fragment and the order dependency relationship of different feature fragments, construct a semantic logic chain across frames. Step S4: Determine the integrity of the comprehensive behavioral similarity quantification value and semantic logic chain. If any indicator exceeds the preset standard, it is determined that the private protocol data stream has not completed a complete protocol conversion, and a detection report containing specific abnormal behavior characteristics and illegal semantic association sequences is generated.

2. The method for parsing and detecting the data stream of a private protocol in a network gateway isolation component according to claim 1, characterized in that: Step S1 includes: Step S1-1: Connect the TX transmitter and RX receiver of the network gateway isolation component through the physical interface, synchronously capture the original electrical signal of bidirectional transmission, convert the electrical signal into a binary byte sequence, and record the transmission timestamp of each byte; continuously monitor the captured byte sequence, identify the frame start mark and frame end mark, remove invalid padding bytes between the marks, extract the complete frame data, and generate the original data sequence with timestamps. Step S1-2: Arrange the extracted complete frames sequentially according to the transmission timestamps to form a frame sequence; associate each frame with all the bytes it contains to form a correspondence between frames and byte contents; calculate the timestamp difference between two adjacent complete frames as the transmission interval value between the two frames; integrate the frame sequence, the byte contents of each frame, and the inter-frame transmission interval value into a unified structured data set, in which each record contains the frame sequence number, the corresponding byte contents, and the transmission interval with the previous frame.

3. The data stream parsing and detection method for a private protocol of a network gateway isolation component according to claim 1, characterized in that: Step S2 includes: Step S2-1: First, perform statistical analysis on the first g inter-frame transmission intervals in the structured data set, and calculate the historical frame interval mean μ = the sum of all historical frame interval values ​​divided by the number of historical frame intervals and the historical frame interval standard deviation σ = the square root of the average of the sum of squares of the differences between each historical frame interval and μ. Establish the session boundary by setting a silent threshold θ: θ = μ + 2σ. The transmission interval ΔT of consecutive frames in the monitoring data stream is used. When the interval ΔT of a certain frame is greater than the silence threshold θ, the frame sequence before and after the interval is divided into two independent session units. The preceding frame sequence is marked as a session, and the subsequent frame sequence is used as the start of a new session. All frame sequences are traversed according to this rule to complete the division of all sessions and assign a unique number to each session. For each segmented conversation unit, a three-dimensional behavioral feature vector F=[f1,f2,f3] is extracted, where: f1 is the coefficient of variation (CV) of the frame length sequence, calculated as the standard deviation of the frame length (σ). l With frame length mean μ l The ratio, i.e., CV = σ l / μ l ; f2 is the Shannon entropy H of the frame interval. It is calculated by dividing the values ​​of all frame intervals within a session into intervals, and calculating the probability of a frame interval occurring within each interval, p(x) = the ratio of the number of frame intervals within that interval to the total number of frame intervals in the session. Then, the probability is expressed using the formula H = -Σ[p(x) × log0]. p (x)] is calculated; f3 is the Pearson correlation coefficient ρ for request-response delay. By identifying the transmission time difference between the request frame and the corresponding response frame within the session, the linear correlation between this time difference sequence and the request frame transmission order sequence is calculated.

4. The data stream parsing and detection method for a private protocol of a network gateway isolation component according to claim 3, characterized in that: Step S2 further includes: Step S2-2: For each session, extract the length sequence L={l1,l2,...,l} of consecutive frames in the session. n }, where l1, l2, l n These represent the lengths of frame 1, frame 2, and frame n, respectively, where n is the total number of frames in the session; The pre-stored standard common protocol standard frame length sequence L is invoked. s L s Includes the frame length distribution characteristics of this public protocol in standard scenarios; calculates L and L s The dynamic time warped distance DTW(L,L) between s ): Constructing L and L s The frame length distance matrix, where the element in the u-th row and j-th column of the matrix is ​​the distance between the u-th frame length and L. s The sum of the squared differences in frame length of the j-th frame is used; a recursive cumulative distance relation is defined, where the cumulative distance at the current position (u,j) is equal to the squared difference in frame length at that position plus the minimum cumulative distance among the positions at its top left, top, or left; finally, backtracking from the end of the two sequences to the beginning of the two sequences, the backtracking path is the optimal alignment path that minimizes the cumulative distance, and the sum of the squared differences in frame length along this path is accumulated to obtain DTW(L,L). s Simultaneously calculate the mean μ of the current session frame length sequence. l and standard deviation σ l The corresponding parameter μ of the standard public protocol s σ s Calculate the relative deviation δ l δ l =|μ l -μ s | / μ s +|σ l -σ s | / σ s ; Step S2-3: Extract the transmission intervals of all adjacent frames in the current session, and divide the interval range into k equal-width intervals, where k is a preset positive integer based on the interval value range; count the number of times the interval appears in each interval, and the ratio of this number to the total number of intervals is the probability density p of that interval. i , i=1,2……k, all p i The frame interval distribution P = {p1, p2, ..., p} that constitutes the current session k }; The frame interval distribution of the standard common protocol is Q={q1,q2,...,q}. k }, where q k Let represent the probability density of the k-th interval in the standard common protocol frame interval distribution; calculate the divergence JSD(P||Q) between P and Q. The calculation process is as follows: first, obtain the average distribution M of P and Q, where the probability density of each interval in M ​​is: (probability density of the corresponding interval of P + probability density of the corresponding interval of Q) / 2; then calculate the KL divergence D(P||M) of P with respect to M and the KL divergence D(Q||M) of Q with respect to M respectively; the KL divergence is expressed by the formula Σ[p i ×ln(p i / m i )]Calculate, where m i Let be the probability density of the i-th interval in M; finally, the divergence value is obtained by calculation: JSD(P||Q)=[D(P||M)+D(Q||M)] / 2.

5. The data stream parsing and detection method for a private protocol of a network gateway isolation component according to claim 1, characterized in that: Step S2 further includes: Step S2-4: Based on the physical transmission direction of the frame and the transaction identifier field in the payload, identify the request frame sequence R={r1,r2,...,r...} in the session. e } and the corresponding response frame sequence A={a1,a2,...,a e }; where r1, r2, r e These represent the timestamps of the 1st, 2nd, and eth request frames, respectively, where e is the total number of request and response pairs in the session; a1, a2, ..., a e These represent the timestamps of the response frames corresponding to each request frame in R; Construct the request frame sequence number sequence X={1,2,...,e} and the delay sequence Y={d1,d2,...,d} e }, where d e =a e -r e Calculate the Pearson correlation coefficient ρ between the two. XY This is used to quantify the linear correlation between the order of request sending and the corresponding delay; it extracts the correlation coefficient ρ between the request-response delay and the sequence number in standard public protocols. s Through the formula δ ρ =|ρ XY -ρ s |Calculate the pattern difference δρ to quantify the difference between the two in their association patterns; Weights w1, w2, and w3 are assigned to three dimensions, with w1 + w2 + w3 = 1. The weight allocation is based on the importance of the core behavioral characteristics of the common protocol and is configured differently according to the protocol type. A preset maximum normalized distance factor, maxDTW, is introduced, based on the maximum DTW distance obtained from a large number of samples, and is used to normalize DTW(L,L) distances. s Normalize to the [0,1] interval; By weighted fusion of the difference indicators from the above three dimensions, a comprehensive behavioral similarity quantification value is generated: S=w1×[1-DTW(L,L)] s ) / maxDTW]+w2×(1-JSD)+w3×(1-δ ρ ).

6. The data stream parsing and detection method for a private protocol of a network gateway isolation component according to claim 1, characterized in that: Step S3 includes: Step S3-1: Perform a byte-by-byte sliding scan on the payload content of all frames in the session, and identify feature fragments based on a predefined network semantic feature rule base, including continuous byte sequences that conform to the standard network address format, numeric fields that conform to the value range of the transmission service port, and feature byte combinations that match public protocol control instructions; record the frame number, byte offset within the frame, and feature type for each identified feature fragment; Step S3-2: Construct an association network of feature fragments based on the session timeline. Each node in the network corresponds to a feature fragment, and the node attributes include fragment content, type, and occurrence time. Nodes are connected by directed lines, with the direction of the line indicating the chronological order of fragment occurrences. The weight of the line is the ratio of the interval between the occurrences of two fragments to the average frame interval of the session. Three types of association parameters are calculated by traversing the network, including the ratio of the total number of times the same feature fragment appears repeatedly in different frames to the total number of frames in the session, the sequence length of different types of feature fragments appearing consecutively in a fixed order, and the number of links in which feature fragments distributed across frames constitute the complete protocol interaction logic. Based on these parameters, a cross-frame semantic logic chain is constructed. The completeness of the chain is the ratio of the number of feature fragment types actually included to the number of types included in the corresponding public protocol standard semantic chain.

7. The method for parsing and detecting the data stream of a private protocol in a network gateway isolation component according to claim 1, characterized in that: Step S4 includes: Step S4-1: Set the behavior similarity judgment threshold T1 and the semantic logic chain completeness standard T2. T1 is determined based on the distribution of similarity quantification values ​​of standard public protocol samples, taking the lowest similarity value of normal protocol interaction in the sample set; T2 is the percentage threshold of the number of feature fragment types that the corresponding public protocol standard semantic chain should contain. The completeness requirement is determined when the number of feature types contained in the actual semantic chain reaches 90% or more of the standard type number. Step S4-2: Compare the comprehensive behavior similarity quantization value S calculated in Step S2 with the threshold T1, and at the same time compare the semantic logic chain integrity obtained in Step S3 with the standard T2; if S < T1, it is determined that there is a significant difference between the current session behavior pattern and the public protocol; if the semantic logic chain integrity < T2, it is determined that there is a lack or abnormality in the cross-frame semantic association; when any one of the determination results is true, output the conclusion that the private protocol data stream has not completed a complete protocol conversion, generate a detection report containing specific abnormal items, and record in detail in the report the feature dimensions with behavior similarity lower than the threshold, the types of missing feature segments in the semantic chain, and the abnormal association order, and mark the corresponding frame numbers and timestamps.

8. A data stream parsing and detection device for a private protocol of a network gateway isolation component, applied to the data stream parsing and detection method for a private protocol of a network gateway isolation component as described in any one of claims 1-7, characterized in that: The device includes a data acquisition module, a behavior analysis module, a semantic parsing module, a judgment output module, a storage module, and a human-computer interaction module; The data acquisition module connects to the communication interfaces of different types of network isolation components through a configurable physical interface adapter, captures the original signals during the communication process, and converts them into a structured data set including frame sequences, byte contents, and transmission intervals; The behavior analysis module divides the data stream into sessions, extracts the frame length change features, timing distribution features, and interaction mode features, and calculates the behavior similarity quantization value by comparing with the standard public protocol behavior patterns pre-stored in the storage module; The semantic parsing module identifies the feature segments of the frame payload content, constructs an association network based on the time axis, and generates a cross-frame semantic logic chain by analyzing the spatio-temporal association relationships between the feature segments; The judgment output module generates a protocol compliance detection report according to the judgment results of the behavior similarity quantization value and the semantic logic chain integrity; The storage module stores the interface protocol rule library, the standard public protocol behavior feature library, the network semantic feature rule library, and various data during the parsing process; The human-computer interaction module provides functions such as detection parameter configuration, visualization display of the detection process, and output of the detection report.

Images