Network information security monitoring method based on data stream correlation analysis

Abstract

The application belongs to the technical field of network security and is used to solve the problem that the prior art cannot identify abnormal communication disguised behind legal field values from the perspective of inter-field cooperative behavior, and specifically discloses a network information security monitoring method based on data stream association analysis, which comprises the following steps: obtaining and analyzing application layer protocol data units in a network session to obtain field instances, each field instance containing a field name, a field value, a transmission direction and a timestamp; extracting a micro-feature set of each field instance; performing matching query on the micro-feature set in a global fingerprint library to obtain an instantaneous deviation degree, calculating a session risk score based on the total sum of barrier values of all fields in the same session and the number of cooperative distortion pairs, and performing a hierarchical response according to the score; and the application constructs a dynamic behavior fingerprint based on a cross-source distribution condition for each field, and converges the missed detection rate and the false alarm rate of field role misjudgment to an acceptable range at the engineering level.

Description

Technical Field

[0001] This invention belongs to the field of network security technology, specifically a network information security monitoring method based on data flow correlation analysis. Background Technology

[0002] In the field of network traffic security monitoring, parsing and compliance checks of application layer protocol data units are commonly used basic detection methods. Existing systems typically extract key-value pairs of protocol fields, verify whether the format, length, and enumeration range of the field values ​​meet the protocol specifications, and supplement this with state machine verification of the reasonableness of the request-response order. As long as the field values ​​can be correctly parsed, have no malformed features, and the state transitions are as expected, the traffic is judged to be normal. The implicit consensus of this logic is that a legitimate field value must correspond to a legitimate communication behavior.

[0003] In real-world network environments, the semantic roles of protocol fields have systematically shifted with the evolution of application ecosystems. For example, in single-page application frameworks, the "Cookie" field, which should be generated by the server, is actively constructed and continuously changed by the client. Various web crawling tools fill in custom identifiers at the end of the version number of "User-Agent." These behaviors are completely compliant in terms of format, but they have deviated from the original role positioning of the fields. Attackers take advantage of this phenomenon to encode control commands or leaked data into those fields whose roles have "shifted," making them fully compliant with format validation rules.

[0004] A more advanced covert technique involves breaking down the complete payload into multiple tiny fragments and injecting them into several different fields within the same transaction. The changes in each field are extremely small, falling within their normal fluctuation range when observed individually, and do not trigger any alarms based on single-field thresholds. However, the changes in these fields are synchronized over time and can be pieced together to form a meaningful sequence. Existing technologies, due to their isolated analysis of each field and reliance on static format specifications or statistical baselines, lack the ability to detect such cross-field collaborative relationships. When attackers use these methods to construct pseudo-normal communication tunnels, traditional methods easily misjudge this structured anomaly as multiple unrelated business fluctuations, leading to the continuous failure to detect highly covert threats. Therefore, how to identify this abnormal communication disguised behind legitimate field values ​​from the perspective of inter-field collaborative behavior has become an urgent technical problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a network information security monitoring method based on data flow correlation analysis, which solves the problem that existing technologies cannot identify abnormal communication disguised behind legitimate field values ​​from the perspective of inter-field collaborative behavior; The technical problem to be solved by this invention is: how to provide a network information security monitoring method based on data flow correlation analysis that can identify abnormal communication disguised behind legitimate field values ​​from the perspective of inter-field collaborative behavior.

[0006] The objective of this invention can be achieved through the following technical solutions: Network information security monitoring methods based on data flow correlation analysis include: Retrieve and parse the application layer protocol data units in the network session to obtain field instances. Each field instance contains a field name, field value, transmission direction, and timestamp. Extract the set of micro-features for each field instance. The set of micro-features includes: the generator tendency score based on the historical distribution of transmission direction, the change characteristics based on the magnitude and direction of field value changes, and the stage identifier based on the session stage. Based on the micro-feature set, a matching query is performed in the global fingerprint database to obtain the instantaneous deviation. The fingerprint database stores the legal behavior patterns and their positive counts. The positive count is increased only when the behavior pattern meets the preset cross-source distribution conditions. Maintain a barrier value for each field in each session, perform exponential decay on the barrier value over time, and determine the incremental value to be added to the barrier based on the comparison between the instantaneous deviation and the preset deviation threshold. Detect synchronous change features and assemblable features among multiple fields within the same transaction, calculate the collaborative amplification coefficient based on the detection results, and apply the coefficient to subsequent incremental values ​​to adjust the barrier accumulation rate. When the barrier value of any field exceeds the dynamic threshold, a suspicious flag is generated. A session risk score is calculated based on the sum of the barrier values ​​of all fields in the same session and the number of cooperative distortion pairs, and a graded response is executed according to the score.

[0007] The present invention has the following beneficial effects: By constructing a dynamic behavioral fingerprint for each field based on cross-source distribution conditions, the system no longer relies on static protocol specifications to determine the legality of fields. Instead, it uses the joint verification of behavioral patterns by at least three different source addresses to establish a positive counting baseline. This allows normal role drift of fields (such as client-initiated cookie construction in single-page applications) to be automatically absorbed as legitimate behavior, while the pollution of fingerprints by attackers through single source addresses or short-term injections is effectively suppressed. Thus, at the engineering level, the false negative rate and false positive rate of field role misjudgment are simultaneously converged to an acceptable range. By independently maintaining an exponentially decaying barrier value for each field in each session, the system transforms a single instantaneous deviation into a continuous cumulative quantity with a memory effect. High deviation events cause the barrier to gain an increment proportional to the degree of deviation, while over time the barrier falls back exponentially with a fixed half-life. This mechanism allows short-term, sudden service transients (such as a one-time User-Agent version upgrade) to automatically fade away after several cycles, while long-term, persistent small deviations (such as each field changing only a few characters per day in a slow heartbeat channel) gradually raise the barrier due to the cumulative effect, eventually exceeding the dynamic threshold to trigger a suspicious flag, thus achieving sensitive capture of slow, covert channels and immunity to transient noise. By detecting the synchronous change characteristics and assemblable characteristics of multiple fields within the same transaction, the system establishes a joint judgment logic for cross-field collaborative distortion. When two or more fields change in the same direction simultaneously within the same transaction, and this synchronous relationship maintains a stable proportion in multiple consecutive transactions, or when the changes in different fields can be spliced ​​into a numerical sequence with a monotonic regularity according to preset rules, the collaborative amplification coefficient is activated and the subsequent barrier increment is amplified by one to three times or more. This design enables attackers to amplify and identify even if they break down the complete payload into weak changes within multiple fields, and the deviation of each field is below the threshold, due to the structural collaboration of multiple fields in time rhythm and numerical splicing, thus breaking through the limitations of single-field island detection. By calculating session risk scores based on the sum of field barriers and collaborative distortion, and then performing log recording, alarm sending, session blocking, and original packet capture according to the score classification, the system achieves a step-by-step convergence from micro-level field behavior to macro-level session risk. The dynamic threshold automatically increases with the current packet rate to prevent alarm storms caused by random noise under high load conditions. Fields participating in collaborative distortion in blocked sessions automatically increase the strictness of cross-source distribution conditions within a preset time period, forming a feedback loop. As a result, in a complex environment where normal business traffic and attack traffic coexist, the system separates the structured anomalies of pseudo-normal communication from background noise, keeping the deviation between the number of alarm events and the number of real threats within an acceptable engineering boundary. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is the main flowchart of Embodiment 1 of the present invention; Figure 2This is a sub-flowchart of step S5 in Embodiment 1 of the present invention. Detailed Implementation

[0010] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] In traditional network traffic security monitoring systems, especially for the parsing and compliance checks of application layer protocols, the legality of field value formats is generally regarded as the core basis for judging whether communication is normal. The underlying logic of this judgment is: as long as the field value can be successfully decomposed by the protocol parser, meets the length and enumeration range, and the request-response order meets the state machine's expectations, the traffic is considered to be normal. This consensus of "format equals security" is highly effective in periods when the network environment is relatively static and protocol implementation strictly follows the specifications.

[0012] With the rapid evolution of web application frameworks, API gateways, browser kernels, and various network clients, the actual semantic roles of protocol fields have undergone systematic shifts. For example, in single-page application frameworks, the "Cookie" field, which should be generated by the server, is actively constructed and continuously changed by the client. Various web crawling tools fill in custom identifiers at the end of the version number of "User-Agent". Attackers exploit this phenomenon to encode control commands or data leakage payloads into those fields whose roles have shifted, making them fully compliant with format validation rules, thereby bypassing detection. A more advanced stealth method is to break the complete payload into multiple tiny fragments and inject them into multiple different fields within the same transaction. The changes in each field are extremely small, and when observed individually, they all fall within the normal fluctuation range. Existing technologies, due to their isolated analysis of each field and reliance on static format specifications or statistical baselines, lack the ability to perceive such cross-field collaborative relationships.

[0013] In real-world enterprise intranet or data center environments, these vulnerabilities can lead to specific failures. For example, when attackers communicate with external C2 servers via slow heartbeat channels, the system treats minor changes in each field as independent business fluctuations (such as version number updates or minor language preference adjustments), failing to recognize the temporal rhythm and numerical concatenation of these changes. This results in misjudging structured anomalies as multiple unrelated normal behaviors. Furthermore, when legitimate applications actively modify fields typically generated by the server (such as "ETag") for business needs, static specifications will mark them as role misalignment, generating numerous invalid alarms and ultimately causing operations personnel to block such detections. The system cannot distinguish between "role drift" and "role hijacking," nor can it identify covert channels using distributed coding.

[0014] If the above problems are not addressed, the security monitoring system will continue to lose its ability to objectively distinguish between normal and false communications. In particular, the failure to identify misaligned field roles will allow attackers to use drifted fields as covert channels for extended periods, resulting in persistently high false negative rates for data leaks and command injections. At the same time, the failure to detect cross-field collaborative distortions will allow attackers to arbitrarily split payloads to evade single-field thresholds, ultimately rendering the system completely blind to structured anomalies. As a result, the inaccuracy of detection capabilities will systematically hinder effective defense against advanced persistent threats and encrypted tunnel communications, affecting the achievement of the core protection objectives of network security infrastructure.

[0015] Example 1: As Figure 1 As shown, the network information security monitoring method based on data flow correlation analysis includes: Step S1: Obtain and parse the application layer protocol data unit in the network session to obtain field instances. Each field instance contains a field name, field value, transmission direction, and timestamp. In the specific implementation of step S1, the original data packets are captured in real time from the network traffic mirror port or the packet capture node deployed in the bypass, and the captured data packets are reassembled and the flow is restored to establish a network session based on the five-tuple (source IP address, destination IP address, source port, destination port, transport layer protocol type); each session consists of a series of data packets with the same five-tuple, each session is assigned a unique session identifier, and the state information of the session is continuously maintained until the session times out or the connection is terminated.

[0016] When the system starts, the fingerprint database is empty; all behavioral patterns need to accumulate positive counts through online learning, without relying on any preset rules.

[0017] For each restored session, its application layer protocol data unit is extracted. Since different application layer protocols have different port numbers and signature characteristics, the system first identifies the protocol based on the port number (e.g., 80, 443, 53) and the first few bytes of the payload. If it is identified as HTTP protocol with plaintext transmission, the HTTP protocol parser is invoked; if it is identified as DNS protocol, the DNS parser is invoked. For encrypted traffic (such as TLS), the system performs the corresponding application layer parsing after obtaining the plaintext in the decrypted enterprise exit environment or through a man-in-the-middle proxy. The parser splits the protocol data unit into a series of field instances according to the syntax rules of the corresponding protocol. For example, for an HTTP request message, the parser reads the request line and header fields line by line, extracts the field name and field value using colons or spaces as delimiters, and records the starting offset of the field in the message (for subsequent editing distance calculation). Each field instance is encapsulated as a structured data object, which contains at least the following four attributes: field name, field value, transmission direction, and timestamp.

[0018] The transmission direction is determined as follows: When the session is established, the source IP address of the first SYN packet sent during the handshake process is recorded as the client-side identifier, and the corresponding one is the server-side identifier; for each data packet in the session, the transmission direction is marked as 0 (client to server) or 1 (server to client) depending on whether its source IP is consistent with the client identifier; this direction label is stored along with the field instance.

[0019] The timestamp is obtained as follows: When capturing each raw data packet, the system obtains a high-precision timestamp from the network card driver or data packet capture library, in seconds (e.g., the floating-point representation of Unix timestamps, accurate to six decimal places, equivalent to microsecond precision), and assigns the timestamp to all field instances parsed from the data packet; different fields belonging to the same data packet share the same timestamp; for fields of different data packets in the same session, the timestamp is used for subsequent timing analysis and barrier decay calculation.

[0020] To support the calculation of the direction of transmission score based on the historical direction queue in subsequent steps, the system maintains a separate circular buffer for each field name in each session. This buffer is initially empty and is used to store the transmission direction labels corresponding to the field instances in the most recent 100 transactions. If the queue length exceeds 100, the oldest value is popped. The definition of a transaction varies depending on the protocol: for request-response protocols such as HTTP, a request and its corresponding response constitute a transaction; for DNS, a query and its corresponding answer constitute a transaction; for one-way protocols or UDP streams without responses, each independent data packet is considered a transaction. Whenever a field instance is resolved, the system determines the order of occurrence of the field in the current transaction according to the transaction boundary to which it belongs and pushes the corresponding direction label into the buffer.

[0021] In addition, the system maintains a stage index counter for each session, initially set to 0. The stage index is used to roughly depict changes in the session's communication process, and its update mechanism relies on pattern jump detection of field values. Specifically, after each transaction is completed, the system compares the value type and edit distance of the same-named fields in the current transaction with those in the previous transaction: if the type changes, or the edit distance is greater than 0.5 times the maximum of the old string length and the new string length, a jump is determined. In a transaction, as long as any field satisfies the jump condition, the stage index is incremented by 1 (at most 1 per transaction), rather than being incremented by field. To prevent the stage index from growing indefinitely, a maximum of 10 stages is set, and it is fixed at 10 after exceeding this limit. This stage index value is also recorded as an additional attribute of the field instance.

[0022] Through the above process, step S1 finally outputs a complete set of field instance objects. Each field instance contains a field name, field value, transmission direction label, high-precision timestamp, stage index of the session to which it belongs, and a reference to the corresponding direction history queue. These data provide structured and traceable input for subsequent feature extraction, fingerprint matching, and barrier calculation.

[0023] Step S2: Extract the set of micro-features for each field instance. The set of micro-features includes: the generator tendency score based on the historical distribution of transmission direction, the change characteristics based on the magnitude and direction of field value changes, and the stage identifier based on the session stage. In step S2, the system calculates three micro-features for each field instance output in step S1: the generating tendency score, the change characteristics (including the change magnitude level and change direction), and the stage identifier that has been maintained in step S1.

[0024] For the generating bias score, the system reads the corresponding direction history queue in the session to which the field instance belongs; assuming the queue length is L (initially 0, maximum 100), the elements stored in the queue are the transmission direction labels (0 or 1) of this field in the most recent L transactions; the current direction label is denoted as... In the queue and The number of equal elements is denoted as "same", which produces the direction score. When L=0 (i.e., the first occurrence), =0; This indicates that, in the absence of historical data, the current transmitter is conservatively considered a non-frequent generator of this field. This value falls within the [0, 0.3) interval defined in step S3, corresponding to the discretized interval 0 (low-frequency generator), so that the first occurrence of the field instance is temporarily not classified as a high-frequency generator during fingerprint matching. The physical meaning of this score is: the degree of historical regularity of the field being generated by the current transmitter (client or server) in the current session; for example, if a field is sent by the client in the first 99 transactions and is also sent by the client in the 100th transaction, then... =100 / 100=1; If the 100th time is changed to be sent by the server, then =0 / 100=0.

[0025] For change features, the system needs to compare the value of the current field instance with the value of the same field in the previous transaction. If there is no value in the previous transaction (i.e., this is the first appearance of the field in the current session), the change magnitude level is marked as "no change" and the change direction is marked as "no direction", and the change feature extraction ends directly. Otherwise, the data type is first determined based on the content of the field value: if the string can be completely parsed into an integer (without leading zeros and without non-numeric characters) or a floating-point number, it is judged as numeric; if the string consists only of letters, numbers and a few separators (hyphens, underscores, periods), it is judged as character; otherwise, it is judged as opaque.

[0026] For numeric fields, let the old value be... The new value is Calculate the relative difference For string fields with a length less than 200 bytes, the normalized edit distance is calculated as follows: First, calculate the edit distance lev (the minimum number of single-character edit operations required to transform the old string into the new string), then divide by the maximum length of the old string and the new string to obtain the result. For strings longer than 200 bytes, to avoid excessive computational overhead, we directly... Set it to 0.15 (corresponding to the intermediate range); We get Then, it is mapped to the level of change according to the following rules: <0.05 indicates level 0 (minor change); 0.05 ≤ <0.3 indicates Level 1 (Medium Variable); ≥0.3 is Level 2 (Significant Change); for numeric fields, the rule for determining the direction of change is: if Then it is upward (direction value +1), if If the current string is greater than the old string in lexicographical order, it is considered to be increasing; if it is less than the old string, it is considered to be decreasing; if they are equal, it is considered to be unchanged. When the length remains the same but the character content is replaced, the result of the lexicographical comparison determines whether it is increasing or decreasing, and it is no longer forced to be considered unchanged. The direction value is also represented by +1, -1, and 0.

[0027] The stage identifier is directly taken from the stage index value (integer 0 to 10) of the record for the field instance in step S1, without any additional calculation.

[0028] For example, suppose we detect the "User-Agent" field in an HTTP session, where the previous transaction value was "Chrome / 120.0.0" and the current transaction value is "Chrome / 120.0.1". This field is identified as a typical string with a length less than 200 characters, an edit distance of 2, and a maximum string length of 13. ≈0.154, falling into the intermediate range (level 1), and the current value length is equal to the old value but the characters have changed. In fact, comparing the lexicographical order of the strings cannot directly determine the increase or decrease. Here, since the length has not changed and the edit distance is not zero, it is judged as unchanged (direction 0) according to the rules. This result will be used for subsequent fingerprint matching and deviation calculation.

[0029] Step S3: Based on the micro-feature set, perform a matching query in the global fingerprint database to obtain the instantaneous deviation. The fingerprint database stores the legal behavior patterns and their positive counts. The positive count is increased only when the behavior pattern meets the preset cross-source distribution conditions. Before step S3 is executed, the system has pre-built a global fingerprint database. This fingerprint database adopts a hash table structure, and the key is a tuple composed of five discrete features: field name, generating interval, stage index, change magnitude level, and change direction. The change direction takes values ​​of -1, 0, and +1. When constructing the hash key, these three values ​​are directly stored as integers in the tuple without string concatenation conversion; or the strings "decreasing", "unchanged", and "increasing" are used to replace -1, 0, and +1 respectively for key concatenation. The specific encoding method is chosen by the engineering implementation, as long as different direction values ​​correspond to different keys; the generating interval is the generating tendency score calculated in step S2. After discretization, the following is obtained: ∈[0,0.3) is mapped to interval 0 (low-frequency generator), [0.3,0.7) is mapped to interval 1 (medium-frequency), and [0.7,1.0] is mapped to interval 2 (high-frequency). The stage index directly uses the integer values ​​0 to 10 maintained in step S1. The change amplitude level and change direction are directly taken from the output values ​​of step S2 (level 0 / 1 / 2, direction -1 / 0 / 1). The value corresponding to each key is a structure containing four fields: positive count. (Initial 0), First appearance time Last update time Source address set (Use hash sets to store IP addresses).

[0030] The update and query logics of the fingerprint database are decoupled. After step S2 outputs the discretized features of a field instance as a quintuple, the system first queries the fingerprint database using this quintuple as the key. If the key does not exist, a new entry is created in the fingerprint database, and settings are configured accordingly. =0、 =Current time ( (Entries will not be modified after creation) ={source IP of the current session} = Current time, and set the instantaneous deviation of this query to 1; if the key exists, check whether two conditions are met at the same time: the number of different source addresses in the source address set reaches the first threshold, and the difference between the current timestamp and the first occurrence time reaches the second threshold; in this embodiment, the first threshold is 3 and the second threshold is 600 seconds (10 minutes). In actual deployment, these two thresholds can be adjusted according to the network environment; The deduplication rules for the source address set are as follows: IPv4 addresses and IPv6 addresses are considered different source addresses; if traffic undergoes network address translation, the translated public IP address is used as the source address identifier for that entry; when multiple internal IP addresses behind the same NAT gateway are mapped to the same public IP address, they are counted as only one source address in the set.

[0031] If this condition is met, the behavior pattern is considered valid after being verified by a sufficient number of independent source addresses over a sufficiently long time window, and the entry is then... Increase by 1 and add the current source IP. (If it does not already exist), update simultaneously. If the cross-source distribution condition is not met, then the original condition is retained. Unchanged, and not updated With timestamp.

[0032] When a new entry is created, the current source IP is added. Forward counting =0; When the key is retrieved again in a subsequent query, it will only be if the cross-source distribution condition is met (i.e., ≥3 and the current time and The current source IP will only be added if the difference is ≥600 seconds. (If it does not already exist), and Add 1; if the cross-source distribution condition is not met, then do not... and Make no modifications.

[0033] The formula for calculating the instantaneous deviation D is: The logarithm in the formula is base 10. It is a non-negative integer, theoretically with no upper limit, but when More than 10 6 hour, The increment tends to level off, with the D value approaching 0.14 and the change range less than 0.01, so its impact on subsequent barrier calculations is negligible. To avoid the logarithmic operation overhead caused by excessively large counts, in engineering implementations, [the following can be used]. The upper limit is truncated to 10 6 That is, when the actual count exceeds this value, it is set to 10. 6 calculate.

[0034] in For each key found, a positive count is generated (if the key does not exist). =0); This formula guarantees that the range of D is (0,1]: when When =0, D=1; When the value is 9, D = 1 / (1+1) = 0.5; When the deviation is 99, D = 1 / (1+3) ≈ 0.33; the lower the deviation, the closer the current behavior pattern is to a fully validated legal pattern.

[0035] For example: Suppose that in a certain enterprise network environment, in a large number of HTTP sessions with different source IPs, the generation range of the "User-Agent" field is 2 (generated by high-frequency clients), the stage index is 0, the change magnitude level is 0 (minor change), and the change direction is 0 (unchanged). After 10 minutes and more than 3 different source IPs have accumulated, the positive count of this quintuple key will gradually increase. At a certain moment, a malicious session attempts to change the change magnitude level of "User-Agent" to 2 (major change) and the direction to +1 (increasing). This new quintuple key may not have a sufficiently high positive count in the fingerprint database, or may not even exist at all. Therefore, the instantaneous deviation is close to 1. The system outputs this high deviation to step S4.

[0036] Step S4: Maintain a barrier value for each field in each session, perform exponential decay on the barrier value according to the time period, and determine the incremental value to be added to the barrier based on the comparison result of the instantaneous deviation degree and the preset deviation degree threshold. In step S4, the system maintains a barrier value V for each field in each session, initially set to 0. This barrier value is not updated in real time, but rather triggers two sub-processes—decay and incremental accumulation—at a fixed time period (period length ΔT = 1 second). Each field also records the timestamp of the last barrier update. The initial value is the session start time.

[0037] At the start of each cycle, the system iterates through all fields that have appeared at least once in the current session; for each field, it retrieves the current barrier value. and the current time Calculate the time difference (Unit: seconds); Exponential decay processing is as follows: ; Where α = 0.05 is the decay rate, that is, the barrier value decays by 5% per second; when Δt is large, the decay factor is (1-0.05). Δt Decline according to the index; if If the value is less than 0.001, then set it to 0 directly.

[0038] Exponential decay makes deviation memories exhibit a half-life characteristic, with the impact of each deviation event decreasing exponentially over time, consistent with the human forgetting pattern of abnormal behavior.

[0039] After decay is complete, the system checks if there is a new field instance (i.e., the instantaneous deviation D output in step S3) in the current period; if not, the barrier value is updated to V= ,renew If so, end the processing of this field; otherwise, determine the deviation threshold between D and the preset threshold. The incremental value ΔV is calculated based on the comparison result of =0.3: ; The gain coefficient β = 0.2; then the decayed barrier value is added to the increment, and an upper limit is set. The specific update process is as follows: ; And order The potential barrier decay and increment accumulation are triggered with a fixed time period ΔT = 1 second, and the period boundary is aligned with the whole second of the system clock; each period Within each field, the instantaneous deviation D of all field instances received within the period is collected, and only the maximum value is used to calculate the increment ΔV; if no instance is received within the period, no increment calculation is performed.

[0040] For example: The current barrier value of a certain field is 0.6, and the last update time was 2 seconds ago; after 2 seconds of decay, =0.6×(0.95) 2≈0.6×0.9025=0.5415; Assuming the system receives a new instance of this field within these 2 seconds, with an instantaneous deviation D=0.7, then ΔV=(0.7-0.3)×0.2=0.08, V=min(1.0,0.5415+0.08)=0.6215; If multiple instances are received within the same period (the same field may appear multiple times within the same period), only the largest D value is taken to calculate the increment, avoiding repeated accumulation; If there is no new instance, then V=0.5415.

[0041] The barrier value decays exponentially over time, so that a short-term single deviation will not have a long-term impact, but a continuous or frequent high deviation will cause the barrier to gradually rise. A suspicious flag will only be triggered when the barrier value exceeds the dynamic threshold in a subsequent step (step S6). The barrier value of each field is maintained independently and does not interfere with each other.

[0042] Step S5: Detect synchronous change features and assemblable features among multiple fields within the same transaction, calculate the co-amplification coefficient based on the detection results, and apply the coefficient to subsequent incremental values ​​to adjust the barrier accumulation rate; Step S5 operates based on transaction boundaries; the definition of a transaction is the same as in step S1: for request-response protocols such as HTTP, a request and its corresponding response constitute a transaction; for DNS, a query and its corresponding answer constitute a transaction; for one-way protocols or UDP streams without responses, each independent data packet is considered a transaction; when each transaction is completed, the system collects all field instances within that transaction and reads the cached values ​​of the same-named fields from the previous transaction for comparison of changes.

[0043] The detection of synchronous change characteristics targets any two fields A and B within the same transaction; let's say the value of A in the previous transaction was... , direction is The value of A in the current transaction is , direction is Similarly for B; first, determine whether both A and B have changed in the current transaction, i.e. and (For numeric or string types, whether a change occurs is determined by whether the magnitude of the change in step S2 is non-zero); if both change, then check whether the direction of change is consistent and whether both are non-zero: that is... symbols and If the signs of the fields are the same and the sign is not 0, then the field pair (A,B) is said to satisfy the single synchronization condition in the transaction.

[0044] The detection method for synchronous change characteristics is as follows: In the same transaction, for any two fields, if both have changed relative to the previous transaction and the direction of change is consistent, then the field pair is determined to meet the single synchronization condition in this transaction; a transaction sliding window of length 20 is maintained for each field pair, and a boolean value is stored at each position in the window, indicating whether the corresponding transaction meets the single synchronization condition; after each transaction is processed, the judgment result of the current transaction is added to the end of the window, and if the window length exceeds 20, the head element is removed; the window is initially empty, and when the number of valid transactions in the window reaches 20, the proportion of true values ​​in the window is calculated and recorded as the synchronization ratio. Preset a synchronization threshold In this embodiment, the value is taken as 0.7; if If the value is greater than 0.7, it is determined that the field pair has synchronous change characteristics; the old data in the window is naturally overwritten by the new data, and no additional clearing is required; when the session ends, the sliding window of the field pair is released.

[0045] The detection of assemblable features only applies to field pairs from which numerical values ​​can be extracted. The system has several pre-defined concatenation rule tables. For example, for the HTTP protocol, the last digit of the version number in "User-Agent" is concatenated with the low-order digits of the value of a specific key name in "Cookie". For the field pair (A, B) in the current transaction, if the numerical change can be extracted from both, the detection will be successful. and (That is, the difference between the current value and the previous transaction value; whether to take the absolute value or the signed difference depends on the concatenation rules), then they are combined into a single concatenation value according to the rules. , where K is the preset bit offset base (e.g., 16 or 256). Maintain a length of for each field pair The sliding window W stores the most recent The calculated splicing value Z; This is a configurable parameter; in this embodiment, it is set to 10. This window is independent of the Boolean sliding window used for synchronous change feature detection, and its length can be different. (The concatenation formula is missing from the original text.) The offset base K is preset according to the numerical range of the field pair: for example, when ΔA and ΔB are both integers from 0 to 15, K is 16; when ΔA is from 0 to 255 and ΔB is from 0 to 255, K is 256; the specific value is configured by the protocol analyst according to the actual value range of the field. When the number of valid values ​​in the window reaches (In this embodiment, the value is 5) The monotonicity check begins; the check method is to calculate the adjacent difference sequence. If all Same sign and not 0, and The variance is less than its mean. If the ratio of monotonic variance to the threshold is multiples of 1, then an assemblable feature is determined to exist; for windows not full In this situation, Take 0.05; window full back, Set to 0.1; the above parameters , , All of these are configurable parameters and can be adjusted according to the actual deployment scenario.

[0046] The concatenation rule table is pre-configured by protocol analysts, defining which field pairs may have numerical concatenation relationships for common protocols (such as HTTP and DNS). For example, in the HTTP protocol, the last four bytes of the "User-Agent" version number and the lower four bytes of the "_ga" value in "Cookie" are considered concatenable. If there is no preset rule, the concatenation detection of this field pair will not be performed.

[0047] Synergistic amplification factor The calculation method is as follows: First, determine the total number of fields in the current transaction. :like If <0, then directly set =1 (no co-amplification), end the calculation; otherwise, calculate the maximum possible number of field pairs. Suppose that the number of field pairs satisfying the synchronous change characteristic in the current transaction is . Assembleable mark =1 (if at least one pair of fields satisfies the assemblable feature), otherwise 0; then Among them, the contribution of synchronous change features to the amplification coefficient is set to 1.0, and the contribution of assemblable features is set to 2.0. This difference in weights reflects that assemblable features have a stronger indication of cooperative distortion than simple synchronous change. The weight values ​​can be adjusted according to the actual deployment environment.

[0048] For a field that participates in synchronization or assemblability detection, if the field participates in multiple field pairs (e.g., appears in two synchronization pairs simultaneously), then the result calculated from all field pairs involving that field is taken. The maximum value is used as the final amplification factor for this field, instead of taking the average or multiplying. When performing the incremental calculation in step S4 in the next transaction cycle, the ΔV calculated for this field is multiplied by this amplification factor and then added to the barrier value.

[0049] For example: An HTTP transaction contains three fields: "User-Agent", "Accept-Language", and "Cookie". Assume that "User-Agent" and "Accept-Language" are synchronized for four consecutive transactions, while "Cookie" and "User-Agent" are not synchronized. Then... =1, =3, =3, =0.333; if no assembly features are available, then =1 + 0.333 = 1.333; In the next cycle, the incremental values ​​of each of these three fields will be amplified by a factor of 1.333 before being accumulated into the potential barrier; If the changes in "Cookie" and "User-Agent" can be pieced together into a regular sequence, then =1, =1+0.333+2=3.333, resulting in a stronger magnification effect.

[0050] Step S6: When the barrier value of any field exceeds the dynamic threshold, a suspicious label is generated. The session risk score is calculated based on the sum of the barrier values ​​of all fields in the same session and the number of cooperative distortion pairs. A graded response is then executed based on the score.

[0051] During step S6, the system has obtained the current barrier value V for each field, as well as the list of synchronized change feature field pairs and the list of assemblable feature field pairs output in step S5.

[0052] Let the number of raw packets received in the previous second be . The system maintains a smooth packet rate. The initial value is equal to the first one. Each cycle is based on Update: For each field, the dynamic threshold is set to... .

[0053] Where 10000 is a reference rated rate, the denominator can be adjusted accordingly if the actual deployment environment has different bandwidth; when As the threshold increases, the threshold also rises, meaning a higher barrier is needed to trigger a suspicious flag under high traffic load, thus avoiding excessive alarms due to random noise; if the current field's barrier value... If so, add the field to the list of suspicious fields and record its barrier value.

[0054] For each session, the session risk score is calculated to reflect both single-field anomaly accumulation and cross-field collaborative distortion; let the sum of the barrier values ​​of all fields in the current session be... ,Right now The number of coordinated distortion pairs is (The sum of the number of field pairs with a synchronization ratio exceeding 0.7 and the number of field pairs with assemblable features), then Where λ is the cooperative distortion weighting coefficient, and in this embodiment, λ = 0.5; when When =0, This ensures that the accumulation of a single-field barrier can independently trigger a response; when When the value is ≥1, the co-distortion contributes an additional risk increment; the logarithm is base 2, so that for every doubling of the number of co-distortion pairs, the contribution value increases by a fixed unit (λ), reflecting the risk differentiation of the order-of-magnitude difference in co-distortion.

[0055] Set the first risk threshold =0.5, second risk threshold =1.0; The judgment rules for graded response are as follows: If Score < 0.5, it is judged as level 0, and only the statistical information of the session is written to the local log; if 0.5 ≤ Score < 1.0, it is judged as level 1, and an alarm event (including session quintuple, suspicious field name, and current barrier value) is sent to the security information event management platform; if Score ≥ 1.0 or any assemblable feature exists, it is judged as level 2, and in addition to sending an alarm, the session is immediately blocked and the original packet capture is triggered (the payload of the last 10 seconds is stored); the fields involved in the cooperative distortion in the blocked session will have their cross-source distribution conditions tightened for the next 24 hours.

[0056] In sessions classified as Level 2, the system records the field names involved in the collaborative distortion. For these fields, the threshold for the number of source addresses in the conservative cross-source condition is increased from 3 to 5, and the time window threshold is increased from 10 minutes to 30 minutes within the next 24 hours. Specifically, each field maintains a strengthened verification flag and an expiration timestamp. When the flag is valid, the increased threshold is used when determining the cross-source distribution condition after querying the fingerprint database in step S3. The flag is automatically cleared after 24 hours, and the default threshold is restored.

[0057] The various thresholds, weighting coefficients, and preset constants involved in this embodiment are mainly based on three aspects: statistical patterns of network traffic behavior, performance trade-offs in engineering implementation, and empirical thresholds in the field of security detection; these will be explained step by step below: Edit distance jump threshold of 0.5 times string length: When the content difference of the same field in two transactions exceeds half, it indicates that the information carried by the field has changed significantly, which is suitable for triggering the increase of the index in the stage; taking 0.5 as the threshold conforms to the intuitive judgment of "more than half means qualitative change"; Directional history queue length 100: 100 transactions are sufficient to cover most behavior patterns of a session, while memory usage is controllable (100 bits per field per session). In the cross-source condition, the threshold of the number of source IPs is 3 (first threshold) and the time window is 600 seconds (second threshold): 3 different source addresses can exclude the accidental behavior of a single test or attack source; 10 minutes (600 seconds) ensures the continuity of the behavior pattern over time; the combination of the two can effectively resist single-source slow fingerprint poisoning, while allowing normal multi-source business patterns to enter the fingerprint database within a reasonable time. 1-second decay period: Matches the time granularity of common network traffic analysis, consistent with the packet rate acquisition period, and is easy to implement; The decay rate is 0.05 (i.e., 5% decay per second): corresponding to a half-life of about 13.5 seconds, which makes the increase in the potential barrier caused by a single deviation dissipate after more than ten cycles, which can tolerate short-term business fluctuations and accumulate continuous anomalies. Lower limit of the potential barrier is 0.001: it is set to zero when it is below this value to avoid computational noise caused by the accumulation of extremely small numbers; Deviation threshold of 0.3 and gain coefficient of 0.2: Deviations above 0.3 are considered noteworthy; multiplying by 0.2 ensures that the increment of the potential barrier for each deviation does not exceed 0.14 (when the deviation is 1), making the potential barrier change smooth and less likely to directly exceed the threshold due to a single large change; A synchronous sliding window length of 20: 20 transactions are sufficient to determine the synchronization trend in a continuous communication, with low computational overhead; Synchronization threshold 0.7: Requires that the two fields change synchronously in more than 70% of transactions, excluding random coincidences; The assemblable window length of 10 and the minimum sample size of 5: 10 samples are sufficient to observe monotonic trends, and starting detection with 5 samples can detect patterns at an early stage; Monotonicity variance proportion thresholds of 0.05 (partial window) and 0.1 (full window): When the sample size is small, the requirements are more stringent (variance less than 5% of the mean), while when the sample size is large, they can be appropriately relaxed to 10% to balance detection sensitivity and confidence. Offset base K (16 or 256): Set according to the common range of field values, such as 16 for 4-bit binary (0-15) and 256 for 8-bit (0-255) to ensure that the concatenated values ​​do not overlap; The weights of the synergistic amplification coefficients 1.0 and 2.0 are: synchronous changes contribute 1 times, while assemblable features contribute 2 times, reflecting that the latter is a stronger indicator of anomalies; Reference rate 10000pps: the packet rate of a typical enterprise's gigabit network. Cooperative distortion weighting coefficient of 0.5: This ensures that the contribution of cooperative distortion is moderate, so as not to mask the single-field barrier, but to provide an increment. First risk threshold 0.5 and second risk threshold 1.0: low-risk areas only record, medium-risk areas trigger alarms, and high-risk areas block detection. The empirical values ​​take into account both missed and false alarms. Strengthen the verification period to 24 hours: If anomalies are repeatedly detected within a day, maintain strict conditions; if no anomalies are detected for more than a day, restore the conditions to balance defense and resource consumption.

[0058] The values ​​mentioned above are exemplary configurations for this embodiment. In actual deployment, they can be dynamically adjusted according to the network environment, performance requirements, and security policies. All parameters are designed as configurable items and do not constitute a limitation on the scope of protection.

[0059] After processing through steps S1 to S6, the system no longer judges the format validity of individual fields in isolation. Instead, it establishes a behavioral fingerprint baseline for each field based on the commonalities of the group, and achieves sensitive capture of long-term small deviations through the exponential decay and incremental accumulation of the potential barrier. The joint detection of synchronous changes and combinable features across fields means that even if an attacker breaks down the payload into weak changes within multiple fields, the structural coordination of multiple fields in terms of time rhythm and numerical splicing will amplify and identify them. The final output risk score and graded response fuse field-level suspicious markers with cross-field collaborative distortion evidence, enabling the system to effectively distinguish between normal service drift and covert channel communication while maintaining a low false alarm rate. This reduces the false alarm boundary and alarm noise of pseudo-normal traffic at the engineering level.

[0060] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

[0061] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0062] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A network information security monitoring method based on data flow correlation analysis, characterized in that, include: Retrieve and parse the application layer protocol data units in the network session to obtain field instances. Each field instance contains a field name, field value, transmission direction, and timestamp. Extract the set of micro-features for each field instance. The set of micro-features includes: the generator tendency score based on the historical distribution of transmission direction, the change characteristics based on the magnitude and direction of field value changes, and the stage identifier based on the session stage. Based on the micro-feature set, a matching query is performed in the global fingerprint database to obtain the instantaneous deviation. The fingerprint database stores the legal behavior patterns and their positive counts. The positive counts are increased only when the behavior pattern meets the preset cross-source distribution conditions. Maintain a barrier value for each field in each session, perform exponential decay on the barrier value over time, and determine the incremental value to be added to the barrier based on the comparison between the instantaneous deviation and the preset deviation threshold. Detect synchronous change features and assemblable features among multiple fields within the same transaction, calculate the collaborative amplification coefficient based on the detection results, and apply the coefficient to subsequent incremental values ​​to adjust the barrier accumulation rate. When the barrier value of any field exceeds the dynamic threshold, a suspicious flag is generated. A session risk score is calculated based on the sum of the barrier values ​​of all fields in the same session and the number of cooperative distortion pairs, and a graded response is executed according to the score.

2. The network information security monitoring method based on data flow correlation analysis according to claim 1, characterized in that, The method for extracting the producer's bias score is as follows: maintain a historical direction queue for each field in each session, push the current transmission direction into the queue, count the proportion of elements in the queue that are the same as the current direction, and use this proportion as the producer's bias score.

3. The network information security monitoring method based on data flow correlation analysis according to claim 1, characterized in that, The method for extracting change features is as follows: calculate the relative difference for numerical fields and the normalized edit distance for string fields. Based on the calculation results, the change magnitude is divided into three levels: slight change, medium change, and huge change. The direction of change is determined by comparing the current value with the historical value, whether it is increasing, decreasing, or remaining unchanged.

4. The network information security monitoring method based on data flow correlation analysis according to claim 1, characterized in that, The key of the global fingerprint database consists of five parts: field name, generation interval, stage index, change magnitude level, and change direction. The cross-source distribution condition refers to the number of different source addresses in the source address set corresponding to the matching entry in the fingerprint database reaching a first threshold, and the duration of the entry reaching a second threshold.

5. The network information security monitoring method based on data flow correlation analysis according to claim 4, characterized in that, The fingerprint database is updated as follows: when the current feature combination does not exist in the fingerprint database, an entry is created and the positive count is initialized to zero; when a matching entry exists and the cross-source distribution condition is met, the positive count of the entry is increased and the current source address is added to the set; the instantaneous deviation is inversely related to the positive count, and the higher the positive count, the lower the instantaneous deviation.

6. The network information security monitoring method based on data flow correlation analysis according to claim 1, characterized in that, The exponential decay processing of the barrier value is as follows: record the last barrier update time of each field, calculate the time difference with the current time, multiply the barrier value by the decay factor calculated based on the time difference, and the decay factor decreases exponentially as time increases; if the decayed barrier value is lower than the lower limit, it is set to zero.

7. The network information security monitoring method based on data flow correlation analysis according to claim 1, characterized in that, The detection method for synchronous change characteristics is as follows: In the same transaction, for any two fields, if both have changed relative to the previous transaction and the change direction is consistent, then the field pair is determined to meet the single synchronization condition in the transaction; a sliding window with a preset length is maintained for each field pair to record whether the single synchronization condition is met in the most recent transactions. Once the number of valid transactions within the window reaches the preset length, the percentage of transactions within the window that meet the single synchronization condition is calculated. If this percentage exceeds the synchronization threshold, it is determined that there are synchronization change characteristics.

8. The network information security monitoring method based on data flow correlation analysis according to claim 1, characterized in that, The detection method for assemblable features is as follows: for field pairs with extractable values, the changes in the two fields are combined into a concatenated value according to a preset concatenation rule; a sliding window of the most recent concatenated values ​​is maintained, and when the window is full and the values ​​in the window show monotonous and regular changes, it is determined that there are assemblable features.

9. The network information security monitoring method based on data flow correlation analysis according to claim 1, characterized in that, The collaborative amplification factor is determined based on the proportion of the number of synchronously changed field pairs detected in the current transaction to the total number of possible field pairs, as well as the presence of assemblable features. When assemblable features exist, the increment of the amplification factor is greater than the increment brought about by the synchronously changed features. This collaborative amplification factor is used to amplify the subsequent incremental value of the corresponding field.

10. The network information security monitoring method based on data flow correlation analysis according to claim 1, characterized in that, The dynamic threshold is dynamically adjusted according to the current network packet rate; the higher the packet rate, the larger the dynamic threshold. The session risk score is calculated based on the sum of barrier values ​​and the number of cooperative distortion pairs. When the number of cooperative distortion pairs is zero, the risk score is equal to the sum of barrier values. The risk score increases with the increase of the sum of barrier values ​​and grows logarithmically with the increase of the number of cooperative distortion pairs. The graded response includes: logging only when the risk is below the first risk threshold, sending an alarm when the risk is between the first and second risk thresholds, and blocking the session and triggering original packet capture when the risk is above the second risk threshold or when there are assembleable features.

Images