Security detection methods, devices and equipment for communication protocols

Abstract

This disclosure provides a security detection method, apparatus, and device for communication protocols, comprising: obtaining the structure of the communication protocol through unsupervised learning and structural parsing based on bidirectional raw traffic data; utilizing sequence alignment algorithms to find common patterns and variable fields among packets in different protocol session streams; identifying high-frequency byte sequences that consistently appear in application layer payload data; dividing the payload of a single packet into multiple fields and inferring the semantic type of each field; and constructing a protocol syntax tree for the communication protocol; then, based on the protocol syntax tree, modeling normal communication behavior to obtain a behavioral baseline of the behavioral patterns under normal communication behavior; and performing anomaly detection on real-time traffic obtained from the communication network based on the protocol syntax tree and the behavioral baseline. Thus, without prior knowledge of the detailed protocol specifications, it can intelligently learn from communication behavior to discover potential threats, reducing the application limitations of this security detection method.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of network security technology, and more specifically, to a security detection method, apparatus, and device suitable for communication protocols. Background Technology

[0002] With the rapid development of internet and IoT technologies, the complexity and diversity of network communication are increasing daily. In addition to standard communication protocols such as HTTP, HTTPS, and TCP / IP, a large number of proprietary communication protocols, industry-customized communication protocols, and encrypted communication protocols (such as TLS 1.3 and QUIC) are widely used. While these protocols improve performance and protect privacy, they also pose significant challenges to network security monitoring.

[0003] Existing security monitoring methods require prior knowledge of detailed protocol specifications, which limits their application. Summary of the Invention

[0004] The embodiments described herein provide a security detection method, apparatus, and device for communication protocols that overcome the aforementioned problems.

[0005] Firstly, based on the content of this disclosure, a security detection method for communication protocols is provided, the method comprising: Collect bidirectional raw traffic data in the communication network and preprocess the bidirectional raw traffic data to obtain bidirectional communication session stream and application layer load data; Unsupervised learning and structure analysis are performed based on bidirectional raw traffic data to obtain the structure of the communication protocol, which includes: One protocol session stream in a bidirectional communication session stream is regarded as a sequence of multiple messages. Sequence alignment algorithms are used to find common patterns and variable fields among messages in different protocol session streams. Identify high-frequency byte sequences that consistently appear in application layer payload data; Based on common patterns, variable fields, and high-frequency byte sequences, the payload of a single message is divided into multiple fields, and the semantic type of each field is inferred. Constructing a protocol syntax tree for a communication protocol includes: the root node of the protocol syntax tree representing the entire message, the intermediate nodes of the protocol syntax tree representing composite fields, and the leaf nodes of the protocol syntax tree representing basic fields, and calculating the offset, length, and inferred semantic type of each field. Based on the protocol syntax tree, normal communication behavior is modeled to obtain a behavioral baseline that characterizes the behavior pattern of the communication protocol under normal communication behavior. Based on the protocol syntax tree and behavioral baseline, anomaly detection is performed on real-time traffic obtained from the communication network.

[0006] Secondly, according to the present disclosure, a security detection device for communication protocols is provided, the device comprising: The extraction module is used to collect bidirectional raw traffic data in the communication network and preprocess the bidirectional raw traffic data to obtain bidirectional communication session streams and application layer load data. The learning and structure parsing module is used to perform unsupervised learning and structure parsing based on bidirectional raw traffic data to obtain the structure of the communication protocol. It includes a comparison module, an identification module, a field partitioning and inference module, and a protocol syntax tree construction module. The comparison module treats a protocol session stream in a bidirectional communication session stream as a sequence of multiple messages, and uses a sequence comparison algorithm to find common patterns and variable fields between messages in different protocol session streams. The identification module is used to identify high-frequency byte sequences that appear regularly in application layer payload data; The field segmentation and inference module is used to segment the payload of a single message into multiple fields based on common patterns, variable fields, and high-frequency byte sequences, and to infer the semantic type of each field. The protocol syntax tree construction module is used to construct the protocol syntax tree of the communication protocol. It includes: the root node of the protocol syntax tree represents the entire message, the intermediate nodes of the protocol syntax tree represent composite fields, the leaf nodes of the protocol syntax tree represent basic fields, and the offset, length and inferred semantic type of each field are calculated. The behavior baseline construction module is used to model normal communication behavior based on the protocol syntax tree, and obtain a behavior baseline to characterize the behavior pattern of the communication protocol under normal communication behavior. The anomaly detection module is used to detect anomalies in real-time traffic obtained from the communication network based on the protocol syntax tree and behavior baseline.

[0007] Thirdly, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the security detection method for communication protocols as described in any of the above embodiments.

[0008] Fourthly, a computer-readable storage medium is provided, on which a computer program is stored, and when executed by a processor, the computer program implements the steps of the security detection method for communication protocols as described in any of the above embodiments.

[0009] The security detection method for communication protocols provided in this application obtains the structure of the communication protocol through unsupervised learning and structural parsing based on bidirectional raw traffic data. It includes using sequence alignment algorithms to find common patterns and variable fields among packets in different protocol session flows, identifying high-frequency byte sequences that consistently appear in application layer payload data, dividing the payload of a single packet into multiple fields based on the common patterns, variable fields, and high-frequency byte sequences, inferring the semantic type of each field, and constructing a protocol syntax tree (TCP). Then, based on the TCP, normal communication behavior is modeled to obtain a behavioral baseline characterizing the communication protocol's behavior patterns under normal communication conditions. Finally, anomaly detection is performed on real-time traffic obtained from the communication network based on the TCP and behavioral baseline. Thus, without prior knowledge of the detailed protocol specifications, it can intelligently learn from communication behavior to discover potential threats, enabling its application in network security detection scenarios with a plethora of proprietary communication protocols or rapid iterations of protocol versions. This improves the adaptability of the security detection method and reduces its application limitations.

[0010] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. It should be understood that the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure, wherein: Figure 1 This is a flowchart illustrating a security detection method for communication protocols provided in this disclosure.

[0012] Figure 2 This is a flowchart illustrating an unsupervised learning and structure parsing process based on bidirectional raw traffic data, as disclosed in this publication.

[0013] Figure 3 This is a schematic diagram of a security detection device for communication protocols provided in this disclosure.

[0014] Figure 4 This is a schematic diagram of the structure of a computer device provided in this disclosure.

[0015] It should be noted that the elements in the attached diagram are schematic and not drawn to scale. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are also within the scope of protection of this disclosure.

[0017] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, the statement of “connecting” or “coupling” two or more parts together shall mean that these parts are directly joined together or joined through one or more intermediate components.

[0018] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of the phrase "embodiment" in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0019] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists, A and B exist simultaneously, or B exists. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Terms such as "first" and "second" are only used to distinguish one component (or part of a component) from another component (or another part of a component).

[0020] In the description of this application, unless otherwise stated, "multiple" means two or more (including two), and similarly, "multiple groups" means two or more (including two groups).

[0021] The existing security monitoring methods and their corresponding limitations are mainly as follows: (1) Signature-based detection (such as traditional IDS / IPS): relies on the signature library of known attacks, and cannot effectively identify zero-day attacks, attacks targeting unknown vulnerabilities in specific protocols, and attacks using encrypted traffic. (2) Detection based on fixed rule parsing: requires prior knowledge of the target protocol, and its parser needs to be developed in advance according to the protocol specification. For the endless stream of private protocols or the rapid iteration of protocol versions, the rule maintenance cost is extremely high, and it cannot cope with the "unknown unknown" threat. (3) Detection based on traffic statistics features: only focuses on metadata such as message length, timing, and traffic size, lacks a deep understanding of the protocol payload content, has low detection accuracy, and high false alarm rate. (4) Existing deep packet inspection (DPI) technology: although it can parse some protocol content, its effectiveness is greatly reduced when facing encrypted traffic. Even for plaintext protocols, its detection logic is relatively rigid, and it is difficult to detect hidden attacks that deviate from the protocol specification but have the correct format (such as protocol compliance abuse, data leakage, etc.).

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0023] To address at least some of the technical problems described above, embodiments of this disclosure provide a security detection method for communication protocols. By performing deep analysis, behavioral modeling, and anomaly detection on communication protocol data, this method is applicable to identifying and issuing warnings about security threats such as, but not limited to, unknown protocols, proprietary protocols, and encrypted protocol traffic. Figure 1 This is a flowchart illustrating a security detection method for communication protocols provided in an embodiment of this disclosure, as shown below. Figure 1 As shown, the specific process of the security detection method for communication protocols mainly includes: First, step S110 is executed. In step S110, bidirectional raw traffic data in the communication network is collected, and the bidirectional raw traffic data is preprocessed to obtain bidirectional communication session flow and application layer load data. That is, step S110 mainly performs traffic collection and preprocessing.

[0024] There are several ways to collect bidirectional raw traffic data in a communication network. For example, bidirectional raw traffic data can be collected at the gateway through network splitting or mirroring.

[0025] There are various methods for preprocessing bidirectional raw traffic data, some of which are exemplified below. In some embodiments, preprocessing bidirectional raw traffic data to obtain bidirectional communication session streams and application layer load data may include the following steps: Step 1: Reassemble the bidirectional raw traffic data into a bidirectional communication session stream based on the source IP, destination IP, source port, destination port, and transport layer communication protocol; that is, reassemble the data packets of the bidirectional raw traffic data into a complete bidirectional communication session stream based on the five-tuple (source IP, destination IP, source port, destination port, and transport layer communication protocol) to perform session stream reassembly.

[0026] Step 2: Strip the Ethernet header, IP header, and transport layer header from the bidirectional communication session stream to extract the application layer payload data. This involves stripping the Ethernet header, IP header, and transport layer (TCP / UDP) header from the bidirectional communication session stream to extract the application layer payload data. During this step, if the bidirectional communication session stream is encrypted, extract the plaintext portion of the handshake phase and the metadata of the encrypted records. Specifically, for encrypted traffic (such as TLS), the plaintext portion of the handshake phase (such as SNI, cipher suites, certificate information, etc. in ClientHello and ServerHello) and the metadata of subsequent encrypted records (such as message length and arrival interval) can be extracted for payload extraction.

[0027] Step 3: Perform generalization processing on the variable fields in the application layer load data. The variable fields should include at least IP address and timestamp to normalize the data and reduce noise in subsequent modeling.

[0028] Next, refer to Figure 1 and Figure 2 Then, proceed to step S120. In step S120, unsupervised learning and structure parsing are performed based on the bidirectional raw traffic data to obtain the structure of the communication protocol. This involves unsupervised learning and structure parsing of the communication protocol format, aiming to automatically infer the structure of the communication protocol without requiring protocol documentation. (Reference) Figure 2 It may include the following sub-steps S121 to S124: refer to Figure 2 In step S121, a protocol session stream within a bidirectional communication session is considered as a sequence of multiple messages. A sequence alignment algorithm is used to find common patterns and variable fields among messages in different protocol session streams for message sequence segmentation and alignment. Specifically, a protocol session stream is treated as a sequence of multiple messages. A sequence alignment algorithm is used to find common patterns and variable fields among messages in different protocol session streams. The choice of sequence alignment algorithm can be varied; for example, it can include improved multi-sequence alignment algorithms and / or sequence modeling algorithms based on recurrent neural networks.

[0029] refer to Figure 2In S122, high-frequency byte sequences that consistently appear in the application layer payload data are identified to perform keyword / delimiter discovery. Specifically, statistical analysis methods are used to identify high-frequency byte sequences that consistently appear in the application layer payload data; these sequences are likely to be protocol keywords, command words, or delimiters.

[0030] There are various methods for identifying recurring high-frequency byte sequences in application layer payload data. For example, these include: using statistical analysis methods to identify recurring high-frequency byte sequences in application layer payload data; wherein the statistical analysis algorithm includes frequency analysis methods and / or information entropy analysis methods; and the high-frequency byte sequences include keyword information, command word information, and / or delimiter information of the communication protocol.

[0031] It should be noted that the above-mentioned "high-frequency byte sequence that appears regularly" refers to a byte sequence that appears more frequently than a set threshold in a manner such as, but not limited to, a fixed position or a fixed period.

[0032] refer to Figure 2 In step S123, based on common patterns, variable fields, and high-frequency byte sequences, the payload of a single message is divided into multiple fields, and the semantic type of each field is inferred to perform field boundary and semantic inference. The method for inferring the semantic type of each field can be varied. For example, inferring the semantic type of each field may include: inferring the semantic type of each field by analyzing the distribution characteristics of its values; where distribution characteristics include: value range, entropy value, and / or character type; and semantic types include length fields, identifier fields, counter fields, and / or data payload fields.

[0033] Specifically, in S123, based on the alignment and keyword information in the aforementioned steps S121 and S122, a single message payload is divided into multiple fields. Furthermore, its semantic type (e.g., length field, identifier field, counter field, data payload field, etc.) is inferred by analyzing the distribution characteristics of field values ​​(e.g., value range, entropy value, character type).

[0034] refer to Figure 2 In S124, a protocol syntax tree (TCP) is constructed, which formally represents the parsed communication protocol structure as a TCP. This TCP can include: the root node representing the entire message, intermediate nodes representing composite fields, and leaf nodes representing basic fields. The offset, length, and inferred semantic type of each field are also statistically recorded. Composite fields can include message headers and message bodies, while basic fields can include integers and strings.

[0035] Next, refer to Figure 1Then, proceed to step S130. In step S130, based on the protocol syntax tree, normal communication behavior is modeled to obtain a behavioral baseline that characterizes the behavior pattern of the communication protocol under normal communication behavior, that is, communication behavior modeling and baseline establishment are performed.

[0036] There are various approaches to modeling normal communication behavior based on protocol syntax trees. For example, based on the parsed communication protocol structure, normal communication behavior can be modeled from multiple dimensions to establish a multi-dimensional behavioral baseline. The following is an example of one method for constructing a multi-dimensional behavioral baseline.

[0037] In some embodiments, modeling normal communication behavior based on the protocol syntax tree to obtain a behavioral baseline characterizing the behavior pattern of the communication protocol under normal communication behavior may include the following steps S131 to S134: In step S131, the order and dependencies of different message types in the protocol session flow are analyzed. A state machine learning algorithm is used to automatically construct a state machine model of the communication protocol to describe the interaction logic of the communication protocol under normal communication behavior. Specifically, state machine learning is performed by analyzing the order and dependencies of different message types (determined by the command words parsed in step 120) in the session process under normal communication behavior. A finite state machine model of the protocol is automatically constructed using state machine learning algorithms, such as, but not limited to, the L-algorithm, to describe the interaction logic of the communication protocol. For example, the interaction logic of this communication protocol under normal communication behavior may include login-request-response-logout.

[0038] In step S132, the distribution of response times among the defined message types is statistically analyzed to establish a timing model of the communication protocol under normal communication behavior, thereby performing timing behavior modeling. Specifically, the distribution of response times among predetermined defined message types (such as mean and variance) can be statistically analyzed to establish a timing model under normal communication behavior. For example, the delay between a request message and its corresponding response message should typically be within a defined range.

[0039] In step S133, the data throughput transmitted in the protocol session stream is modeled to obtain a data flow model of the communication protocol under normal communication behavior, i.e., data flow modeling is performed. Specifically, the data throughput transmitted in the protocol session stream is modeled. For example, the total number of bytes in a single session, the ratio of uplink to downlink traffic, and the distribution of the payload length of a single message are statistically analyzed.

[0040] In step S134, at least a portion of the state machine model, timing model, data flow model, and normal value ranges of key fields are integrated to obtain a multi-dimensional behavioral baseline that characterizes the behavior pattern of the communication protocol under normal communication behavior from multiple dimensions. Specifically, the aforementioned state machine model, timing model, data flow model, and normal value ranges of key fields (such as the monotonically increasing nature of a counter) are integrated to form a comprehensive multi-dimensional behavioral baseline profile, which characterizes the behavior pattern of the communication protocol under normal conditions.

[0041] Next, refer to Figure 1 Then, proceed to step S140. In S140, anomaly detection is performed on the real-time traffic obtained from the communication network based on the protocol syntax tree and behavioral baseline. Various anomaly detection methods can be employed. Some methods are illustrated below. For example, the real-time traffic can be compared with the behavioral baseline established in step S130 to perform anomaly detection at different levels.

[0042] In some embodiments, anomaly detection of real-time traffic obtained from a communication network based on a protocol syntax tree and behavioral baseline may include the following steps S141 to S142: In step S141, anomaly detection of real-time traffic is performed at multiple levels, which are divided into sub-steps S1411 to S1415 according to the level type.

[0043] In step S1411, the real-time packets of the real-time traffic are checked to see if they conform to the protocol syntax tree of the constructed communication protocol, in order to perform a structural compliance level check. Specifically, the real-time packets are checked to see if they conform to the learned protocol syntax tree structure. For example, whether there are unknown packet types, whether the field length is abnormal, and whether the field types match (such as characters appearing in numeric fields), thereby performing a structural compliance check.

[0044] In step S1412, the sequence of real-time traffic packets is checked to see if it conforms to the state machine model of the constructed communication protocol, in order to perform state compliance level detection. Specifically, it checks whether the sequence of packets conforms to the learned state machine model. For example, it checks whether there are unauthorized state transitions (such as sending a data request directly without being logged in), thereby performing state compliance detection.

[0045] In steps S1413 to S1415 below, behavior anomaly detection is performed to detect whether the communication behavior of real-time traffic deviates from the behavior baseline.

[0046] In step S1413, based on the timing model, timing anomaly detection is performed on the response times between specified message types in the real-time traffic. For example, a timing anomaly is detected when the request-response time is much longer or shorter than the timing model in the multi-dimensional baseline.

[0047] In step S1414, based on the data flow model, traffic anomaly detection is performed on the data throughput of real-time traffic. For example, when the data throughput of real-time traffic suddenly surges (which may indicate data leakage) or sharply decreases (which may indicate denial of service) compared to the data flow model, traffic anomalies are detected.

[0048] In step S1415, based on the normal value range of the key fields, a hierarchical detection of numerical anomalies is performed on the values ​​of the key fields in the real-time traffic. For example, when an anomaly is detected in the value of a key field, such as a counter reset or an abnormal change in the identifier, the detection result is a numerical anomaly.

[0049] Next, step S142 is executed. Based on preset rules and a scoring mechanism, correlation analysis is performed on the detection results at multiple levels to generate the final security threat events and alarms, thereby performing correlation and threat determination. Specifically, correlation analysis is performed on the anomalies detected at each of the above levels. Considering that a minor anomaly at a single level may not be sufficient to determine a threat, but if anomalies occur simultaneously at multiple levels (e.g., both state machine violations and data flow anomalies exist), the probability of malicious behavior is extremely high. Based on preset rules and a scoring mechanism, the final security threat events and alarms are generated, thereby improving the accuracy of the final generated security threat events and alarms and reducing the probability of false positives or false negatives.

[0050] In some embodiments, the security detection method may further include step S150, in which false positive and false negative samples are acquired, and the parameters of the behavioral baseline are dynamically corrected based on the false positive and false negative samples, so that the corrected behavioral baseline can correctly handle false positive and false negative samples. Specifically, step S150 mainly involves updating and optimizing the behavioral baseline model, thereby enabling the security monitoring system to have self-evolution capabilities. For example, false positive and false negative samples confirmed by expert analysis can be fed back to the behavioral baseline model. Then, incremental learning or online learning algorithms are used to dynamically adjust and update the parameters of the behavioral baseline (such as state machine, timing parameters, traffic thresholds), enabling the security monitoring system to adapt to the normal evolution of network services, while continuously optimizing detection accuracy and reducing the false positive rate.

[0051] The invention will be further described in detail below with reference to a specific application embodiment. This embodiment takes monitoring the control protocol of an unknown Internet of Things (IoT) device as an example. The implementation environment includes: a smart home network containing a new brand of smart light, and the communication network between the smart light and the control mobile app uses an undisclosed, TCP-based proprietary encryption protocol. The security detection method includes the following steps: (1) Step S100: Mirror all bidirectional raw traffic data at the home gateway. Collect the TCP stream between the mobile app (IP: 192.168.1.10) and the smart light (IP: 192.168.1.20). After stripping the TCP header, obtain the encrypted application layer payload data. At the same time, record the length, direction and timestamp of each packet.

[0052] (2) Step S200: The security detection system analyzes a large number of normal session flows of the communication protocol, which include the following.

[0053] 1) In S210, through sequence alignment, it is found that all sessions always start with a fixed-length (e.g., 50 bytes) message, followed by a variable-length message.

[0054] 2) In S220, frequency analysis revealed that the first two bytes of each message (such as 0xFA0x01) always appeared in a fixed manner, which was inferred to be the identifier of the message type.

[0055] 3) Further analysis in S230 revealed that the last four bytes of the message with identifier 0xFA0x01 are an integer value, and this value is equal to the length of the subsequent message payload, suggesting that this field is the "length field". The message with identifier 0xFA0x02 is usually followed by a large segment of high-entropy data, suggesting that it is the "encrypted data payload field".

[0056] 4) In S240, construct the protocol syntax tree for this communication protocol: root node → message type field (2 bytes) → length field (4 bytes) → payload field (N bytes).

[0057] (3) Perform step S300, which includes S310 to S330.

[0058] 1) S310: State machine learning revealed that the communication protocol interaction always starts with a 0xFA0x01 (login) message. After success, the server replies with 0xFA0x03 (ACK). Then the client can send 0xFA0x02 (control command), the server replies with 0xFA0x04 (execution result), and finally ends with 0xFA0x05 (exit).

[0059] 2) S320: Timing modeling revealed that the response time from control command (0xFA0x02) to execution result (0xFA0x04) is typically between 100ms and 50ms.

[0060] 3) S330: Data flow modeling reveals that the total data volume of a single session is typically less than 2KB.

[0061] (4) Step S400: During monitoring, the system detected an abnormal session. This session did not start with a 0xFA0x01 message and directly sent a 0xFA0x02 message (state compliance abnormality). The payload length of the 0xFA0x02 message was abnormally large, reaching 50KB (structure compliance abnormality). Subsequently, the smart light replied with a 0xFA0x04 message, but the response time was extremely short (<1ms), and the total data volume of the session surged (behavioral abnormality: timing abnormality + traffic abnormality).

[0062] (5) Step S500: The system, considering the multiple anomalies mentioned above, generates a high-risk security alarm: "A suspected data penetration attack using the smart light protocol has been detected." After confirmation by the security analyst, the attack pattern is added to the knowledge base and fed back to the baseline model to optimize subsequent detection.

[0063] In summary, this embodiment provides a security monitoring method that can effectively handle encrypted traffic without relying on prior protocol specifications and can intelligently learn from communication behavior to discover potential threats. It offers an intelligent, adaptive, and in-depth security monitoring method oriented towards communication protocols. This method aims to achieve structural analysis of communication protocols, establishment of behavioral baselines, and monitoring of security anomalies through automatic learning and modeling of network traffic, without prior knowledge of detailed protocol specifications. This effectively detects various security threats, including unknown attacks, malicious behavior in encrypted channels, and deviations from protocol compliance, successfully detecting abnormal attack behavior against IoT devices without understanding protocol details.

[0064] The structure of a communication protocol is obtained through unsupervised learning and structural parsing based on bidirectional raw traffic data. This includes using sequence alignment algorithms to find common patterns and variable fields among packets in different protocol session flows, identifying high-frequency byte sequences that consistently appear in application layer payload data, dividing the payload of a single packet into multiple fields based on common patterns, variable fields, and high-frequency byte sequences, inferring the semantic type of each field, and constructing a protocol syntax tree (TCP). Then, based on the TCP, normal communication behavior is modeled to obtain a behavioral baseline characterizing the communication protocol's behavior patterns under normal communication conditions. Finally, anomaly detection is performed on real-time traffic obtained from the communication network based on the TCP and behavioral baseline. This allows for intelligent learning from communication behavior to discover potential threats without prior knowledge of detailed protocol specifications. Therefore, it can be applied to network security detection in scenarios with a proliferation of proprietary communication protocols or rapid iterations of protocol versions, improving the adaptability of the security detection method and reducing its application limitations.

[0065] Compared with existing technologies, the present invention has the following significant advantages: (1) Powerful unknown protocol processing capability: It does not rely on any prior protocol specification documents and can automatically parse protocol structure and behavior through unsupervised learning, which is particularly suitable for security monitoring of private and unknown protocols. (2) Encrypted traffic analysis capability: Even for fully encrypted traffic, the present invention can still perform effective behavior modeling and anomaly detection based on metadata (message length, timing, sequence), breaking through the bottleneck of traditional DPI technology. (3) Deep-level, multi-dimensional detection: It goes beyond simple feature matching and statistical anomaly, and performs deep correlation analysis from multiple dimensions such as protocol structure, state logic, timing behavior, and data traffic, which greatly improves the detection capability of covert and advanced persistent threats (APT). (4) High adaptability and low maintenance cost: The system can automatically learn and establish a normal behavior baseline and can continuously optimize through feedback loops, reducing the dependence on manually writing and maintaining detection rules and reducing operating costs. (5) Low false alarm rate: Through the correlation analysis of multi-level anomalies, it effectively avoids false alarms that may be caused by single-dimensional anomalies and improves the credibility of alarms.

[0066] Figure 3 This embodiment provides a schematic diagram of a security detection device oriented towards communication protocols. The security detection device oriented towards communication protocols may include: Extraction module 310 is used to collect bidirectional raw traffic data in the communication network and preprocess the bidirectional raw traffic data to obtain bidirectional communication session stream and application layer load data. The learning and structure parsing module 320 is used for unsupervised learning and structure parsing based on bidirectional raw traffic data to obtain the structure of the communication protocol. The learning and structure parsing module 320 may include a comparison module 321, an identification module 322, a field partitioning and inference module 323, and a protocol syntax tree construction module 324. Specifically, the comparison module 321 treats a protocol session stream in a bidirectional communication session stream as a sequence of multiple messages, and uses a sequence alignment algorithm to find common patterns and variable fields among messages in different protocol session streams. The identification module 322 is used to identify high-frequency byte sequences that appear regularly in the application layer payload data. The field partitioning and inference module 323 is used to partition the payload of a single message into multiple fields based on common patterns, variable fields, and high-frequency byte sequences, and infer the semantic type of each field. The protocol syntax tree construction module 324 is used to construct the protocol syntax tree of the communication protocol, which includes: the root node of the protocol syntax tree representing the entire message, the intermediate nodes of the protocol syntax tree representing composite fields, and the leaf nodes of the protocol syntax tree representing basic fields, and it calculates the offset, length, and inferred semantic type of each field. The behavior baseline construction module 330 is used to model normal communication behavior based on the protocol syntax tree, obtaining a behavior baseline that characterizes the behavior pattern of the communication protocol under normal communication behavior; and... The anomaly detection module 340 is used to perform anomaly detection on real-time traffic obtained from the communication network based on the protocol syntax tree and behavior baseline.

[0067] In this embodiment, optionally, the identification module 322 is specifically used for: Statistical analysis methods are used to identify high-frequency byte sequences that consistently appear in application layer load data. The statistical analysis algorithms include frequency analysis methods and / or information entropy analysis methods. The high-frequency byte sequences include keyword information, command word information, and / or delimiter information of communication protocols.

[0068] In this embodiment, optionally, the field segmentation and inference module 323 is specifically used to: infer the semantic type of each field by analyzing the distribution characteristics of the field values ​​of each field; wherein, the distribution characteristics include: value range, entropy value and / or character type; the semantic type includes length field, identifier field, counter field and / or data load field.

[0069] In this embodiment, optionally, the extraction module 310 is specifically used for: Based on the source IP, destination IP, source port, destination port, and transport layer communication protocol, the bidirectional raw traffic data is reassembled into a bidirectional communication session stream. The Ethernet header, IP header, and transport layer header are stripped from the bidirectional communication session stream to extract application layer payload data, which includes: if the bidirectional communication session stream is encrypted traffic, the plaintext portion of the handshake phase and the metadata of the encrypted record are extracted from the bidirectional communication session stream; the variable fields in the application layer payload data are generalized, wherein the variable fields include at least IP address and timestamp.

[0070] In this embodiment, optionally, the learning and structure parsing module 320 is specifically used for: The order of occurrence and dependencies of different message types in the protocol session flow are analyzed, and a state machine learning algorithm is used to automatically construct a state machine model of the communication protocol to describe the interaction logic of the communication protocol under normal communication behavior. The distribution of response times among different message types is statistically analyzed, and a timing model of the communication protocol under normal communication behavior is established. Model the data throughput transmitted in the protocol session stream to obtain the data flow model of the communication protocol under normal communication behavior; By integrating at least a portion of the state machine model, timing model, data flow model, and normal value ranges of key fields, a multidimensional behavioral baseline is obtained to characterize the behavior patterns of communication protocols under normal communication behavior from multiple dimensions.

[0071] In this embodiment, optionally, the anomaly detection module 340 is specifically used for: Anomaly detection of real-time traffic is performed at multiple levels, including: checking whether the real-time packets of real-time traffic conform to the protocol syntax tree of the constructed communication protocol for structural compliance level detection; checking whether the sequence of real-time traffic packets conforms to the state machine model of the constructed communication protocol for state compliance level detection; performing timing anomaly level detection based on the timing model to assess the response time between specified packet types in real-time traffic; performing traffic anomaly level detection based on the data flow model to assess the data throughput of real-time traffic; and performing numerical anomaly level detection based on the normal value range of key fields in real-time traffic. Based on preset rules and scoring mechanisms, correlation analysis is performed on the detection results at multiple levels to generate the final security threat events and alerts.

[0072] In this embodiment, optionally, the security detection device may further include a correction module. The correction module is used to acquire false alarm and false negative samples, and dynamically correct the parameters of the behavioral baseline based on the false alarm and false negative samples, so that the corrected behavioral baseline can correctly handle false alarm and false negative samples.

[0073] The security detection device for communication protocols provided in this disclosure can execute the above-described method embodiments. Its specific implementation principle and technical effects can be found in the above-described method embodiments, and will not be repeated here.

[0074] This application also provides a computer device. Please refer to the following for details. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0075] The computer device includes a memory 410 and a processor 420 that are interconnected via a system bus. It should be noted that only a computer device with memory 410 and processor 420 is shown in the figure; however, it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented alternatively. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0076] Computer devices can include desktop computers, laptops, handheld computers, and cloud servers. These devices allow for human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0077] The memory 410 includes at least one type of readable storage medium, including non-volatile memory or volatile memory, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. RAM may include static RAM or dynamic RAM. In some embodiments, the memory 410 may be an internal storage unit of a computer device, such as the hard disk or memory of the computer device. In other embodiments, the memory 410 may also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card equipped on the computer device. Of course, the memory 410 may include both internal storage units and external storage devices of the computer device. In this embodiment, the memory 410 is typically used to store the operating system and various application software installed on the computer device, such as the program code of the method described above. In addition, the memory 410 may also be used to temporarily store various types of data that have been output or will be output.

[0078] Processor 420 is typically used to perform overall operations of a computer device. In this embodiment, memory 410 is used to store program code or instructions, including computer operation instructions, and processor 420 is used to execute the program code or instructions stored in memory 410 or process data, such as program code that runs the methods described above.

[0079] In this article, the bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus system can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0080] Another embodiment of this application also provides a computer-readable medium, which may be a computer-readable signal medium or a computer-readable medium. A processor in a computer reads computer-readable program code stored in the computer-readable medium, enabling the processor to execute the functional actions specified in each step or combination of steps in the above method; and to generate means for implementing the functional actions specified in each block or combination of blocks in the block diagram.

[0081] Computer-readable media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared memory or semiconductor systems, devices or apparatuses, or any suitable combination thereof, wherein the memory is used to store program code or instructions, the program code including computer operation instructions, and the processor is used to execute the program code or instructions of the above-described methods stored in the memory.

[0082] The definitions of memory and processor can be found in the description of the foregoing computer device embodiments, and will not be repeated here.

[0083] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0084] In the various embodiments of this application, the functional units or modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0085] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" as described in this application does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims listing several means, several units of these means may be embodied by the same item of hardware. The use of "first," "second," and "third," etc., does not indicate any order and these words should be interpreted as names. Unless otherwise specified, the steps in the above embodiments should not be construed as limiting the order of execution.

[0087] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A security detection method oriented towards communication protocols, characterized in that, include: Collect bidirectional raw traffic data in the communication network and preprocess the bidirectional raw traffic data to obtain bidirectional communication session stream and application layer load data; Based on the bidirectional raw traffic data, unsupervised learning and structure parsing are performed to obtain the structure of the communication protocol, which includes: One protocol session stream in the bidirectional communication session stream is regarded as a sequence of multiple messages, and a sequence alignment algorithm is used to find the common patterns and variable fields between messages in different protocol session streams; Identify high-frequency byte sequences that consistently appear in the application layer load data; Based on the common pattern and variable fields, as well as the high-frequency byte sequence, the payload of a single message is divided into multiple fields, and the semantic type of each field is inferred. Constructing the protocol syntax tree of the communication protocol includes: the root node of the protocol syntax tree represents the entire message, the intermediate nodes of the protocol syntax tree represent composite fields, the leaf nodes of the protocol syntax tree represent basic fields, and the offset, length and inferred semantic type of each field are calculated. Based on the protocol syntax tree, normal communication behavior is modeled to obtain a behavioral baseline that characterizes the behavior pattern of the communication protocol under normal communication behavior. Based on the protocol syntax tree and the behavior baseline, anomaly detection is performed on the real-time traffic obtained from the communication network.

2. The method as described in claim 1, characterized in that, The identification of a consistently occurring high-frequency byte sequence in the application layer payload data includes: Statistical analysis methods were used to identify high-frequency byte sequences that consistently appear in the application layer load data; The statistical analysis algorithm includes frequency analysis method and / or information entropy analysis method; The high-frequency byte sequence includes keyword information, command word information, and / or delimiter information of the communication protocol.

3. The method as described in claim 1, characterized in that, The inference of the semantic type of each field includes: By analyzing the distribution characteristics of the field values ​​of each field, the semantic type of each field can be inferred; The distribution characteristics include: value range, entropy value, and / or character type; The semantic types include length fields, identifier fields, counter fields, and / or data payload fields.

4. The method as described in claim 1, characterized in that, The composite field includes a message header and a message body; the basic fields include integers and strings; and / or, The sequence alignment algorithms include multi-sequence alignment algorithms and / or sequence modeling algorithms based on recurrent neural networks.

5. The method as described in claim 1, characterized in that, The preprocessing of the bidirectional raw traffic data to obtain bidirectional communication session streams and application layer load data includes: Based on the source IP, destination IP, source port, destination port, and transport layer communication protocol, the bidirectional raw traffic data is reassembled into the bidirectional communication session stream; Stripping the Ethernet header, IP header, and transport layer header from the bidirectional communication session stream to extract the application layer payload data includes: if the bidirectional communication session stream is encrypted traffic, extracting the plaintext portion of the handshake phase and the metadata of the encrypted record from the bidirectional communication session stream; The variable fields in the application layer load data are generalized, wherein the variable fields include at least IP address and timestamp.

6. The method as described in claim 1, characterized in that, The step of modeling normal communication behavior based on the protocol syntax tree to obtain a behavioral baseline characterizing the behavior pattern of the communication protocol under normal communication behavior includes: The order of occurrence and dependencies of different message types in the protocol session flow are analyzed, and a state machine learning algorithm is used to automatically construct the state machine model of the communication protocol to describe the interaction logic of the communication protocol under normal communication behavior. The distribution of response times among different message types is statistically analyzed, and a timing model of the communication protocol under normal communication behavior is established. Model the data throughput transmitted in the protocol session stream to obtain the data flow model of the communication protocol under normal communication behavior; By integrating at least a portion of the state machine model, the timing model, the data flow model, and the normal value range of the key fields, a multidimensional behavioral baseline is obtained to characterize the behavior pattern of the communication protocol under normal communication behavior from multiple dimensions.

7. The method as described in claim 6, characterized in that, The anomaly detection of real-time traffic obtained from the communication network based on the protocol syntax tree and the behavior baseline includes: Anomaly detection is performed on the real-time traffic at multiple levels, as follows: Check whether the real-time packets of the real-time traffic conform to the constructed protocol syntax tree of the communication protocol to perform structural compliance level detection; Check whether the sequence of packets in the real-time traffic conforms to the state machine model of the constructed communication protocol to perform state compliance level detection; Based on the time series model, time series anomaly detection is performed on the response time between specified message types in the real-time traffic. Based on the data flow model, traffic anomaly level detection is performed on the data throughput of the real-time traffic; Based on the normal value range of the key fields, numerical anomaly detection is performed on the values ​​of the key fields of the real-time traffic. Based on preset rules and scoring mechanisms, the detection results at multiple levels are correlated and analyzed to generate the final security threat events and alarms.

8. The method as described in claim 1, characterized in that, Also includes: False positives and false negatives are obtained, and the parameters of the behavioral baseline are dynamically corrected based on the false positives and false negatives, so that the corrected behavioral baseline can correctly handle the false positives and false negatives.

9. A security detection device for communication protocols, characterized in that, include: The extraction module is used to collect bidirectional raw traffic data in the communication network and preprocess the bidirectional raw traffic data to obtain bidirectional communication session stream and application layer load data. The learning and structure parsing module is used to perform unsupervised learning and structure parsing based on the bidirectional raw traffic data to obtain the structure of the communication protocol. It includes a comparison module, an identification module, a field partitioning and inference module, and a protocol syntax tree construction module. The comparison module is used to: treat one protocol session stream in the bidirectional communication session stream as a sequence of multiple messages, and use a sequence comparison algorithm to find common patterns and variable fields between messages in different protocol session streams; The identification module is used to identify high-frequency byte sequences that appear regularly in the application layer load data; The field segmentation and inference module is used to: divide the payload of a single message into multiple fields based on the common pattern and variable fields, as well as the high-frequency byte sequence, and infer the semantic type of each field; The protocol syntax tree construction module is used to: construct the protocol syntax tree of the communication protocol, which includes: the root node of the protocol syntax tree represents the entire message, the intermediate nodes of the protocol syntax tree represent composite fields, the leaf nodes of the protocol syntax tree represent basic fields, and calculate the offset, length and inferred semantic type of each field. The behavior baseline construction module is used to model normal communication behavior based on the protocol syntax tree to obtain a behavior baseline that characterizes the behavior pattern of the communication protocol under normal communication behavior. An anomaly detection module is used to perform anomaly detection on real-time traffic obtained from the communication network based on the protocol syntax tree and the behavior baseline.

10. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the security detection method for communication protocols as described in any one of claims 1-8.

Images