Machine learning-based protocol knowledge base construction method, apparatus and device, and medium

Abstract

The present application relates to the technical field of network security, and specifically relates to a machine learning-based protocol knowledge base construction method, apparatus and device, and a medium. The method comprises: acquiring network environment traffic data, and inputting the network environment traffic data into a traffic detection machine learning model for parsing processing to obtain protocol parsing data corresponding to the network environment traffic data; performing protocol comparison on the protocol parsing data and a protocol knowledge base to screen for an unknown protocol in the network environment traffic data; and performing protocol parsing on the unknown protocol using the traffic detection machine learning model, extracting unknown protocol features, naming the unknown protocol on the basis of the unknown protocol features, and storing the named unknown protocol into the protocol knowledge base to construct a new protocol knowledge base. According to the method, an unknown protocol is discovered reversely, so that a protocol knowledge base is expanded, thereby improving the security and reliability of network communication.

Description

Methods, apparatus, devices, and media for constructing protocol knowledge bases based on machine learning

[0001] This application claims priority to Chinese Patent Application No. 202410571858.7, filed on May 9, 2024, entitled “Method, Apparatus, Device and Medium for Constructing Protocol Knowledge Base Based on Machine Learning”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of network security technology, and in particular to a method, apparatus, device and medium for constructing a protocol knowledge base based on machine learning. Background Technology

[0003] With the emergence of various innovative network applications, network protocol structures are constantly evolving. To meet diverse needs and ensure privacy and personalization, custom network protocols for different application scenarios often have their related knowledge kept secret, becoming unknown protocols. Currently, unknown protocol traffic accounts for 45% of total backbone network traffic, posing a significant challenge to network security management. Protocol parsing and traffic content monitoring will become increasingly difficult.

[0004] In existing technologies, parsing network traffic data requires specific plugins or scripts for each protocol. If an unknown protocol is found during the network traffic data extraction process, the packet can only be discarded to avoid affecting the parsing of other content.

[0005] However, when cyberattacks lurk in unknown protocols and risks cannot be detected in time, network communications become potentially vulnerable, leading to insufficient security in business systems. Summary of the Invention

[0006] This application provides a method, apparatus, device, and medium for constructing a protocol knowledge base based on machine learning, in order to solve the problem that potential risks in network communication lead to insufficient security during the operation of business systems.

[0007] Firstly, this application provides a method for constructing a protocol knowledge base based on machine learning, including:

[0008] Acquire network environment traffic data and input the network environment traffic data into a traffic detection machine learning model for parsing and processing to obtain the protocol parsing data corresponding to the network environment traffic data;

[0009] The protocol parsing data is compared with the protocol knowledge base to filter out unknown protocols in the network environment traffic data.

[0010] The unknown protocol is parsed using a traffic detection machine learning model to extract its features. The unknown protocol is then named based on these features and stored in the protocol knowledge base to construct a new protocol knowledge base.

[0011] Optionally, before acquiring network environment traffic data, the method further includes:

[0012] Multiple known protocol specification files are obtained, and natural language parsing technology is used to decompose each known protocol specification file to obtain the known protocol feature information corresponding to each known protocol specification file;

[0013] Based on the known protocol feature information, the known protocol specification files are extracted and categorized to obtain known protocol specification files with classification identifiers;

[0014] A protocol knowledge base is constructed based on known protocol description documents with classification identifiers and the characteristic information of the known protocols.

[0015] Optionally, the traffic detection machine learning model includes HsMM, and the protocol parsing data includes basic protocol information and protocol timing. The step of inputting the network environment traffic data into the traffic detection machine learning model for parsing processing to obtain the protocol parsing data corresponding to the network environment traffic data includes:

[0016] Based on HsMM, the network environment traffic data is parsed and processed to obtain the keywords and protocol field lengths corresponding to the network environment traffic data.

[0017] Based on the keywords and the protocol field length, the basic protocol information and the protocol timing are determined;

[0018] The step of comparing the protocol parsing data with a protocol knowledge base to filter out unknown protocols in the network environment traffic data includes:

[0019] The protocol basic information, the protocol timing, and the protocol knowledge base are compared to identify unknown protocols in the network environment traffic data.

[0020] Optionally, the protocol comparison includes: static comparison and dynamic comparison. The step of comparing the basic protocol information, the protocol timing, and the protocol knowledge base to filter out unknown protocols in the network environment traffic data includes:

[0021] The basic information of the protocol is statically compared with the protocol knowledge base to obtain a static protocol matrix. The static protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is statically compared.

[0022] The protocol time sequence is dynamically compared with the protocol knowledge base to obtain a dynamic protocol matrix. The dynamic protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is dynamically compared.

[0023] Based on the static protocol matrix and the dynamic protocol matrix, the unknown protocols in the network environment traffic data are determined.

[0024] Optionally, the step of using a traffic detection machine learning model to parse the unknown protocol and extract its features includes:

[0025] The unknown protocol is parsed sequentially, and the empty spaces in each sequence are identified to obtain a standard-aligned unknown protocol data sequence with empty spaces;

[0026] The unknown protocol data sequence is divided into fields to obtain protocol fields;

[0027] Semantic recognition is performed on the protocol fields to extract the features of the unknown protocol.

[0028] Optionally, determining the unknown protocols in the network environment traffic data based on the static protocol matrix and the dynamic protocol matrix includes:

[0029] Determine whether each value in the static protocol matrix is ​​less than a first preset value and whether each value in the dynamic protocol matrix is ​​less than a second preset value;

[0030] When each value in the static protocol matrix is ​​less than a first preset value and each value in the dynamic protocol matrix is ​​less than a second preset value, the protocols corresponding to the static protocol matrix and the dynamic protocol matrix are unknown protocols in the network environment traffic data.

[0031] Secondly, this application provides a protocol knowledge base construction apparatus based on machine learning, comprising:

[0032] The acquisition module is used to acquire network traffic data.

[0033] The processing module is used to input the network environment traffic data into the traffic detection machine learning model for parsing and processing, so as to obtain the protocol parsing data corresponding to the network environment traffic data.

[0034] The processing module is also used to compare the protocol parsing data with the protocol knowledge base to filter out unknown protocols in the network environment traffic data.

[0035] The processing module is also used to perform protocol parsing on the unknown protocol using a traffic detection machine learning model and extract the features of the unknown protocol.

[0036] The determination module is used to name the unknown protocol according to the characteristics of the unknown protocol, store the named unknown protocol in the protocol knowledge base, and construct a new protocol knowledge base.

[0037] Optionally, the acquisition module is further configured to acquire multiple known protocol specification files;

[0038] The processing module is also used to decompose each known protocol specification file using natural language parsing technology to obtain the known protocol feature information corresponding to each known protocol specification file;

[0039] The processing module is further configured to extract and classify the known protocol specification files according to the known protocol feature information to obtain known protocol specification files with classification identifiers;

[0040] The determining module is used to construct a protocol knowledge base based on known protocol description documents with classification identifiers and the known protocol feature information.

[0041] Optionally, the processing module is further configured to parse the network environment traffic data based on HsMM to obtain the keyword and protocol field length corresponding to the network environment traffic data;

[0042] The determining module is further configured to determine the basic protocol information and the protocol timing based on the keyword and the protocol field length;

[0043] The step of comparing the protocol parsing data with a protocol knowledge base to filter out unknown protocols in the network environment traffic data includes:

[0044] The processing module is specifically used to compare the basic protocol information, the protocol timing, and the protocol knowledge base to filter out unknown protocols in the network environment traffic data.

[0045] Optionally, the processing module is further configured to perform a static comparison between the basic protocol information and the protocol knowledge base to obtain a static protocol matrix, wherein the static protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is statically compared;

[0046] The processing module is further configured to dynamically compare the protocol timing with the protocol knowledge base to obtain a dynamic protocol matrix, wherein the dynamic protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is dynamically compared;

[0047] The determining module is further configured to determine the unknown protocols in the network environment traffic data based on the static protocol matrix and the dynamic protocol matrix.

[0048] Optionally, the processing module is further configured to perform sequence parsing on the unknown protocol and identify the empty spaces in each sequence to obtain a standard-aligned unknown protocol data sequence with empty spaces.

[0049] The processing module is also used to divide the unknown protocol data sequence into fields to obtain protocol fields;

[0050] The processing module is also used to perform semantic recognition on the protocol field, thereby extracting the features of the unknown protocol.

[0051] Optionally, the machine learning-based protocol knowledge base construction device further includes: a judgment module;

[0052] The judgment module is used to determine whether each value in the static protocol matrix is ​​less than a first preset value and whether each value in the dynamic protocol matrix is ​​less than a second preset value;

[0053] The determining module is further configured to determine that when each value in the static protocol matrix is ​​less than a first preset value and each value in the dynamic protocol matrix is ​​less than a second preset value, the protocol corresponding to the static protocol matrix and the dynamic protocol matrix is ​​an unknown protocol in the network environment traffic data.

[0054] Thirdly, this application provides a protocol knowledge base construction device based on machine learning, comprising:

[0055] Memory;

[0056] processor;

[0057] The memory stores computer-executed instructions;

[0058] The processor executes computer execution instructions stored in the memory to implement the machine learning-based protocol knowledge base construction method as described in the first aspect and various possible implementations of the first aspect above.

[0059] Fourthly, this application provides a computer storage medium, characterized in that the computer storage medium stores computer execution instructions, which, when executed by a processor, are used to implement the machine learning-based protocol knowledge base construction method as described in the first aspect and various possible implementations of the first aspect.

[0060] The protocol knowledge base construction method based on machine learning provided in this application acquires network environment traffic data and inputs it into a traffic detection machine learning model for parsing to obtain protocol parsing data corresponding to the network environment traffic data. The protocol parsing data is then compared with a protocol knowledge base to filter out unknown protocols in the network environment traffic data. The unknown protocols are then parsed using the traffic detection machine learning model to extract their features, and named according to these features. The named unknown protocols are then stored in the protocol knowledge base to construct a new protocol knowledge base. This method expands the protocol knowledge base by discovering unknown protocols in reverse, thereby improving the security and reliability of network communication. Attached Figure Description

[0061] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0062] Figure 1 is a flowchart of the protocol knowledge base construction method based on machine learning provided in this application;

[0063] Figure 2 is a flowchart of the protocol knowledge base construction method based on machine learning provided in this application;

[0064] Figure 3 is a schematic diagram of the structure of the protocol knowledge base construction device based on machine learning provided in this application;

[0065] Figure 4 is a schematic diagram of the structure of the protocol knowledge base construction device based on machine learning provided in this application.

[0066] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0067] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0068] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0069] With the emergence of various innovative network applications, network protocol structures are constantly evolving. To meet diverse needs and ensure privacy and personalization, custom network protocols for different application scenarios often have their related knowledge kept secret, becoming unknown protocols. Currently, unknown protocol traffic accounts for 45% of total backbone network traffic, posing a significant challenge to network security management. Protocol parsing and traffic content monitoring will become increasingly difficult.

[0070] In existing technologies, parsing network traffic data requires specific plugins or scripts for each protocol. If an unknown protocol is found during the network traffic data extraction process, the packet can only be discarded to avoid affecting the parsing of other content.

[0071] However, when cyberattacks lurk in unknown protocols and risks cannot be detected in time, network communications become potentially vulnerable, leading to insufficient security in business systems.

[0072] To address the aforementioned issues, this application proposes a machine learning-based protocol knowledge base construction method. This method utilizes collectable protocol documentation and employs machine learning to extract semantic, syntactic, and temporal features, recording them in the knowledge base as a reserve. Simultaneously, machine learning is used to analyze protocols in network traffic data, identifying, matching, and naming unknown protocols to supplement the knowledge base. This method expands the protocol knowledge base by reversely discovering unknown protocols, thereby improving the security and reliability of network communication.

[0073] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0074] Figure 1 is a flowchart of the protocol knowledge base construction method based on machine learning provided in this application. The execution subject of this embodiment is, for example, a machine learning system. As shown in Figure 1, the protocol knowledge base construction method based on machine learning shown in this embodiment includes:

[0075] S101: Acquire network environment traffic data and input the network environment traffic data into the traffic detection machine learning model for parsing and processing to obtain the protocol parsing data corresponding to the network environment traffic data.

[0076] Network environment traffic data refers to information about the amount of data transmitted and the flow of data within a network. This data includes network protocol information (known and unknown protocols). Network protocols are the rules and conventions that govern the transmission and exchange of data within a network; different network protocols define different data formats, transmission methods, and communication rules.

[0077] Traffic detection machine learning models are models that use machine learning algorithms to analyze and detect network traffic. These models can learn and be trained on network traffic data to detect and identify abnormal behavior within the network traffic.

[0078] Protocol parsing refers to the parsing and analysis of network traffic data to identify the protocol information and related content it contains. When network traffic data is transmitted over a network, it is packaged and transmitted according to specific protocol formats; different protocols have different data formats and structures. The purpose of protocol parsing is to extract information from network traffic data, such as source IP address, destination IP address, port number, protocol type, and data content.

[0079] Network traffic monitoring tools or packet capture tools (such as Wireshark) can be used to capture network traffic data. Next, the collected network traffic data is preprocessed, including data cleaning, noise removal, and data format conversion. Protocol parsing data is extracted from the preprocessed data, which may include information such as packet size, source and destination IP addresses, port numbers, and protocol types.

[0080] By analyzing network traffic data, abnormal traffic or potential network attacks can be detected. Employing machine learning models for traffic detection can help identify abnormal patterns and improve network security.

[0081] S102: Compare the protocol parsing data with the protocol knowledge base to filter out unknown protocols in the network traffic data.

[0082] The protocol knowledge base contains basic protocol information, including protocol syntax, semantics, and timing. Basic protocol information includes: protocol name, protocol definition, protocol description, and protocol features; protocol syntax includes: protocol syntax components, protocol length, protocol content, and protocol features; protocol semantics includes: semantic components, semantic length, semantic content, and protocol features; protocol timing includes: timing, sender, sender status, receiver, and receiver status. The protocol knowledge base extracts commonalities from the included protocols as basic fields for the protocol feature library. Protocol anisotropy data is recorded by creating feature files and associated with the protocols.

[0083] Protocol comparison refers to comparing the protocols in protocol parsing data with those in a protocol knowledge base. Protocol comparison is used to identify and parse the protocol types in network packets. By comparing the protocols in the protocol parsing data, the type of protocol used in the parsing data can be determined.

[0084] Unknown protocols are used to indicate protocols that do not exist in the protocol knowledge base. When a machine learning system encounters an unrecognized protocol type while parsing network traffic data, and cannot find a corresponding definition in the known protocol knowledge base, it will mark the protocol as an unknown protocol.

[0085] In network analysis and security, the identification and handling of unknown protocols are crucial. Comparing protocol parsing data with a protocol knowledge base to filter out unknown protocols from network traffic data is essential for further analysis of these protocols. This process clarifies their characteristics, assigns them names, and stores these named and clearly defined unknown protocols in the protocol knowledge base. Ultimately, this improves the security and reliability of network communication.

[0086] For example, taking the UDP protocol as an example, the protocol knowledge base contains:

[0087]

Basic Protocol Information

[0088] [Protocol Syntax] - The protocol syntax consists of a header and a data portion. The header is 20 bytes long and contains SourcePort, DestinationPort, Length, and Checksum. It is a connectionless protocol used for data transmission in unreliable network environments. The data portion has a variable length and contains the actual application data.

[0089]

Protocol Semantics

[0090] [Protocol Sequence] - The sequence includes: 1, 2, 3, 4, and 5; 1. There is no sender or sender state. The receiver is checking the port and its state is active. The receiver is checking a certain port and is in a working state; 2. The sender initiates a request. The sender's state is active. The receiver receives the request and its state is active. The sender actively initiates a connection request and sends a UDP datagram to the receiver. This datagram includes the sender's source port number and destination port number; 3. The sender receives feedback. The sender's state is active. The receiver sends feedback. Its state is active. After receiving the sender's UDP datagram, if the receiver agrees to the connection, it returns an acknowledgment message to the sender. The acknowledgment message contains the receiver's source port number and destination port number. At this time, the receiver enters the connected state; 4. The sender sends data. The sender's state is active. The receiver receives data and its state is active. After receiving the receiver's acknowledgment message, the sender also enters the connected state and begins sending data. The sender and receiver can transmit data via UDP datagrams until one party closes the connection or an error occurs.

[0091] S103: Use a traffic detection machine learning model to parse unknown protocols, extract features of unknown protocols, name unknown protocols according to the features, store the named unknown protocols in the protocol knowledge base, and build a new protocol knowledge base.

[0092] First, a traffic detection machine learning model is used to parse unknown protocols. During parsing, key features are extracted. Then, based on these features, the unknown protocol is named. This naming process can determine an appropriate name based on the protocol's function, structure, or other characteristics. Finally, the named unknown protocols are stored in a protocol knowledge base. This knowledge base can contain information on known protocols, as well as unknown protocols identified and named using this method. This helps build a comprehensive protocol database. By continuously identifying, naming, and storing unknown protocols, a new protocol knowledge base is gradually built. This new knowledge base can better manage the ever-evolving network communication protocols.

[0093] The protocol knowledge base construction method based on machine learning provided in this embodiment acquires network traffic data and inputs it into a traffic detection machine learning model for parsing. This yields protocol parsing data corresponding to the network traffic data. The protocol parsing data is then compared with the protocol knowledge base to filter out unknown protocols in the network traffic data. The traffic detection machine learning model is used to parse the unknown protocols, extract their features, and name them according to these features. The named unknown protocols are then stored in the protocol knowledge base, thus constructing a new protocol knowledge base. This method expands the protocol knowledge base by discovering unknown protocols in reverse, thereby improving the security and reliability of network communication.

[0094] Figure 2 is a flowchart of the protocol knowledge base construction method based on machine learning provided in this application. As shown in Figure 2, this embodiment is based on the embodiment in Figure 1, and provides a detailed description of the protocol knowledge base construction method based on machine learning. The protocol knowledge base construction method based on machine learning shown in this embodiment includes:

[0095] S201: Obtain multiple known protocol specification files, and use natural language parsing technology to decompose each known protocol specification file to obtain the known protocol feature information corresponding to each known protocol specification file.

[0096] Among them, the known protocol specification document is used to indicate the network protocol specification information.

[0097] Natural language processing (NLP) technology plays a crucial role in parsing known protocol specification documents.

[0098] Known protocol characteristic information refers to a series of attributes or features used to describe or identify a specific protocol in network communication.

[0099] In collecting known protocol documentation, relevant information needs to be obtained from authoritative sources such as official documents, technical manuals, and standards organization websites. These documentation documents are then cleaned and formatted to remove irrelevant information and ensure the accuracy and consistency of the text content.

[0100] Using natural language processing techniques for syntactic and semantic analysis is a crucial step, helping to understand the structure and meaning of text. These analytical techniques can extract characteristic information contained in each known protocol specification document, including but not limited to the protocol's purpose, specifications, data format, and communication flow.

[0101] S202: Extract and classify known protocol description files based on known protocol feature information to obtain known protocol description files with classification identifiers.

[0102] Extracting and categorizing known protocol documentation based on known protocol characteristics can be achieved through automated tools or manual processing. First, each known protocol documentation file is categorized based on the extracted characteristics, determining its protocol type or category. This can include classification criteria based on protocol purpose, communication method, data format, etc. After obtaining the classification identifier, each known protocol documentation file can be marked or labeled with the corresponding category. These known protocol documentation files with classification identifiers can help users more quickly find and understand different types of protocol specifications, improving work efficiency and accuracy.

[0103] S203: Construct a protocol knowledge base based on known protocol description documents with classification identifiers and known protocol feature information.

[0104] The known protocol specification documents and extracted protocol feature information are integrated into the protocol knowledge base. The known protocol specification documents and known protocol feature information are stored, and the protocols are classified and labeled using a tagging mechanism to facilitate use and identification, ensuring the integrity and accuracy of the information. The information is classified and stored according to the classification labels.

[0105] S204: Obtain network environment traffic data.

[0106] Among them, network traffic data is mainly obtained through packet interception, link detection, etc., and is generally collected through network traffic detection tools (such as Wireshark) for protocol traffic.

[0107] S205: Based on HsMM, the network environment traffic data is parsed and processed to obtain the keywords and protocol field lengths corresponding to the network environment traffic data.

[0108] Among them, HsMM, or Hidden Semi-Markov Model, also known as Continuously Variable Persistent HMM or Piecewise Model, is a stochastic model that can well describe the statistical characteristics of real space through observable data.

[0109] Based on HsMM, the specific implementation of parsing and processing network traffic data to obtain the corresponding keywords and protocol field lengths is as follows: Duration or length is introduced into the HMM model. In HsMM, each field represents an implicit state f. i The detected samples are the captured grouped sequences {m1, m2, ..., m}. w The duration is the length of each field. The calculation requires that the sum of the lengths of all fields equals the length of the group. λ = (A, B, P, π)

[0110] λ represents the parameter set of the entire HsMM model, where: A: state transition matrix, representing the probability of transitioning from one state to another.

[0111] B: The emission probability matrix, also known as the output probability of the hidden state, represents the probability of generating a specific element in the observation sequence under a specific state. In the HTTP protocol example, this means that under a certain state (i... n When generating a specific field (f) n The probability of ).

[0112] P: The state is persistently distributed. Unlike HMM, for HsMM, each state can persist for a different duration (d). n P is the probability distribution describing different durations.

[0113] π: Initial probability distribution, representing the probability of the model being in each state at the beginning.

[0114] Taking the HTTP protocol as an example, the observation sequence is an HTTP message m, consisting of a series of fields (f1, f2, ..., f...). n It consists of ) and each field f. n It is considered a hidden state i n Its duration d n It will be determined based on the actual situation. n and i n+1 This is used to indicate the state i during the HTTP message parsing process. n Transition to the next state i n+1 The probability of.

[0115] S206: Determine the basic protocol information and protocol timing based on the keywords and protocol field lengths.

[0116] The key and protocol field lengths are obtained using tshark, assuming that all captured key and protocol field lengths belong to the target protocol to be analyzed. Then, the process involves five steps: session reconstruction, message reassembly, HsMM, message segmentation, and type inference.

[0117] Unsupervised clustering algorithms are used for message classification. Each group is treated as a node in the network. The distance between nodes is updated by calculating and updating the attractiveness (responsibility) and availability of each node in the network. Finally, the clusters converge to a number of centers, which is the number of classifications, thereby determining the basic information and timing of the protocol.

[0118] S207: Perform a static comparison between the basic protocol information and the protocol knowledge base to obtain a static protocol matrix. The static protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is statically compared.

[0119] Static comparison is based on multi-dimensional matching of protocol name, protocol port, protocol composition, protocol header fields and length, etc.

[0120] S208: Dynamically compare the protocol timing data with the protocol knowledge base to obtain a dynamic protocol matrix. The dynamic protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is dynamically compared.

[0121] Among them, dynamic comparison is based on protocol timing, and obtains the values ​​sent and fed back by relevant protocols for feature matching.

[0122] S209: Based on the static protocol matrix and the dynamic protocol matrix, determine the unknown protocols in the network environment traffic data.

[0123] Optionally, determine the specific implementation of unknown protocols in network traffic data:

[0124] Determine whether each value in the static protocol matrix is ​​less than a first preset value and whether each value in the dynamic protocol matrix is ​​less than a second preset value;

[0125] When each value in the static protocol matrix is ​​less than a first preset value and each value in the dynamic protocol matrix is ​​less than a second preset value, the protocols corresponding to the static protocol matrix and the dynamic protocol matrix are unknown protocols in the network environment traffic data.

[0126] The first preset value can be 0.1 or 0.05. This embodiment does not specifically limit the first preset value.

[0127] The second preset value can be 0.2 or 0.1. This embodiment does not specifically limit the second preset value.

[0128] The first preset value can be equal to the second preset value, the first preset value can be less than the second preset value, or the first preset value can be greater than the second preset value.

[0129] When each value in the static protocol matrix is ​​not less than a first preset value and / or each value in the dynamic protocol matrix is ​​not less than a second preset value, the protocols corresponding to the static protocol matrix and the dynamic protocol matrix are known protocols in the network environment traffic data.

[0130] S210: Perform sequence parsing on the unknown protocol and identify the empty spaces in each sequence to obtain a standard-aligned unknown protocol data sequence with empty spaces.

[0131] First, the Smith-Waterman algorithm is used to pairwise align the data sequences, obtaining the aligned longest common subsequence and a similarity matrix. Based on this, the UPGMA algorithm (a computer language function) is used to cluster the data sequences according to the similarity matrix, resulting in a protocol tree. Progressive sequence alignment is performed starting from the leaf nodes of the protocol tree and ending at the root node. During the alignment process, gaps in each sequence are identified, resulting in a standard-aligned data sequence with gaps.

[0132] For text protocols, the fields are divided according to delimiters to obtain several text character segments, with the field length being the data length between two delimiters; for binary protocols, the fields are divided according to the rate of change, and the standard-aligned data sequence obtained in the previous step is taken.

[0133] S211: Divide the unknown protocol data sequence into fields to obtain the protocol fields.

[0134] The change rate is calculated byte by byte. Bytes with a change rate close to 0 form a static field. The data sequence is divided with the static field as the boundary, resulting in a series of concatenated sequences of static and dynamic fields. The length of the field is the length of the corresponding data block after the division.

[0135] S212: Perform semantic recognition on protocol fields to extract features of unknown protocols.

[0136] Semantic recognition of fields includes analyzing the semantics that need to be recognized and designing different recognition methods for different fields. Protocols have a common format framework, that is, one or more important fields such as protocol number, sequence number, length, checksum, and text field constitute the common framework of the protocol. For other custom fields of unknown protocols, the control information they carry will be maintained according to the data generation rules, thereby ensuring the integrity of the protocol information.

[0137] S213: Name the unknown protocols according to their characteristics, store the named unknown protocols in the protocol knowledge base, and build a new protocol knowledge base.

[0138] Based on the characteristics of unknown protocols and machine learning models, features of unknown protocols are extracted and named, and the named unknown protocols are stored in the protocol knowledge base to construct a new protocol knowledge base.

[0139] The protocol knowledge base construction method based on machine learning provided in this embodiment acquires multiple known protocol specification files and uses natural language processing (NLP) to decompose each known protocol specification file, obtaining the known protocol feature information corresponding to each file. Based on this feature information, the known protocol specification files are extracted and categorized to obtain known protocol specification files with classification identifiers. A protocol knowledge base is constructed based on these categorized files and the known protocol feature information. Network environment traffic data is acquired, and HsMM is used to parse and process the data to obtain the keywords and protocol field lengths corresponding to the traffic data. Based on the keywords and protocol field lengths, the basic protocol information and protocol timing are determined. A static protocol matrix is ​​obtained by statically comparing information with a protocol knowledge base. A dynamic protocol matrix is ​​obtained by dynamically comparing protocol time sequences with the protocol knowledge base. Based on the static and dynamic protocol matrices, unknown protocols in network traffic data are identified. The unknown protocols are sequence parsed, and the empty spaces in each sequence are marked to obtain a standard-aligned unknown protocol data sequence with empty spaces. The unknown protocol data sequence is divided into fields to obtain protocol fields. Semantic recognition is performed on the protocol fields to extract the features of the unknown protocols. The unknown protocols are named according to their features and stored in the protocol knowledge base to construct a new protocol knowledge base. This method expands the protocol knowledge base by discovering unknown protocols in reverse, thereby improving the security and reliability of network communication.

[0140] Figure 3 is a schematic diagram of the structure of the protocol knowledge base construction device based on machine learning provided in this application. As shown in Figure 3, the protocol knowledge base construction device 300 based on machine learning provided in this embodiment includes:

[0141] Module 301 is used to acquire network environment traffic data;

[0142] Processing module 302 is used to input network environment traffic data into the traffic detection machine learning model for parsing and processing, so as to obtain the protocol parsing data corresponding to the network environment traffic data;

[0143] Processing module 302 is also used to compare the protocol parsing data with the protocol knowledge base to filter out unknown protocols in the network environment traffic data;

[0144] The processing module 302 is also used to perform protocol parsing on unknown protocols using a traffic detection machine learning model and extract features of unknown protocols;

[0145] The determination module 303 is used to name the unknown protocol according to its characteristics, store the named unknown protocol in the protocol knowledge base, and build a new protocol knowledge base.

[0146] Optionally, module 301 can also be used to acquire multiple known protocol specification files;

[0147] The processing module 302 is also used to decompose each known protocol specification file using natural language parsing technology to obtain the known protocol feature information corresponding to each known protocol specification file;

[0148] The processing module 302 is also used to extract and classify known protocol description files based on known protocol feature information to obtain known protocol description files with classification identifiers;

[0149] The determination module 303 is used to construct a protocol knowledge base based on known protocol description documents with classification identifiers and known protocol feature information.

[0150] Optionally, the processing module 302 is also used to parse and process network environment traffic data based on HsMM to obtain the keyword and protocol field length corresponding to the network environment traffic data;

[0151] The determination module 303 is also used to determine the basic protocol information and protocol timing based on the keywords and protocol field lengths;

[0152] The protocol parsing data is compared with the protocol knowledge base to filter out unknown protocols in the network traffic data, including:

[0153] The processing module 303 is specifically used to compare protocol basic information, protocol timing and protocol knowledge base to filter out unknown protocols in network environment traffic data.

[0154] Optionally, the processing module 302 is also used to perform a static comparison between the basic protocol information and the protocol knowledge base to obtain a static protocol matrix. The static protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is statically compared.

[0155] Processing module 302 is also used to dynamically compare the protocol timing with the protocol knowledge base to obtain a dynamic protocol matrix. The dynamic protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is dynamically compared.

[0156] The determination module 303 is also used to determine unknown protocols in network environment traffic data based on the static protocol matrix and the dynamic protocol matrix.

[0157] Optionally, the processing module 302 is also used to perform sequence parsing on the unknown protocol and identify the empty spaces in each sequence to obtain a standard-aligned unknown protocol data sequence with empty spaces.

[0158] The processing module 302 is also used to divide the unknown protocol data sequence into fields to obtain protocol fields;

[0159] The processing module 302 is also used to perform semantic recognition on the protocol fields, thereby extracting features of unknown protocols.

[0160] Optionally, the machine learning-based protocol knowledge base construction device also includes: a judgment module 304;

[0161] The judgment module 304 is used to determine whether each value in the static protocol matrix is ​​less than a first preset value and whether each value in the dynamic protocol matrix is ​​less than a second preset value;

[0162] The determination module 303 is further configured to determine that when each value in the static protocol matrix is ​​less than a first preset value and each value in the dynamic protocol matrix is ​​less than a second preset value, the protocol corresponding to the static protocol matrix and the dynamic protocol matrix is ​​an unknown protocol in the network environment traffic data.

[0163] Figure 4 is a schematic diagram of the structure of the protocol knowledge base construction device based on machine learning provided in this application. As shown in Figure 4, this application provides a protocol knowledge base construction device based on machine learning. The protocol knowledge base construction device 400 based on machine learning includes: a receiver 401, a transmitter 402, a processor 403, and a memory 404.

[0164] Receiver 401 is used to receive instructions and data;

[0165] Transmitter 402 is used to send commands and data;

[0166] Memory 404 is used to store instructions executed by the computer;

[0167] Processor 403 is used to execute computer execution instructions stored in memory 404 to implement the various steps performed by the machine learning-based protocol knowledge base construction method in the above embodiments. For details, please refer to the relevant descriptions in the foregoing embodiments of the machine learning-based protocol knowledge base construction method.

[0168] Alternatively, the memory 404 can be either standalone or integrated with the processor 403.

[0169] When the memory 404 is set up independently, the electronic device also includes a bus for connecting the memory 404 and the processor 403.

[0170] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the machine learning-based protocol knowledge base construction method performed by the aforementioned machine learning-based protocol knowledge base construction device.

[0171] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0172] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0173] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0174] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0175] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0176] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0177] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as these combinations of technical features do not contradict each other, they should be considered within the scope of this specification.

[0178] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0179] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for constructing a protocol knowledge base based on machine learning, characterized in that, The method includes: Acquire network environment traffic data and input the network environment traffic data into a traffic detection machine learning model for parsing and processing to obtain the protocol parsing data corresponding to the network environment traffic data; The protocol parsing data is compared with the protocol knowledge base to filter out unknown protocols in the network environment traffic data. The unknown protocol is parsed using a traffic detection machine learning model to extract its features. The unknown protocol is then named based on these features and stored in the protocol knowledge base to construct a new protocol knowledge base.

2. The method according to claim 1, characterized in that, Before acquiring network environment traffic data, the method further includes: Multiple known protocol specification files are obtained, and natural language parsing technology is used to decompose each known protocol specification file to obtain the known protocol feature information corresponding to each known protocol specification file; Based on the known protocol feature information, the known protocol specification files are extracted and categorized to obtain known protocol specification files with classification identifiers; A protocol knowledge base is constructed based on known protocol description documents with classification identifiers and the characteristic information of the known protocols.

3. The method according to claim 1, characterized in that, The traffic detection machine learning model includes HsMM, and the protocol parsing data includes basic protocol information and protocol timing. The process of inputting the network environment traffic data into the traffic detection machine learning model for parsing and processing to obtain the protocol parsing data corresponding to the network environment traffic data includes: Based on HsMM, the network environment traffic data is parsed and processed to obtain the keywords and protocol field lengths corresponding to the network environment traffic data. Based on the keywords and the protocol field length, the basic protocol information and the protocol timing are determined; The step of comparing the protocol parsing data with a protocol knowledge base to filter out unknown protocols in the network environment traffic data includes: The protocol basic information, the protocol timing, and the protocol knowledge base are compared to identify unknown protocols in the network environment traffic data.

4. The method according to claim 3, characterized in that, The protocol comparison includes static comparison and dynamic comparison. The step of comparing the basic protocol information, the protocol timing data, and the protocol knowledge base to filter out unknown protocols in the network traffic data includes: The basic information of the protocol is statically compared with the protocol knowledge base to obtain a static protocol matrix. The static protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is statically compared. The protocol time sequence is dynamically compared with the protocol knowledge base to obtain a dynamic protocol matrix. The dynamic protocol matrix is ​​used to indicate the similarity matrix obtained after the protocol parsing data is dynamically compared. Based on the static protocol matrix and the dynamic protocol matrix, the unknown protocols in the network environment traffic data are determined.

5. The method according to claim 1, characterized in that, The step of using a traffic detection machine learning model to parse the unknown protocol and extract its features includes: The unknown protocol is parsed sequentially, and the empty spaces in each sequence are identified to obtain a standard-aligned unknown protocol data sequence with empty spaces; The unknown protocol data sequence is divided into fields to obtain protocol fields; Semantic recognition is performed on the protocol fields to extract the features of the unknown protocol.

6. The method according to claim 4, characterized in that, The step of determining the unknown protocols in the network environment traffic data based on the static protocol matrix and the dynamic protocol matrix includes: Determine whether each value in the static protocol matrix is ​​less than a first preset value and whether each value in the dynamic protocol matrix is ​​less than a second preset value; When each value in the static protocol matrix is ​​less than a first preset value and each value in the dynamic protocol matrix is ​​less than a second preset value, the protocols corresponding to the static protocol matrix and the dynamic protocol matrix are unknown protocols in the network environment traffic data.

7. A protocol knowledge base construction device based on machine learning, characterized in that, include: The acquisition module is used to acquire network traffic data. The processing module is used to input the network environment traffic data into the traffic detection machine learning model for parsing and processing, so as to obtain the protocol parsing data corresponding to the network environment traffic data. The processing module is also used to compare the protocol parsing data with the protocol knowledge base to filter out unknown protocols in the network environment traffic data. The processing module is also used to perform protocol parsing on the unknown protocol using a traffic detection machine learning model and extract the features of the unknown protocol. The determination module is used to name the unknown protocol according to the characteristics of the unknown protocol, store the named unknown protocol in the protocol knowledge base, and construct a new protocol knowledge base.

8. The apparatus according to claim 7, characterized in that, include: The processing module is also used to perform sequence parsing on the unknown protocol and identify the empty spaces in each sequence to obtain a standard-aligned unknown protocol data sequence with empty spaces. The processing module is also used to divide the unknown protocol data sequence into fields according to the delimiter to obtain protocol fields; The processing module is also used to perform semantic recognition on the protocol field, thereby extracting the features of the unknown protocol.

9. A protocol knowledge base construction device based on machine learning, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

Images