Binary protocol state extraction method, system, storage medium and electronic device

Abstract

The present invention relates to the technical field of protocol reverse engineering, and in particular to a binary protocol state extraction method, system, storage medium and electronic device. The method comprises: preprocessing each binary protocol data to be extracted to obtain multiple binary protocol data with the same standard data format; using a trained autoencoder model to obtain feature information of each binary protocol data; using an Unified Mapping (UMAP) dimensionality reduction algorithm to obtain two-dimensional feature information corresponding to each feature information; in the absence of prior knowledge, using an improved DBSCAN clustering algorithm to cluster all the two-dimensional feature information, accurately dividing the same-state data of a stateful protocol into a cluster, then constructing a state machine based on the multiple clusters, combining grammatical analysis and semantic analysis of the stateful protocol, obtaining the specific content of each binary protocol data to be extracted, and also deducing the specific format of the stateful protocol.

Description

Technical Field

[0001] The present invention relates to the technical field of protocol reverse engineering, and in particular to a binary protocol state extraction method, system, storage medium and electronic equipment. Background Art

[0002] Ensuring the efficient and accurate transmission of massive amounts of information on the internet requires that communicating parties adhere to certain pre-agreed regulations, known as network protocols, which are essentially rules. With the increasing refinement of protocol application areas and the growing emphasis on network security, the number of network protocols has grown dramatically. This includes not only the expansion of existing protocols but also the creation of entirely new protocols. Analyzing the burgeoning number of network protocols is a crucial task in cyberspace security and essential for secure development in the internet age. Network protocol analysis can be categorized as standard protocols, proprietary protocols, and unknown protocols, depending on the target of analysis. In specific fields such as industrial control, military communications, and financial information, unknown protocols are widely used, making it difficult to understand the specific data, content, and meaning of the protocols.

[0003] Protocol analysis technology primarily involves identifying and analyzing known protocols and reverse engineering unknown protocols. Known protocols typically have publicly available specifications and standards, and can be identified and analyzed through protocol features such as format characteristics and port numbers. Unknown protocols, on the other hand, are difficult to identify and analyze due to their unknown specifications. Protocol reverse engineering, which involves analyzing protocol messages or execution processes to derive relevant features without prior knowledge, is crucial for analyzing unknown protocols. Most existing protocol analysis techniques require a certain level of understanding of protocol specifications and rely on manual knowledge, making them unsuitable for proprietary or unknown protocols.

[0004] Protocol-related features include the three elements of the protocol, namely syntax, semantics, and state machine. Among them, the state machine describes the transition rules between different states of the stateful protocol. The protocol state machine reversal is an indispensable part of the reverse analysis of the stateful protocol. Usually, the protocol state machine can be constructed based on the inference algorithm based on the clustering results. The existing technology only includes the protocol-level differentiation of mixed message data. How to efficiently, accurately and without relying on artificial knowledge to complete the differentiation of different state data messages of the same protocol remains to be achieved. The patent "A Mixed Data Frame Clustering of Binary Protocols under Zero Knowledge" (Publication No.: CN114722961A) efficiently distinguishes the mixed data of binary protocols under zero knowledge, but this method only completes the protocol identification and does not take into account the differences between different state messages of the specific protocol. At the same time, the K-Means-based clustering algorithm needs to provide cluster values. When the protocol specification is unknown, the number of its states is also unclear, which greatly reduces the accuracy of protocol state clustering. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and provide a binary protocol state extraction method, system, storage medium and electronic device.

[0006] The technical solution of a binary protocol state extraction method of the present invention is as follows:

[0007] Preprocessing each binary protocol data to be extracted to obtain a plurality of binary protocol data having the same standard data format, wherein all the binary protocol data to be extracted are based on the same stateful protocol;

[0008] Using the trained autoencoder model, the feature information of each binary protocol data is obtained;

[0009] Use the UMAP dimensionality reduction algorithm to reduce the dimensionality of each feature information to obtain the two-dimensional feature information corresponding to each feature information;

[0010] All two-dimensional feature information is clustered using an improved DBSCAN clustering algorithm, and the same-state data of the stateful protocol is divided into a cluster to obtain multiple clusters.

[0011] The beneficial effects of a binary protocol state extraction method of the present invention are as follows:

[0012] In the absence of prior knowledge, the improved DBSCAN clustering algorithm is used to cluster all two-dimensional feature information, which can accurately divide the same-state data of the stateful protocol into a cluster. Then, a state machine can be constructed based on multiple clusters. Combined with the grammatical analysis and semantic analysis of the stateful protocol, the specific content of each binary protocol data to be extracted can be obtained, and the specific format of the stateful protocol can be deduced, that is, the unknown stateful protocol can be turned into a known one.

[0013] The technical solution of a binary protocol state extraction system of the present invention is as follows:

[0014] Including preprocessing module, acquisition module, dimensionality reduction processing module and clustering module;

[0015] The preprocessing module is used to preprocess each binary protocol data to be extracted to obtain a plurality of binary protocol data having the same standard data format, wherein all the binary protocol data to be extracted are based on the same stateful protocol;

[0016] The acquisition module is used to: obtain feature information of each binary protocol data using the trained autoencoder model;

[0017] The dimensionality reduction processing module is used to: perform dimensionality reduction processing on each feature information using the UMAP dimensionality reduction algorithm to obtain two-dimensional feature information corresponding to each feature information;

[0018] The clustering module is used to cluster all two-dimensional feature information using an improved DBSCAN clustering algorithm, and divide the same-state data of the stateful protocol into a cluster to obtain multiple clusters.

[0019] The beneficial effects of a binary protocol state extraction system of the present invention are as follows:

[0020] By clustering all two-dimensional feature information using the improved DBSCAN clustering algorithm, the same-state data of the stateful protocol can be accurately divided into a cluster. Then, a state machine can be constructed based on multiple clusters. Combined with the grammatical analysis and semantic analysis of the stateful protocol, the specific content of each binary protocol data to be extracted can be obtained, and the specific format of the stateful protocol can be deduced, that is, the unknown stateful protocol can be turned into a known one.

[0021] A storage medium of the present invention stores instructions, and when a computer reads the instructions, the computer is caused to execute any one of the above-mentioned binary protocol state extraction methods.

[0022] An electronic device of the present invention includes a processor and the above-mentioned storage medium, wherein the processor executes instructions in the storage medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of a binary protocol state extraction method according to an embodiment of the present invention;

[0024] Figure 2 Schematic diagram of the structure of the autoencoder model;

[0025] Figure 3 The figure is a structural diagram of a binary protocol state extraction system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] like Figure 1 As shown, a binary protocol state extraction method according to an embodiment of the present invention includes the following steps:

[0027] S1. Preprocess each binary protocol data to be extracted to obtain a plurality of binary protocol data having the same standard data format, wherein all the binary protocol data to be extracted are based on the same stateful protocol;

[0028] Among them, binary protocol data refers to network traffic data represented in binary;

[0029] Different stateful protocols have different protocol states. Binary protocol states refer to the different states of stateful protocols.

[0030] S2. Use the trained autoencoder model to obtain the feature information of each binary protocol data;

[0031] Feature information specifically refers to reducing the dimensionality of the binary protocol data to obtain more concise and simple features for representing the binary protocol data, which can be represented in matrix form. In other words, the trained autoencoder model compresses the binary protocol data to extract the most representative information, namely the feature information, which can be represented in matrix form. The purpose of the trained autoencoder model is to reduce the dimensionality of the input information, namely the binary protocol data, while ensuring that important features of the binary protocol data are not lost.

[0032] S3, using the UMAP dimensionality reduction algorithm to perform dimensionality reduction processing on each feature information to obtain two-dimensional feature information corresponding to each feature information;

[0033] Among them, since the data dimension of the feature information of each binary protocol data is still relatively high, the UMAP dimensionality reduction algorithm is used to reduce the dimension of each feature information to obtain feature information with two-dimensional data, that is, two-dimensional feature information;

[0034] S4. Use the improved DBSCAN clustering algorithm to cluster all two-dimensional feature information, divide the same-state data of the stateful protocol into one cluster, and obtain multiple clusters.

[0035] The same-state data of a stateful protocol refers to protocol data in the stateful protocol that has similar characteristics (such as format, etc.), that is, is in the same state.

[0036] In the absence of prior knowledge, the improved DBSCAN clustering algorithm is used to cluster all two-dimensional feature information, which can accurately divide the same-state data of the stateful protocol into a cluster. Then, a state machine can be constructed based on multiple clusters. Combined with the grammatical analysis and semantic analysis of the stateful protocol, the specific content of each binary protocol data to be extracted can be obtained, and the specific format of the stateful protocol can be deduced, that is, the unknown stateful protocol can be turned into a known one.

[0037] Optionally, in the above technical solution, the training process of the trained autoencoder model includes:

[0038] Preprocessing each sample binary protocol data to obtain a plurality of sample binary protocol data having the same standard data format, and dividing the data into a training set and an evaluation set, that is, dividing all sample binary protocol data having the same standard data format into a training set and an evaluation set;

[0039] Build an autoencoder model, set the loss function of the autoencoder model, and train the autoencoder model based on the training set and evaluation set to obtain a trained autoencoder model.

[0040] Optionally, in the above technical solution, constructing an autoencoder model includes:

[0041] Set the number of dense layers for the encoder of the autoencoder model, and set the number of dense layers for the decoder of the autoencoder model;

[0042] Set the input data dimensions and output data dimensions of each dense layer of the encoder of the autoencoder model, and set the input data dimensions and output data dimensions of each dense layer of the decoder of the autoencoder model.

[0043] Optionally, in the above technical solution, all sample binary protocol data having the same standard data format are divided into a training set and an evaluation set, including:

[0044] All sample binary protocol data having the same standard data format are randomly divided into a training set and an evaluation set.

[0045] The following is a detailed description of a binary protocol state extraction method of the present invention through an embodiment, which specifically includes:

[0046] S10. Preprocess the binary protocol data of each sample to obtain a training set and an evaluation set, specifically including S100 to S101:

[0047] S100: Preprocess each binary protocol data to be extracted to obtain a plurality of binary protocol data having the same standard data format. Specifically:

[0048] Read the protocol data file in pcap format, process the protocol data file, obtain multiple binary protocol data, all of which are used as sample binary protocol data, remove irrelevant data in each sample binary protocol data, such as excessively long payload data in the protocol data file, and align all sample binary protocol data to the same given length l, for example, l = 128 or l = 256. The specific value of the given length l can be set according to actual conditions. Sample binary protocol data with insufficient length is left-aligned and padded with zeros, thereby completing the preprocessing of each sample binary protocol data to obtain a protocol data sequence X = {X1, X2, X3, ..., Xn}, where X1, X2, X3, …, X n They respectively represent a sample binary protocol data with the same standard data format, as shown in Table 1.

[0049] Table 1:

[0050] Sample binary protocol data with the same canonical data format <![CDATA[X1]]> 101000…101100,l＝256 <![CDATA[X2]]> 101000…010000,l＝256 <![CDATA[X3]]> 101000…100101,l＝256 … … <![CDATA[X n ]]> 101000…111010,l＝256

[0051] In Table 1, the protocol data sequence is all the binary protocol data after preprocessing, with uniform length, the same standard data format, and stored in a list;

[0052] S101. All sample binary protocol data with the same standard data format are randomly divided into a training set and an evaluation set, and each sample binary protocol data with the same standard data format is marked as preprocessed sample binary protocol data. Then:

[0053] Given the extraction ratio R∈[0,1], randomly generate a real number r in the range [0,1]. When r>R, add the first preprocessed sample binary protocol data X1 to the evaluation set X eval Otherwise, add training set X train , until each preprocessed sample binary protocol data is randomly divided into the training set X train and the evaluation set X eval middle.

[0054] S11, constructing an autoencoder model and training it to obtain a trained autoencoder model, specifically including S110 to S112, such as Figure 2 shown.

[0055] S110, construct an encoder of the autoencoder model, the number of dense layers is 4, marked as dense, dense_1, dense_2 and dense_3 respectively, the input data dimension of the dense layer dense is (None, 256), the output data dimension of the dense layer dense is (None, 128), and the protocol data sequence X = {X1, X2, X3, ..., X n Any preprocessed sample binary protocol data in} is passed through the dense layer dense to obtain a sequence h with a data dimension of (None, 128) e1 , h e1 =relu(W1x+b1); x represents any pre-processed sample binary protocol data;

[0056] The input data dimension of dense layer dense_1 is (None, 128), and the output data dimension of dense layer dense_1 is (None, 64). e1 After passing through the dense layer dense_1, we get a sequence h with a data dimension of (None, 64) e2 , h e2 =relu(W2h e1 +b2);

[0057] The input data dimension of dense layer dense_2 is (None, 64), and the output data dimension of dense layer dense_2 is (None, 32). e2 After passing through the dense layer dense_2, we get a sequence h with a data dimension of (None, 32) e3 , h e3 =relU(W3h e2 +b3);

[0058] The input data dimension of dense layer dense_3 is (None, 32), and the output data dimension of dense layer dense_3 is (None, 5). e3 After passing through the dense layer dense_3, we get a sequence h with a data dimension of (None, 5), h = σ(W4h e3 +b x );

[0059] Among them, None represents the number of preprocessed sample binary protocol data, that is, None = n, W1, W2, W3, W4 represent the weights of each dense layer of the encoder, and b1, b2, b3, b4 represent the partial derivatives of each dense layer of the encoder.

[0060] S111. Construct a decoder with 4 dense layers, labeled dense_4, dense_5, dense_6, and dense_7.

[0061] The input data dimension of dense layer dense_4 is (None, 5), and the output data dimension of dense layer dense_4 is (None, 32). After the sequence h with data dimension (None, 5) passes through dense layer dense_4, the sequence h with data dimension (None, 32) is obtained. d1 , h d1 =relu(W1′h+b1′);

[0062] The input data dimension of dense layer dense_5 is (None, 32), and the output data dimension of dense layer dense_5 is (None, 64). d1 After passing through the dense layer dense_5, we get a sequence h with a data dimension of (None, 64) d2 , h d2 =relu(W2′h d1 +b2′);

[0063] The input data dimension of dense layer dense_6 is (None, 64), and the output data dimension of dense layer dense_6 is (None, 128). d2 After passing through the dense layer dense_6, we get a sequence h with a data dimension of (None, 128) d3 , h d3 =relu(W3′h d2 +b3′);

[0064] The input data dimension of dense layer dense_7 is (None, 128), and the output data dimension of dense layer dense_7 is (None, 256). d3 After passing through the dense layer dense_7, we get a sequence Y with a data dimension of (None, 256), Y=relu(w4′h d3 +b4′), thus obtaining the prediction result sequence Y={Y1, Y2, Y3, ..., Y n}, where Y1 represents the prediction result of the first preprocessed sample binary protocol data X1, and Y2 represents the prediction result of the second preprocessed sample binary protocol data X2;

[0065] Among them, W1′, W2′, W3′, W4′ represent the weights of each dense layer of the decoder, and b1′, b2′, b3′, b4′ represent the partial derivatives of each dense layer of the decoder.

[0066] S112. Select the loss function loss for the autoencoder model:

[0067]

[0068] Among them, w i >0, i represents a positive integer, 1≤i≤n;

[0069] S12. Conduct training, specifically:

[0070] Using the training set Xtrain Train the autoencoder model to obtain the initial autoencoder model and evaluate the set X eval Apply it to the initial autoencoder model and judge the accuracy of the prediction results based on the loss function results. If it meets the expectations, the trained autoencoder model is obtained;

[0071] S13. Preprocess each binary protocol data to be extracted to obtain multiple binary protocol data with the same standard data format, wherein all binary protocol data to be extracted are based on the same stateful protocol. The preprocessing process refers to the above S100 and is not described here. The trained autoencoder model is used to obtain the feature information of each binary protocol data. The sequence h with a data dimension of (None, 5) corresponding to any binary protocol data to be extracted is the feature information of the binary protocol data to be extracted. The feature information of each binary protocol data is obtained, and a feature sequence set F = {f1, f2, f3, ..., f N}, where f1 represents the feature information corresponding to the first binary protocol data to be extracted, f2 represents the feature information corresponding to the second binary protocol data to be extracted, ..., f N Indicates the feature information corresponding to the Nth binary protocol data to be extracted, f1, f2, f3, ..., f N The data dimension is (None, 5), where N represents the number of binary protocol data to be extracted, as shown in Table 2.

[0072] Table 2:

[0073] Feature information corresponding to the binary protocol data to be extracted <![CDATA[f1]]> 3.83029e-01, 9.09113e-01, 1.82446e-01, 8.85571e-01, 9.99926e-01 <![CDATA[f2]]> 6.25224e-01, 9.87998e-01, 6.84445e-01, 8.61275e-01, 9.99974e-01 <![CDATA[f3]]> 9.96169e-01, 2.18363e-02, 7.53869e-01, 4.51517e-01, 9.97260e-01 … … <![CDATA[f N ]]> 5.25830e-01, 3.19561e-01, 1.64232e-01, 3.94478e-01, 9.97930e-01

[0074] S14, using the UMAP dimensionality reduction algorithm to perform dimensionality reduction processing on each feature information to obtain two-dimensional feature information corresponding to each feature information;

[0075] S140, select the parameter neighbors∈[5,50], which represents the number of neighboring points in the local approximation of the manifold structure. The larger the value, the more global structure is preserved. The default value is in the range [10,15].

[0076] S141. Select the parameter minDist∈[0.001, 0.5], which represents the density of embedding. The smaller the value, the more accurate the algorithm is in optimizing the local structure. The default value is 0.1.

[0077] S142, select the parameter metric, which represents the metric used to measure the distance in the input space;

[0078] S143, apply the unified manifold approximation and projection algorithm (UMAP algorithm) to reduce the dimension, use the UMAP package that depends on scikit-learn, select the default parameters, and input the feature sequence set F obtained in S13 = {f1, f2, f3, ..., f N}, the output is the feature sequence set F′={f1', f2', f3', ..., f N '}, f1', f2', f3', ..., f N The data dimension of ' is (None, 2), f1' is the two-dimensional protocol feature information corresponding to f1, f2' is the two-dimensional protocol feature information corresponding to f2, ..., f N 'It is f N The corresponding two-dimensional protocol feature information is shown in Table 3.

[0079] Table 3:

[0080] Two-dimensional protocol feature information <![CDATA[f1’]]> 7.28205442e-01,-4.56155315e-02 <![CDATA[f2’]]> 3.81078035e-01,-4.68073517e-01 <![CDATA[f3’]]> -5.01878381e-01, 1.42931357e-01 … … <![CDATA[f N ’]]> 1.25205293e-01, 4.20141697e-01

[0081] S15. Cluster all two-dimensional feature information using the improved DBSCAN clustering algorithm, and divide the same-state data of the stateful protocol into one cluster to obtain multiple clusters. Specifically:

[0082] S150, select parameter ε, representing the scanning radius, parameter minPts, representing the minimum number of included points, and select parameter dim=2, representing that the output data is two-dimensional;

[0083] S151, initialize cluster set Cluster number cou nt=0, scanned point set

[0084] Define the object c in the cluster set, including parameters name, dim, points = {p1, p2, p3, ..., p n′}, name represents the name of the cluster, dim represents the dimension of the cluster, points represents the points contained in the cluster, and n′ represents the number of points representing the cluster;

[0085] S152, input feature sequence set F′={f1′, f2′, f3′, …, f n '}, choose any unvisited point f, where f represents f1', f2', f3', ..., f n Any one of ', and satisfies Find its neighborhood N ε (f), contains all points whose Euclidean distance d(f, p) to the unvisited point f is less than or equal to ε;

[0086] N ε (f)={f∈F′|d(f, p)≤ε}

[0087] S153. Initialize a cluster object

[0088] S154. Obtain the neighborhood N of the unvisited point f. ε (f) The number of neighboring points |N ε (f)|, specifically the following two situations:

[0089] 1) The first case: If |N ε (f)|<minPts, then add the unvisited point f to the cluster object c initialized in S153 noise In, c(name='Noise', dim=2, points={p1, p2, p3,..., p n′}), at the same time, mark the unvisited point f as a visited point;

[0090] 2) The second case: If |N ε (f)|≥minPts, initialize a cluster object c cou nt And update the number of clusters cou nt;

[0091] c cou nt (name=count, dim=2, points={p1, p2, p3,…,p n′})

[0092] S155: When the second condition in S154 is met, scan N ε (f) = {p1, p2, p3, ..., p n′} for all points p in the array, get the neighborhood N of point p ε (p) and the number of its neighbors |N ε (p)|, choose any unvisited core point p, satisfying And;|N ε (p)|≥minPts, initialize the core spacing d core =1;

[0093] Calculate the core point distance d core_new =d(f, p), given the threshold value of the change rate of the core point spacing T r =5, calculate the core point spacing change rate r,

[0094] If r>T r , no processing is performed on the point p;

[0095] If r≤T r , add point p to the scan point set visited = {p1, p2, p3, ..., p n′}, at the same time, all its neighboring points n∈N ε(p) Add the initialized cluster object c in the second case in S154 count ;

[0096] S156, scan the existing cluster object c cou nt If point p is neither in the current cluster nor in other clusters, add point p to the initialized cluster object c in the second case in S154. cou nt ;

[0097] S157, repeat steps S154 to S156 until all points in the input sequence set F' are scanned, that is, the scanned point set visited contains all input data, visited = F' = {f1', f2', f3', ..., f n '}, to determine {f1', f2', f3', ..., f n '} are all scanned, and the clustering is ended. At this time, the same state data of the stateful protocol is divided into a cluster, and multiple clusters are obtained, that is, multiple cluster objects c are obtained. cou nt .

[0098] Among them, the acquisition process of the improved DBSCAN clustering algorithm:

[0099] By adding two parameters, the change rate threshold and the core point spacing change rate, to the existing DBSCAN clustering algorithm, an improved DBSCAN clustering algorithm is obtained.

[0100] The beneficial effects of the present invention are as follows:

[0101] 1) The present invention identifies different states of stateful protocols without prior knowledge and manual assistance. It can be applied to standard protocols as well as proprietary and unknown protocols whose specifications are not publicly available or clearly defined.

[0102] 2) The existing technology only focuses on how to distinguish different protocol messages, i.e., binary protocol data, from mixed protocol frames. The present invention proposes a state classification method for stateful protocols to distinguish different states from a stateful protocol message, i.e., binary protocol data;

[0103] 3) The present invention relates to the steps of the clustering algorithm. In view of the shortcoming that the DBSCAN algorithm is not sensitive enough to points with similar density during clustering, the algorithm content is improved so that binary protocol data of different states in areas with similar density can be marked as different clusters.

[0104] In the above embodiments, although the steps are numbered S1, S2, etc., these are only specific embodiments given in this application. Those skilled in the art can adjust the execution order of S1, S2, etc. according to actual conditions, which is also within the scope of protection of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0105] like Figure 3 As shown, a binary protocol state extraction system 200 according to an embodiment of the present invention includes a pre-processing module 210, an acquisition module 220, a dimensionality reduction processing module 230 and a clustering module 240;

[0106] The preprocessing module 210 is used to preprocess each binary protocol data to be extracted to obtain a plurality of binary protocol data having the same standard data format, wherein all the binary protocol data to be extracted are based on the same stateful protocol;

[0107] The acquisition module 220 is used to: obtain feature information of each binary protocol data using the trained autoencoder model;

[0108] The dimensionality reduction processing module 230 is used to: perform dimensionality reduction processing on each feature information using the UMAP dimensionality reduction algorithm to obtain two-dimensional feature information corresponding to each feature information;

[0109] The clustering module 240 is used to cluster all two-dimensional feature information using an improved DBSCAN clustering algorithm, and divide the same-state data of the stateful protocol into a cluster to obtain multiple clusters.

[0110] By clustering all two-dimensional feature information using the improved DBSCAN clustering algorithm, the same-state data of the stateful protocol can be accurately divided into a cluster. Then, a state machine can be constructed based on multiple clusters. Combined with the grammatical analysis and semantic analysis of the stateful protocol, the specific content of each binary protocol data to be extracted can be obtained, and the specific format of the stateful protocol can be deduced, that is, the unknown stateful protocol can be turned into a known one.

[0111] Optionally, the above technical solution further includes a training module, which is used to:

[0112] Preprocessing each sample binary protocol data to obtain multiple sample binary protocol data with the same standard data format, and dividing them into a training set and an evaluation set;

[0113] Build an autoencoder model, set the loss function of the autoencoder model, and train the autoencoder model based on the training set and evaluation set to obtain a trained autoencoder model.

[0114] Optionally, in the above technical solution, the process of constructing the autoencoder model by the training module includes:

[0115] Set the number of dense layers for the encoder of the autoencoder model, and set the number of dense layers for the decoder of the autoencoder model;

[0116] Set the input data dimensions and output data dimensions of each dense layer of the encoder of the autoencoder model, and set the input data dimensions and output data dimensions of each dense layer of the decoder of the autoencoder model.

[0117] Optionally, in the above technical solution, the process of the training module dividing all sample binary protocol data having the same standard data format into a training set and an evaluation set includes:

[0118] All sample binary protocol data having the same standard data format are randomly divided into a training set and an evaluation set.

[0119] Regarding the various parameters and steps of implementing corresponding functions of each unit module in the binary protocol state extraction system 200 of the present invention, reference may be made to the various parameters and steps in the embodiment of the binary protocol state extraction method above, and no further details will be given here.

[0120] A storage medium according to an embodiment of the present invention stores instructions, and when a computer reads the instructions, the computer is caused to execute any one of the above-mentioned binary protocol state extraction methods.

[0121] An electronic device according to an embodiment of the present invention includes a processor and the aforementioned storage medium, wherein the processor executes instructions in the storage medium.

[0122] Those skilled in the art will appreciate that the present invention may be implemented as a system, method or computer program product.

[0123] Therefore, the present disclosure may be embodied in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present disclosure may be embodied in the form of a computer program product embodied in one or more computer-readable media, wherein the computer-readable media contains computer-readable program code.

[0124] Any combination of one or more computer-readable media can be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device.

[0125] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A binary protocol state extraction method, characterized in that: include: Preprocessing each binary protocol data to be extracted to obtain a plurality of binary protocol data having the same standard data format, wherein all the binary protocol data to be extracted are based on the same stateful protocol; Using the trained autoencoder model, the feature information of each binary protocol data is obtained; Use the UMAP dimensionality reduction algorithm to reduce the dimensionality of each feature information to obtain the two-dimensional feature information corresponding to each feature information; Using the improved DBSCAN clustering algorithm to cluster all two-dimensional feature information, the same-state data of the stateful protocol is divided into a cluster, and multiple clusters are obtained; Among them, the improved DBSCAN clustering algorithm is used to cluster all two-dimensional feature information, and the same-state data of the stateful protocol is divided into one cluster, resulting in multiple clusters, including: S150, select parameter ε to represent the scanning radius, parameter minPts to represent the minimum number of included points, and select parameter dim=2 to represent that the output data is two-dimensional; S151, initialize cluster set Cluster count = 0, scanned point set Define the object c in the cluster set, including parameters name, dim, points = {p1, p2, p3, ..., p n′ }, name represents the name of the cluster, dim represents the dimension of the cluster, points represents the points contained in the cluster, and n′ represents the number of points representing the cluster; S152, input feature sequence set F′={f1',f2',f3',…,f n '}, choose any unvisited point f, where f represents f1',f2',f3',…,f n Any one of ', and satisfies Find the neighborhood N ε (f), contains all points whose Euclidean distance d(f,p) to the unvisited point f is less than or equal to ε: N ε (f)={f∈F′|d(f,p)≤ε} S153. Initialize a cluster object S154. Obtain the neighborhood N of the unvisited point f. ε (f) The number of neighboring points |N ε (f)|, specifically the following two situations: 1) The first case: If |N ε (f)| < minPts, then add the unvisited point f to the cluster object c initialized in S153 noise where c(name = 'Noise', dim = 2, points = {p1, p2, p3, …, p n′}), and at the same time, mark the unvisited point f as an visited point; 2) The second case: If |N ε (f)|≥minPts, initialize a cluster object c count And update the number of clusters count: c count (name=count,dim=2,points={p1,p2,p3,…,p n′ }) S155: When the second condition in S154 is met, scan N ε (f)={p1,p2,p3,…,p n′ } for all points p in the array, get the neighborhood N of point p ε (p) and the number of its neighbors |N ε (p)|, choose any unvisited core point p, satisfying and|N ε (p)|≥minPts, initialize the core spacing d core =1; Calculate the core point distance d core_new =p(f,p), given the threshold value of the change rate of the core point spacing T r =5, calculate the core point spacing change rate r, If r>T r , no processing is performed on the point p; If r≤T r , add point p to the scan point set visited={p1,p2,p3,…,p n ′}, and at the same time, all its neighboring points n∈N ε (p) Add the initialized cluster object c in the second case in S154 count ; S156, scan the existing cluster object c count If point p is neither in the current cluster nor in other clusters, add point p to the initialized cluster object c in the second case in S154. count ; S157, repeat steps S154 to S156 until all points in the input sequence set F' are scanned, that is, the scanned point set visited contains all input data, visited = F' = {f1', f2', f3', ..., f n '}, to determine {f1',f2',f3',…,f n '} are all scanned, and the clustering is ended. At this time, the same-state data of the stateful protocol is divided into a cluster to obtain multiple clusters.

2. A binary protocol state extraction method according to claim 1, characterized in that: The training process of the trained autoencoder model includes: Preprocessing each sample binary protocol data to obtain a plurality of sample binary protocol data having the same standard data format, and dividing the data into a training set and an evaluation set; Construct an autoencoder model, set a loss function for the autoencoder model, and train the autoencoder model based on the training set and the evaluation set to obtain the trained autoencoder model.

3. A binary protocol state extraction method according to claim 2, characterized in that: Constructing the autoencoder model includes: Setting the number of dense layers of the encoder of the autoencoder model, and setting the number of dense layers of the decoder of the autoencoder model; The input data dimension and the output data dimension of each dense layer of the encoder of the autoencoder model are set, and the input data dimension and the output data dimension of each dense layer of the decoder of the autoencoder model are set.

4. A binary protocol state extraction method according to claim 2, characterized in that: All sample binary protocol data having the same standard data format are divided into a training set and an evaluation set, including: All sample binary protocol data having the same standard data format are randomly divided into a training set and an evaluation set.

5. A binary protocol state extraction system, characterized in that: Including preprocessing module, acquisition module, dimensionality reduction processing module and clustering module; The preprocessing module is used to preprocess each binary protocol data to be extracted to obtain a plurality of binary protocol data having the same standard data format, wherein all the binary protocol data to be extracted are based on the same stateful protocol; The acquisition module is used to: obtain feature information of each binary protocol data using the trained autoencoder model; The dimensionality reduction processing module is used to: perform dimensionality reduction processing on each feature information using the UMAP dimensionality reduction algorithm to obtain two-dimensional feature information corresponding to each feature information; The clustering module is used to: cluster all two-dimensional feature information using an improved DBSCAN clustering algorithm, divide the same-state data of the stateful protocol into a cluster, and obtain multiple clusters; Among them, the improved DBSCAN clustering algorithm is used to cluster all two-dimensional feature information, and the same-state data of the stateful protocol is divided into one cluster, resulting in multiple clusters, including: S150, select parameter ε to represent the scanning radius, parameter minPts to represent the minimum number of included points, and select parameter dim=2 to represent that the output data is two-dimensional; S151, initialize cluster set Cluster count = 0, scanned point set Define the object c in the cluster set, including parameters name, dim, points = {p1, p2, p3, ..., p n′ }, name represents the name of the cluster, dim represents the dimension of the cluster, points represents the points contained in the cluster, and n′ represents the number of points representing the cluster; S152, input feature sequence set F′={f1',f2',f3',…,f n '}, choose any unvisited point f, where f represents f1',f2',f3',…,f n Any one of ', and satisfies Find the neighborhood N ε (f), contains all points whose Euclidean distance d(f,p) to the unvisited point f is less than or equal to ε: N ε (f)={f∈F′|d(f,p)≤ε} S153. Initialize a cluster object S154. Obtain the neighborhood N of the unvisited point f. ε (f) The number of neighboring points |N ε (f)|, specifically the following two situations: 1) The first case: If |N ε (f)| < minPts, then add the unvisited point f to the cluster object c initialized in S153 noise where c(name = 'Noise', dim = 2, points = {p1, p2, p3, …, p n′}), and at the same time, mark the unvisited point f as a visited point; 2) The second case: If |N ε (f)|≥minPts, initialize a cluster object c count And update the number of clusters count: c count (name=count,dim=2,points={p1,p2,p3,…,p n′ }) S155: When the second condition in S154 is met, scan N ε (f)={p1,p2,p3,…,p n′ } for all points p in the array, get the neighborhood N of point p ε (p) and the number of its neighbors |N ε (p)|, choose any unvisited core point p, satisfying and|N ε (p)|≥minPts, initialize the core spacing d cOre =1; Calculate the core point distance d core_new =d(f,p), given the threshold value of the change rate of the core point spacing T r =5, calculate the core point spacing change rate r, If r>T r , no processing is performed on the point p; If r≤T r , add point p to the scan point set visited={p1,p2,p3,…,p n ′}, and at the same time, all its neighboring points n∈N ε (p) Add the initialized cluster object c in the second case in S154 count ; S156, scan the existing cluster object c count If point p is neither in the current cluster nor in other clusters, add point p to the initialized cluster object c in the second case in S154. count ; S157, repeat steps S154 to S156 until all points in the input sequence set F' are scanned, that is, the scanned point set visited contains all input data, visited = F' = {f1', f2', f3', ..., f n '}, to determine {f1',f2',f3',…,f n '} are all scanned, and the clustering is ended. At this time, the same-state data of the stateful protocol is divided into a cluster to obtain multiple clusters.

6. A binary protocol state extraction system according to claim 5, characterized in that: Also included is a training module, the training module being used to: Preprocessing each sample binary protocol data to obtain a plurality of sample binary protocol data having the same standard data format, and dividing the data into a training set and an evaluation set; Construct an autoencoder model, set a loss function for the autoencoder model, and train the autoencoder model based on the training set and the evaluation set to obtain the trained autoencoder model.

7. A binary protocol state extraction system according to claim 6, characterized in that: The process of constructing the autoencoder model by the training module includes: Setting the number of dense layers of the encoder of the autoencoder model, and setting the number of dense layers of the decoder of the autoencoder model; The input data dimension and the output data dimension of each dense layer of the encoder of the autoencoder model are set, and the input data dimension and the output data dimension of each dense layer of the decoder of the autoencoder model are set.

8. A binary protocol state extraction system according to claim 6, characterized in that: The training module divides all sample binary protocol data having the same standard data format into a training set and an evaluation set, including: All sample binary protocol data having the same standard data format are randomly divided into a training set and an evaluation set.

9. A storage medium, characterized in that: The storage medium stores instructions, and when a computer reads the instructions, the computer is caused to execute a binary protocol state extraction method according to any one of claims 1 to 4.

10. An electronic device, characterized in that: The device comprises a processor and the storage medium according to claim 9, wherein the processor executes instructions in the storage medium.

Images