Protocol fuzzing test method and system based on improved SeqGAN

Abstract

The application relates to a protocol fuzzing test method and system based on an improved SeqGAN, and belongs to the technical field of protocol fuzzing tests. The method comprises the following steps: collecting basic data, pre-processing the data, designing a generator G for generating test cases α and discriminators Dgamma and Dlambda respectively used for evaluating the passing rate and diversity of the test cases, designing a hybrid reward, and evaluating the test cases by using the two discriminators based on the hybrid reward; corresponding loss functions are designed for the generator and the two discriminators, and gradient updating and training are performed. The test case generation module of the system is used for executing the protocol fuzzing test method to obtain test cases, the test cases are executed by a fuzzy test execution module, and a log module records test results and intermediate process responses. The application ensures the accuracy of generated sequences, significantly improves the diversity of generated test cases, more effectively covers the boundary conditions of a protocol, and enhances the efficiency and coverage rate of protocol fuzzing tests.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of protocol fuzzing testing, and relates to a protocol fuzzing testing method and system based on an improved SeqGAN. BACKGROUND

[0002] With the rapid development of information technology, network communication protocols have been widely used in various fields of modern society, especially in the fields of Internet of Things devices, industrial control systems, intelligent terminals and cloud computing. As the standard for data transmission, communication protocols ensure reliable communication between network devices. However, with the increasing complexity of network protocols, potential vulnerabilities inevitably exist in protocol design, which can be exploited by malicious attackers, leading to serious security problems such as information leakage and system paralysis. How to effectively detect protocol vulnerabilities is a problem to be solved. Fuzzing, as an effective security testing technique, sends a large amount of random or semi-random data to the system under test, observes its abnormal behavior, and discovers potential vulnerabilities, which has become the mainstream method in protocol security testing.

[0003] Traditional fuzzing has some challenges, such as strong randomness of input data, low test efficiency, lack of targetedness of generated test cases, and difficulty in covering the boundary conditions of the protocol. In recent years, with the development of deep learning and adversarial learning, protocol fuzzing methods based on generative adversarial networks (GAN) have gradually attracted attention. Adversarial learning can simulate attackers through generative models to generate more threatening test data, improving the efficiency and coverage of fuzzing. However, traditional GANs have difficulty in generating discrete sequences (such as protocol messages), as their generation process cannot be directly optimized by gradient descent. Therefore, researchers have proposed SeqGAN based on reinforcement learning, modeling the sequence generation problem as a reinforcement learning task, with the generator as a policy network and using Monte Carlo search to estimate the rewards of future sequences. The discriminator acts as a reward function, distinguishing between generated and real sequences. Although this method solves the problem of traditional GANs not being able to generate discrete sequences, it still has the following limitations in practical applications:

[0004] 1. Low pass rate of generated test cases: Since the generation strategy of SeqGAN has high randomness at the beginning of training, the pass rate of generated test cases in the protocol environment is often not ideal, and it is difficult to stably trigger the boundary behavior or vulnerabilities of the protocol.

[0005] 2. Lack of diversity: SeqGAN may tend to generate similar sequences during training, resulting in poor diversity of generated test cases and difficulty in effectively covering various situations of the protocol. This mode collapse limits its exploration ability for complex protocols. SUMMARY

[0006] Therefore, the protocol fuzzing method and system based on improved SeqGAN are provided.

[0007] To achieve the above purpose, the present application provides the following technical solutions.

[0008] The protocol fuzzing method based on improved SeqGAN comprises the following steps.

[0009] S1, collecting basic data and preprocessing the data, wherein the preprocessing comprises data cleaning, data set division, data encoding and data set padding;

[0010] S2, designing a generator G for generating test cases α , wherein the generator G α learning the space-time features of sequences through a long short-term memory network (LSTM), and capturing long-distance dependencies in time series;

[0011] S3, designing discriminators Dγ and Dλ for respectively evaluating the passing rate and diversity of test cases, wherein the discriminators capture multi-scale local features through multi-scale convolution kernels, capture global time sequence dependencies through LSTM layers, and then discriminate the test cases after fusion;

[0012] S4, designing a hybrid reward, and evaluating the test cases based on the hybrid reward using the two discriminators;

[0013] S5, designing corresponding loss functions for the generator and the two discriminators, and respectively performing gradient updates to perform and complete training.

[0014] Further, in step S1, data collection refers to collecting real protocol messages in an actual communication environment through network packet capture or an existing protocol message library;

[0015] Data cleaning refers to removing noise and irrelevant information in training data, repairing errors in data, and handling missing values;

[0016] Data set division refers to re-dividing the data set based on the entropy value of the message, and the process includes: calculating the entropy value index for measuring the complexity and diversity of the content of each message; then, according to the entropy value, the data set is divided into high-entropy and low-entropy messages, and the high-entropy messages are used as positive samples of diversified messages, and the low-entropy messages are used as negative samples of non-diversified messages;

[0017] Data encoding refers to converting discrete fields in protocol messages into high-dimensional vectors through an embedding layer or a pre-trained embedding model;

[0018] Data padding refers to padding the message sequence, and L maxThe maximum length after padding, the actual length of message Y is L Y , then the filling operation P is expressed as:

[0019]

[0020] Here, <PAD> represents a special character used to fill in a message shorter than the maximum length.

[0021] Furthermore, in step S2, the long short-term memory network LSTM is used as the generator G of the adversarial network. α , which includes the following processes:

[0022] Input the real data sample into the embedding layer to map the discrete characters into a continuous low-dimensional vector space. The process is expressed as follows:

[0023] e(x i )=W[x i ]

[0024] Among them, x i Represents a discrete symbol; e(x i ) is the continuous variable after mapping; W is the lookup matrix of the embedding layer, whose size is |V|×d, |V| is the vocabulary size, d is the vector dimension, and each row of W i The embedding vector corresponding to the i-th symbol in the vocabulary;

[0025] The continuous variable e(x i ) is passed to the LSTM layer to learn the time series features, where forward propagation is first performed in the LSTM layer:

[0026] f t =σ(W f *[h t-1 ,X t ]+b f )

[0027] i t =σ(W i *[h t-1 ,X t ]+b i )

[0028] o t =σ(W o *[h t-1 ,X t ]+b o )

[0029] Among them, f t 、i t 、o tRepresent the forget gate, input gate and output gate respectively, through the hidden state h of the previous time step t-1 and the input X at the current time step t The linear transformation and nonlinear activation are obtained, * represents element multiplication, σ represents the sigmoid activation function, W f 、W i 、W o represents the weight matrix, b f 、b i 、b o represents bias;

[0030] The corresponding gate is associated with the previous cell state C t-1 and the current new cell state C t By element-by-element multiplication, the unit state update process is expressed as:

[0031]

[0032] h t =o t *tanh(C t )

[0033] The above process determines the cell state C at the current moment through the cell state controlled by the output gate t The amount of information output h t ; Where tanh(·) represents the activation function, W C represents the weight matrix, b C represents the bias term, Represents the candidate memory unit value;

[0034] After the LSTM network, a fully connected layer and a sigmoid activation function are connected to convert the output h of the LSTM network into t Convert to the probability distribution of the sequence and get the test case.

[0035] Further, in step S2, for each generation step, the generator G α It can only describe the first t generated elements, and uses Monte Carlo search to simulate the generated elements from the t+1th to the Tth step to obtain the complete sequence Y 1:T , the process is expressed as:

[0036] Given the current partial sequence Y 1:t , use the generator Gα to generate multiple complete sequence samples Y through Monte Carlo search based on the currently generated characters (1) 1:T ,Y (2) 1:T ,...,Y (M) 1:T .

[0037] Furthermore, in step S3, the model structures of the two discriminators Dγ and Dλ are the same, and both include an embedding layer that introduces multi-sized convolution kernels, a feature fusion layer, an LSTM layer, a Highway layer, a Dense layer, and a sigmoid layer.

[0038] In the embedding layer, the input sequence is represented as X∈R T×d , T is the time step, d is the feature dimension, and multiple convolution kernel sizes are used to extract features:

[0039] F1=Conv(X,W1)+b1

[0040] F2=Conv(X,W2)+b2

[0041] F3=Conv(X,W3)+b3

[0042] Among them, W1, W2, and W3 are convolution kernels of different scales, and b1, b2, and b3 are bias items of convolution kernels of different scales. In the feature fusion layer, the multi-scale convolution features are fused by splicing:

[0043] F=[F1;F2;F3]

[0044] Among them, [;] represents the splicing operation;

[0045] In the LSTM layer, the multi-scale features extracted by the convolution kernel are concatenated as sequence features, and the dependencies between the multi-scale features are learned through temporal modeling.

[0046] The Highway layer includes two key gates: the transformation gate and the carrying gate, whose formulas are:

[0047] y=transform(H)·T(H)+H·(1-T(H))

[0048] T(H) is the carrying gate that determines the degree of input information retention and is defined as:

[0049] T(H)=σ(W g H+b g )

[0050] Where W g is another set of weight matrices, b g is the bias, σ is the sigmoid function; 1-T(H) is the complement of the transformation gate, which controls the degree of information preservation from the input H;

[0051] After the LSTM and Highway layers, the final output passes through a fully connected layer and a sigmoid activation function to obtain the target output probability distribution.

[0052] Furthermore, in step S4, the discriminator Dγ is used to analyze the complete sequence Y generated by Monte Carlo. (M) 1:T Determine the pass rate and output the probability D that the test case meets the requirements of the application scenario γ (Y); Use the discriminator Dλ to analyze the complete sequence Y generated by Monte Carlo (M) 1:T (Total M) perform diversity judgment and output the diversity probability D of the test case and other generated samples λ (Y);

[0053] Among them, the mixed reward is expressed as:

[0054] Rtotal(Y)=βR γ (Y 1:t )+θR λ (Y 1:t )

[0055] where β and θ are the reward weights of the discriminator Dγ and Dλ respectively, R γ (Y 1:t ) and R λ (Y 1:t ) represent the rewards of the discriminator Dγ and Dλ, which are expressed as:

[0056]

[0057] Wherein, m represents the mth sequence among a total of M sequences.

[0058] Furthermore, in step S4, the simulated annealing algorithm and the sigmoid function are combined to dynamically adjust the weights β and θ. In the simulated annealing, the temperature is dynamically adjusted according to the time step. The following sigmoid temperature adjustment formula is used to obtain the corresponding weight β at a certain time step t: t and θ t :

[0059]

[0060] θ t =1-β t

[0061] Among them, T maxis the initial temperature, which controls the maximum temperature of simulated annealing; k controls the steepness of the sigmoid function. The larger the k value, the faster the temperature changes; x0 is the midpoint of the sigmoid function, which indicates the time step where the "inflection point" of the temperature change is located.

[0062] Furthermore, in step S5, for the update of the generator, the loss function of the generator is:

[0063] loss G(α) =-E Y～G(α) [Rtotal(Y)logG (α) (Y)]

[0064] Among them, Rtotal(Y) is the mixed reward function, G (α) (Y) is the probability of generating sequence Y, and the negative sign indicates minimizing the loss, that is, maximizing the reward of the generator; E Y～G(α) Represents the expectation of generating sequence Y;

[0065] The parameter update of the generator is performed by gradient descent, and the gradient calculation formula is as follows:

[0066]

[0067] Among them, Rtotal(Y 1:t ) is the mixed reward of the current partial sequence obtained by Monte Carlo search, G (α) (Y t |Y 1:t-1 ) is the generator that generates Y given t-1 sequences t The conditional probability of .

[0068] Furthermore, in step S5, for the gradient update of the discriminator, the loss function of the discriminator is:

[0069] loss D =-E Y～Pdata [logD(Y)]-E Y～Gα [log(1-D(Y))]

[0070] Among them, -E Y～Pdata [logD(Y)] represents the loss of the discriminator on the real sample, which aims to increase the output probability D(Y) of the real sample to close to 1; E Y～Gα [log(1-D(Y))] represents the loss of the discriminator on the generated samples, which aims to reduce the output probability D(Y) of the generated samples to close to 0.

[0071] On the other hand, the present invention also proposes a protocol fuzz testing system based on an improved SeqGAN, which includes: a test case generation module, a fuzz test execution module and a log module, wherein the test case generation module is used to execute the aforementioned protocol fuzz testing method based on the improved SeqGAN and to generate test cases after training is completed;

[0072] The fuzz test execution module includes a client unit, a network connection unit, and a listener. The client is used to simulate devices in the actual environment to send generated test cases and receive responses. The network connection unit is used to connect the client to the MQTT server and agent. The listener is used to monitor the network response from the agent in real time.

[0073] The log module is responsible for recording and analyzing the test execution results, including pass rate, unique messages, and coverage.

[0074] The beneficial effects of the present invention are:

[0075] This paper proposes an improved SeqGAN method. By introducing convolutional kernels of multiple sizes into the discriminator, it extracts information at different levels and fuses these features to improve the accuracy of generator training. Furthermore, a new diversity discriminator is added to assess the diversity of generated use cases. At each time step, the discriminator provides a diversity reward, which is weighted and summed with the original reward to generate a mixed reward. This design not only ensures the accuracy of the generated sequences but also significantly improves the diversity of the generated use cases, thereby more effectively covering the edge cases of the protocol and enhancing the efficiency and coverage of protocol fuzz testing.

[0076] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0078] Figure 1 Schematic diagram of the overall structure of the improved SeqGAN of the present invention;

[0079] Figure 2 Schematic diagram of the LSTM layer network structure of the present invention;

[0080] Figure 3 Schematic diagram of the improved SeqGAN training process of the present invention;

[0081] Figure 4 Schematic diagram of the framework of the fuzzy testing system based on the improved SeqGAN model of the present invention. DETAILED DESCRIPTION

[0082] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0083] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0084] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0085] See also Figures 1 to 4 , which is a protocol fuzz testing method and system based on improved SeqGAN.

[0086] This embodiment proposes a specific implementation method of a protocol fuzz testing method and system based on an improved SeqGAN. It extracts information at different levels in the discriminator by introducing convolution kernels of multiple sizes and fuses these features to improve the accuracy of generator training. At the same time, a new diversity discriminator is added to evaluate the diversity of generated use cases. At each time step, the discriminator provides a diversity reward and adds it to the weighted sum of the original reward to generate a mixed reward. This design not only ensures the accuracy of the generated sequence, but also significantly improves the diversity of the generated use cases, thereby more effectively covering the boundary cases of the protocol and enhancing the efficiency and coverage of the protocol fuzz testing.

[0087] Specifically, in this embodiment, the method includes the following steps:

[0088] S1: Collect basic data and preprocess the data, where preprocessing includes data cleaning, data set division, data encoding, and data set filling.

[0089] (1) Data collection

[0090] First, the dataset is derived from real protocol messages in real-world communication environments. These messages are collected through network packet capture or from existing protocol message libraries, ensuring data authenticity and diversity. The real data of the target protocol provides sufficient samples for the generative adversarial network, improving the accuracy of the generated messages while ensuring the threat level and coverage of the test examples.

[0091] (2) Data cleaning

[0092] During this phase, we remove noise and irrelevant information from the training data, fix errors in the data, and handle missing values ​​to ensure data integrity. This step ensures that the cleaned data more accurately reflects the characteristics of actual messages, thereby improving the quality of model training.

[0093] (3) Dataset division

[0094] To implement the diversity discriminator, the dataset needs to be repartitioned based on the entropy of the messages. First, the entropy value of each message is calculated as a measure of the complexity and diversity of the message content. Then, based on the entropy value, the dataset is divided into two categories: high entropy (diversity) and low entropy (low diversity). High-entropy messages are used as positive samples of diverse messages, and low-entropy messages are used as negative samples of non-diversity, which are provided to the diversity discriminator for training. In this way, the discriminator can distinguish the diversity level of the generated messages, thereby providing reward feedback to the generator and promoting the generation of more diverse MQTT messages.

[0095] (4) Data encoding

[0096] Word embedding: This converts discrete fields in protocol messages into high-dimensional vectors using an embedding layer or a pre-trained embedding model (such as Word2Vec). Word embeddings can capture the contextual relationships in protocol messages and improve the generator's learning capabilities.

[0097] (5) Data filling

[0098] Since the length of protocol messages is usually inconsistent, in order to ensure that the model can handle sequences of different lengths, the message sequence is padded. Padding means that messages shorter than the maximum length are replaced with special symbols (such as <pad>) to make all messages of the same length, which is convenient for batch processing. At the same time, by introducing a mask mechanism in the model, it is ensured that the generator and discriminator only process the actual data part and ignore the padding part. Let L max The maximum length after padding, the actual length of message Y is L Y , then we can define a filling operation P:

[0099]

[0100] SeqGAN is an improved architecture that combines deep learning and reinforcement learning for the generation of sequence data, such as message data. Message data can actually be regarded as a series of characters. A message sequence Y 1:T It can be expressed as: 1:T =(e1,e2,e3,····,e T ), where e i ∈E, E is a set of hexadecimal characters, n is the number of characters. Each message is an unknown distribution function Q m Therefore, the goal of learning is to approximate the distribution function Q m .

[0101] The improved SeqGAN model of the present invention is as follows Figure 1 As shown, it includes a generator G α , authenticity discriminator D γ and diversity discriminator D λ .G α A time series deep model is used to gradually generate outputs for each time step, ensuring that the generated sequence conforms to a specific format and semantics. γ Classify the input sequence, calculate the probability of real data and generated data, and provide feedback to the generator. λ A diversity evaluation mechanism is introduced to improve the diversity of generated sequences.

[0102] S2: Design a generator G to generate test cases α :

[0103] The present invention uses the long short-term memory network LSTM as the generator G of the adversarial network α , learning the spatiotemporal features of the sequence, which can effectively capture the long-distance dependencies in the time series. Discriminator D γ 、D λ The improved CNN-LSTM architecture combines the local features extracted by the convolutional layer with the global temporal features learned by the LSTM layer. This feature fusion significantly improves the discriminator's overall understanding of the sequence, enabling it to more accurately distinguish generated sequences from real sequences.

[0104] Building an LSTM-based generator model involves the following steps:

[0105] The input data is real sample data, which is input to the embedding layer to map discrete characters to continuous low-dimensional vector space. The embedding layer maps discrete symbols x i Mapped to a continuous variable e(x i ) is:

[0106] e(x i )=W[x i ]

[0107] W is the lookup matrix of the embedding layer, of size |V|×d, where |V| is the vocabulary size and d is the dimension of the vector. i The embedding vector corresponding to the i-th symbol in the vocabulary.

[0108] The continuous vector processed by the embedding layer can be passed as input to the LSTM layer for learning time series features.

[0109] In the LSTM layer, the network structure is as follows Figure 2 As shown:

[0110] The forward propagation process of the LSTM model involves a series of mathematical operations, which can be expressed as the following equations. These formulas describe the state update and output generation process of the LSTM unit, enabling it to process the input sequence and generate meaningful output:

[0111] f t =σ(W f *[h t-1 ,X t ]+b f )

[0112] i t =σ(W i *[h t-1 ,X t ]+b i )

[0113] o t =σ(W o *[h t-1 ,X t ]+b o )

[0114] where f t 、i t 、o t Represent the forget gate, input gate and output gate respectively, through the hidden state h of the previous time step t-1 and the input x at the current time step t It is obtained by linear transformation and nonlinear activation. * represents element multiplication, and σ represents the sigmoid activation function, which is used to limit the output between 0 and 1. f 、W i 、W o represents the weight matrix, b f 、b i 、b o Represents bias.

[0115] The corresponding gate is associated with the previous cell state C t-1 and the current new cell state C t Element-wise multiplication is performed to preserve historical information and filter current information. In LSTM, the cell state update is defined in the following formula, which is decomposed into historical information preservation and input information selection.

[0116]

[0117] h t =o t *tanh(C t )

[0118] Among them, tanh(·) represents the activation function, W C represents the weight matrix, b C represents the bias term, Represents the candidate memory cell value; the formula expresses the cell state controlled by the output gate, which determines the cell state C at the current moment t How much information should be output.

[0119] After the LSTM network, a fully connected layer (Dense layer) and a sigmoid activation function are usually connected to the output h of the LSTM network. t Convert it into the probability distribution of the sequence and then get the fake sample.

[0120] Generator generation test case: During the process of the generator Gα generating a sequence, for each generation step, the generator can only see the current partial sequence Y 1:t (i.e. the first t generated elements). And the complete sequence Y 1:T It is necessary to wait until the generator completes all steps to obtain it. Therefore, Monte Carlo search is used to simulate the subsequent part of the partial sequence (i.e., the generation from the t+1th to the Tth step).

[0121] The specific implementation is:

[0122] Given the current partial sequence Y 1:t , use the generator Gα to generate multiple complete sequence samples Y through Monte Carlo search MCTS based on the currently generated characters (1) 1:T ,Y (2) 1:T ,...,Y (M) 1:T .

[0123] S3. Design two discriminators:

[0124] Discriminator Dγ: This is used to evaluate the pass rate of generated test cases. By comparing the generated test cases with real test cases, the discriminator Dγ determines whether the generated test cases meet the requirements of the actual application scenario.

[0125] Discriminator Dλ: It is used to evaluate the diversity of the generated test cases. The discriminator Dλ is designed to ensure that the generated test cases are sufficiently different from each other to avoid the generation of repetitive test cases.

[0126] The discriminator model constructed in this paper consists of an embedding layer, a feature fusion layer with multi-sized convolution kernels, an LSTM layer, a Highway layer, a Dense layer, and a sigmoid layer. Its input data is divided into two parts: one is fake samples generated by the generator as negative sample input, and the other is real data as positive sample input.

[0127] This paper proposes an improved discriminator architecture that applies multi-scale convolution kernels (1x1, 3x3, and 5x5) to feature extraction in the embedding and LSTM layers. The multi-scale convolution kernels capture local features at different scales, while the LSTM layer accounts for global temporal dependencies. This feature fusion enhances the discriminator's ability to understand and distinguish sequences. This design not only improves the accuracy of the discriminator but also enhances the generalization of the generative model, enabling it to perform better in complex sequence generation tasks.

[0128] The input sequence can be represented as X∈R T×d Where T is the time step and d is the feature dimension. Using multiple different convolution kernel sizes (for example, 1, 3, 5), the mathematical expression of the features generated by different convolution kernels is:

[0129] F1=Conv(X,W1)+b1

[0130] F2=Conv(X,W2)+b2

[0131] F3=Conv(X,W3)+b3

[0132] Among them, W1, W2, and W3 are convolution kernels of different scales 1×1, 3×3, and 5×5, respectively; b1, b2, and b3 are bias terms of convolution kernels of different scales;

[0133] Feature fusion: These multi-scale convolutional features are fused by splicing.

[0134] F=[F1;F2;F3]

[0135] Where [;] represents the splicing operation.

[0136] By splicing the multi-scale features extracted by the convolution kernel and treating them as sequence features, the LSTM layer learns the dependencies between these features through temporal modeling, thereby generating richer expressions.

[0137] Introducing a Highway Network layer after the LSTM layer helps the model better selectively transfer or transform features. The Highway Network layer allows some information to pass through the network unchanged without undergoing nonlinear transformations, thereby preserving important low-level features.

[0138] The Highway layer is similar to a gated network layer, allowing the model to choose between transforming and preserving input information. It introduces two key gates: the transform gate and the carry gate. Its formula is:

[0139] y=transform(H)·T(H)+H·(1-T(H))

[0140] T(H) is the carrying gate that determines the degree of input information retention and is defined as:

[0141] T(H)=σ(W g H+b g )

[0142] Where W g is another set of weight matrices, b g is the bias, σ is the sigmoid function, which is used to limit the output to the range of [0,1]. 1-T(H) is the complement of the transformation gate, which controls how much information is retained from the input H.

[0143] After the LSTM and Highway layers, the final output goes through a fully connected layer and a sigmoid activation function to obtain the target output probability distribution.

[0144] S4. Use the discriminator to evaluate the generated test cases:

[0145] The discriminator Dγ is used to generate the complete sequence Y for each Monte Carlo (M) 1:T (Total M) Make a pass rate judgment and output the probability D that the test case meets the requirements of the application scenario γ (Y).

[0146] The discriminator Dλ is used to generate the complete sequence Y for each Monte Carlo (M) 1:T (Total M) perform diversity judgment and output the diversity probability D of the test case and other generated samples λ (Y).

[0147] Then the mixed reward can be expressed as:

[0148] Rtotal(Y)=βR γ (Y 1:t )+θR λ (Y 1:t )

[0149] Where β and θ are the reward weights of the discriminators Dγ and Dλ respectively. The simulated annealing algorithm and the sigmoid function are used to dynamically adjust the weights β and θ. If we want to dynamically adjust the temperature according to the time step in simulated annealing, we can use the following sigmoid temperature adjustment formula to get the corresponding weight β at a certain time step t: t and θ t

[0150]

[0151] θ t =1-β t

[0152] Among them, T max is the initial temperature, which controls the maximum temperature for simulated annealing. k controls the steepness of the sigmoid function; larger k values ​​result in a faster temperature drop, while smaller k values ​​result in a slower temperature change. x0 is the midpoint of the sigmoid function, representing the time step at which the temperature change "inflection point" occurs.

[0153] The rewards of the discriminators Dγ and Dλ can be expressed as follows:

[0154]

[0155] The mixed reward output is obtained through the above formula.

[0156] S5. Design corresponding loss functions for the generator and the two discriminators, and perform gradient updates respectively to complete the training.

[0157] Generator gradient update:

[0158] In SeqGAN, the gradient update of the generator is based on the policy gradient method. The reward is obtained through the Monte Carlo search result above. The loss function of the generator is:

[0159] loss G(α) =-E Y～G(α) [Rtotal(Y)logG (α) (Y)]

[0160] Where Rtotal(Y) is the hybrid reward function defined above, G (α) (Y) is the probability of generating sequence Y. The negative sign indicates minimizing the loss, that is, maximizing the reward of the generator; E Y～G(α) Represents the expectation of generating the sequence Y.

[0161] The parameter update of the generator is performed by gradient descent, and the gradient calculation formula is as follows:

[0162]

[0163] Where Rtotal(Y 1:t ) is the mixed reward of the current partial sequence obtained by Monte Carlo search, G (α) (Y t |Y 1:t-1 ) is the generator that generates Y given t-1 sequences t The conditional probability of .

[0164] Through this loss function, the generator can adjust the generation strategy according to the mixed reward, which not only improves the pass rate of generated test cases but also increases the diversity of test cases.

[0165] Gradient update of the discriminator:

[0166] The parameters of the discriminator are trained by gradient updating, usually using the backpropagation algorithm to optimize its loss function. The loss function of the discriminator is:

[0167] loss D =-E Y～Pdata [logD(Y)]-E Y～Gα [log(1-D(Y))]

[0168] Among them, -E Y～Pdata [logD(Y)] represents the loss of the discriminator on the real sample. It is expected that the discriminator can increase the output probability D(Y) of the real sample to as close to 1 as possible. Y～Gα [log(1-D(Y))] represents the loss of the discriminator on the generated samples. It is expected that the discriminator can reduce the output probability D(Y) of the generated samples to close to 0. By minimizing this loss function, the discriminator can better learn how to distinguish between real samples and generated samples, thereby improving the quality of the samples generated by the generator.

[0169] This embodiment also proposes a specific implementation of a protocol fuzzy testing system based on an improved SeqGAN, wherein Figure 4 As shown in the figure, the overall framework of the system is divided into three main parts: test case generation module, fuzz test execution module and log module.

[0170] (1) Test case generation module

[0171] This part is responsible for generating test cases for fuzz testing, which includes the following steps:

[0172] Data Collection: Collect real protocol messages from actual communication environments to ensure the authenticity and diversity of the generated data. This data can be obtained through network packet capture or from an existing protocol message library.

[0173] Data preprocessing: Encode and fill the collected data to improve the model's ability to capture the contextual relationships in protocol messages and enhance the generator's learning ability.

[0174] Model training: The cleaned data is used to train the generative model. The improved SeqGAN model is used for training to generate effective test cases.

[0175] Model running: After training is completed, run the trained model to generate actual fuzz test cases and prepare to send these cases to the fuzz test execution system.

[0176] (2) Fuzz test execution module

[0177] This part is responsible for executing the generated test cases and performing actual fuzz testing. It specifically includes the following components:

[0178] Client construction: Build a client to simulate devices in the actual environment to send generated test cases and receive responses.

[0179] Network connectivity: Ensure stable network connectivity between the client and the MQTT server and broker to enable fast transmission of test cases and receipt of feedback.

[0180] Listener: A listener monitors network responses from the agent in real time, helping the execution system stay informed of test progress and results. During the fuzzing process, all communication messages between the client and the agent are tracked. The client sends MQTT packets to the agent and records the corresponding responses, thereby monitoring the agent's status.

[0181] (3) Log module

[0182] This section is responsible for recording and analyzing the results of test execution, including:

[0183] Pass rate: By analyzing the return code fed back by the agent, the pass rate of the sent test cases is calculated to evaluate the effectiveness of fuzz testing.

[0184] Unique messages: Record unique messages fed back by agents. These messages represent the diversity of test cases, thus helping to evaluate the comprehensiveness of the test.

[0185] Coverage: Measures code coverage and provides a basis for subsequent test optimization.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.< / pad>

Claims

1. A protocol fuzz testing method based on improved SeqGAN, characterized by: The method comprises the following steps: S1. Collect basic data and preprocess the data, where preprocessing includes data cleaning, data set division, data encoding, and data set filling; In step S1, data collection refers to collecting real protocol messages in the actual communication environment by capturing packets on the network or from an existing protocol message library; Data cleaning involves removing noise and irrelevant information from training data, fixing errors in the data, and handling missing values. Dataset partitioning involves repartitioning the dataset based on the entropy of the messages. This process involves calculating an entropy value for each message, which measures the complexity and diversity of the message content. Then, based on the entropy value, the dataset is divided into high-entropy and low-entropy messages. High-entropy messages serve as positive samples indicating diverse messages, while low-entropy messages serve as negative samples indicating non-diversity. Data encoding refers to converting discrete fields in protocol messages into high-dimensional vectors through embedding layers or pre-trained embedding models; Data filling refers to the filling operation on the message sequence. Let L max The maximum length after padding, the actual length of message Y is L Y , then the filling operation P is expressed as: Wherein, <PAD> represents a special character used to fill in a message shorter than the maximum length; S2. Design a generator G to generate test cases α , where the generator G α The long short-term memory network (LSTM) is used to learn the spatiotemporal features of the sequence and capture the long-distance dependencies in the time series. S3. Design the discriminators Dγ and Dλ to evaluate the pass rate and diversity of test cases, respectively. The discriminator captures multi-scale local features through multi-scale convolution kernels and uses LSTM layers to capture global temporal dependencies. The two are then integrated to discriminate the test cases. S4. Design a hybrid reward and use two discriminators to evaluate the test cases based on the hybrid reward; S5. Design corresponding loss functions for the generator and the two discriminators, and perform gradient updates respectively to complete the training. After the training is completed, the generator G α Used to generate test cases for protocol fuzz testing.

2. The protocol fuzz testing method based on improved SeqGAN according to claim 1, characterized in that: In step S2, the long short-term memory network LSTM is used as the generator G of the adversarial network α ,That The following processes are included: Input the real data sample into the embedding layer to map the discrete characters into a continuous low-dimensional vector space. The process is expressed as follows: e(x i )=W[x i ] Among them, x i Represents a discrete symbol; e(x i ) is the continuous variable after mapping; W is the lookup matrix of the embedding layer, whose size is |V|×d, |V| is the vocabulary size, d is the vector dimension, and each row of W i The embedding vector corresponding to the i-th symbol in the vocabulary; The continuous variable e(x i ) is passed to the LSTM layer to learn the time series features, where forward propagation is first performed in the LSTM layer: f t =σ(W f *[h t-1 ,X t ]+b f ) i t =σ(W i *[h t-1 ,X t ]+b i ) the t =σ(W o *[h t-1 ,X t ]+b o ) Among them, f t 、i t 、o t Represent the forget gate, input gate and output gate respectively, through the hidden state h of the previous time step t-1 and the input X at the current time step t The linear transformation and nonlinear activation are obtained, * represents element multiplication, σ represents the sigmoid activation function, W f 、W i 、W o represents the weight matrix, b f 、b i 、b o represents bias; The corresponding gate is associated with the previous cell state C t-1 and the current new cell state C t By element-by-element multiplication, the unit state update process is expressed as: h t =o t *fishy(C) t ) The above process determines the cell state C at the current moment through the cell state controlled by the output gate t The amount of information output h t ; Where tanh(·) represents the activation function, W C represents the weight matrix, b C represents the bias term, Represents the candidate memory unit value; After the LSTM network, a fully connected layer and a sigmoid activation function are connected to convert the output h of the LSTM network into t Convert to the probability distribution of the sequence and get the test case.

3. The protocol fuzz testing method based on improved SeqGAN according to claim 2, characterized in that: In step S2, for each generation step, the generator G α It can only describe the first t generated elements, and uses Monte Carlo search to simulate the generated elements from the t+1th to the Tth step to obtain the complete sequence Y 1:T , the process is expressed as: Given the current partial sequence Y 1:t , use the generator Gα to generate multiple complete sequence samples Y through Monte Carlo search based on the currently generated characters (1) 1:T ,Y (2) 1:T ,...,Y (M) 1:T .

4. The protocol fuzz testing method based on improved SeqGAN according to claim 3, characterized in that: In step S3, the two discriminators Dγ and Dλ have the same model structure, which includes an embedding layer with multi-sized convolution kernels, a feature fusion layer, an LSTM layer, a Highway layer, a Dense layer, and a sigmoid layer. In the embedding layer, the input sequence is represented as X∈R T×d , T is the time step, d is the feature dimension, and multiple convolution kernel sizes are used to extract features: F1=Conv(X,W1)+b1 F2=Conv(X,W2)+b2 F3=Conv(X,W3)+b3 Among them, W1, W2, and W3 are convolution kernels of different scales, and b1, b2, and b3 are bias terms of convolution kernels of different scales; In the feature fusion layer, multi-scale convolutional features are fused by splicing: F=[F1;F2;F3] Among them, [;] represents the splicing operation; In the LSTM layer, the multi-scale features extracted by the convolution kernel are concatenated as sequence features, and the dependencies between the multi-scale features are learned through temporal modeling. The Highway layer includes two key gates: the transformation gate and the carrying gate, whose formulas are: y=transform(H)·T(H)+H·(1-T(H)) T(H) is the carrying gate that determines the degree of input information retention and is defined as: T(H)=σ(W g H+b g ) Where W g is another set of weight matrices, b g is the bias, σ is the sigmoid function; 1-T(H) is the complement of the transformation gate, which controls the degree of information preservation from the input H; After the LSTM and Highway layers, the final output passes through a fully connected layer and a sigmoid activation function to obtain the target output probability distribution.

5. The protocol fuzz testing method based on improved SeqGAN according to claim 4, characterized in that: In step S4, the discriminator Dγ is used to analyze the complete sequence Y generated by Monte Carlo. (M) 1:T Determine the pass rate and output the probability D that the test case meets the requirements of the application scenario γ (Y); Use the discriminator Dλ to analyze the complete sequence Y generated by Monte Carlo (M) 1:T Perform diversity determination, complete sequence Y (M) 1:T There are M in total, and the diversity probability D of the test case and other generated samples is output λ (Y); Among them, the mixed reward is expressed as: Rtotal(Y)=βR γ (AND 1:t )+θR λ (AND 1:t ) where β and θ are the reward weights of the discriminator Dγ and Dλ respectively, R γ (Y 1:t ) and R λ (Y 1:t ) represent the rewards of the discriminator Dγ and Dλ, which are expressed as: Wherein, m represents the mth sequence among a total of M sequences.

6. The protocol fuzz testing method based on improved SeqGAN according to claim 5, characterized in that: In step S4, the simulated annealing algorithm and the sigmoid function are combined to dynamically adjust the weights β and θ. In the simulated annealing, the temperature is dynamically adjusted according to the time step. The following sigmoid temperature adjustment formula is used to obtain the corresponding weight β at a certain time step t: t and θ t : i t =1-β t Among them, T max is the initial temperature, which controls the maximum temperature of simulated annealing; k controls the steepness of the sigmoid function. The larger the k value, the faster the temperature changes; x0 is the midpoint of the sigmoid function, which indicates the time step where the "inflection point" of the temperature change is located.

7. The protocol fuzz testing method based on improved SeqGAN according to claim 5, characterized in that: In step S5, for the update of the generator, the loss function of the generator is: loose G(α) =-E Y～G(α) [Rtotal(Y)logG (α) (Y)] Among them, Rtotal(Y) is the mixed reward function, G (α) (Y) is the probability of generating sequence Y, and the negative sign indicates minimizing the loss, that is, maximizing the reward of the generator; E Y～G(α) Represents the expectation of generating sequence Y; The parameter update of the generator is performed by gradient descent, and the gradient calculation formula is as follows: Among them, Rtotal(Y 1:t ) is the mixed reward of the current partial sequence obtained by Monte Carlo search, G (α) (Y t |Y 1:t-1 ) is the generator that generates Y given t-1 sequences t The conditional probability of .

8. The protocol fuzz testing method based on improved SeqGAN according to claim 5, characterized in that: In step S5, for the gradient update of the discriminator, the loss function of the discriminator is: loss D =-E Y～Pdata [logD(Y)]-E Y～Gα [log(1-D(Y))] Among them, -E Y～Pdata [logD(Y)] represents the loss of the discriminator on the real sample, which aims to increase the output probability D(Y) of the real sample to close to 1; E Y～Gα [log(1-D(Y))] represents the loss of the discriminator on the generated samples, which aims to reduce the output probability D(Y) of the generated samples to close to 0.

9. A protocol fuzz testing system based on improved SeqGAN, characterized by: The system includes: a test case generation module, a fuzz test execution module and a log module, wherein the test case generation module is used to execute the protocol fuzz testing method based on the improved SeqGAN according to any one of claims 1 to 8, and is used to generate test cases after training is completed; The fuzz test execution module includes a client unit, a network connection unit, and a listener. The client is used to simulate devices in the actual environment to send generated test cases and receive responses. The network connection unit is used to connect the client to the MQTT server and agent. The listener is used to monitor the network response from the agent in real time. The log module is responsible for recording and analyzing the test execution results, including pass rate, unique messages, and coverage.

Images