Protocol fuzz testing method and system based on improved SeqGAN

Abstract

The invention relates to a protocol fuzz testing method and system based on an improved SeqGAN, and belongs to the technical field of protocol fuzz testing. The method comprises the steps that basic data are collected and preprocessed, a generator G alpha for generating a test case and discriminators D gamma and D lambda for evaluating the passing rate and diversity of the test case are designed, a mixed reward is designed, and the two discriminators are adopted to evaluate the test case based on the mixed reward; corresponding loss functions are designed for the generator and the two discriminators, gradient updating is carried out, and training is carried out and completed. A test case generation module of the system is used for executing a protocol fuzzy test method to obtain a test case, then a test is executed through a fuzzy test execution module, and a log module records a test result and an intermediate process response. According to the method, the accuracy of the generated sequence is ensured, and the diversity of the generated use cases is remarkably improved, so that the boundary condition of the protocol is more effectively covered, and the efficiency and the coverage rate of the protocol fuzz test are enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of protocol fuzzy testing, and relates to a protocol fuzzy testing method and system based on improved SeqGAN. Background Art

[0002] With the rapid development of information technology, network communication protocols have been widely used in various fields of modern society, especially in the fields of Internet of Things devices, industrial control systems, smart terminals and cloud computing. As the standard for data transmission, communication protocols ensure reliable communication between network devices. However, with the increasing complexity of network protocols, potential vulnerabilities are inevitably present in protocol design. These vulnerabilities may be exploited by malicious attackers, leading to serious security issues such as information leakage and system paralysis. How to effectively detect protocol vulnerabilities is an urgent problem to be solved. Fuzz testing, as an effective security testing technology, has become the mainstream method in protocol security testing by sending a large amount of random or semi-random data to the system under test and observing its abnormal behavior to discover potential vulnerabilities.

[0003] There are some challenges in traditional fuzz testing, such as strong randomness of input data, low test efficiency, lack of pertinence of generated test cases, and difficulty in covering the boundary conditions of the protocol. In recent years, with the development of deep learning and adversarial learning, protocol fuzz testing methods based on generative adversarial networks (GANs) have gradually attracted attention. Adversarial learning can simulate attackers through generative models, generate more threatening test data, and improve the efficiency and coverage of fuzz testing. However, traditional GANs have difficulties in generating discrete sequences (such as protocol messages) because their generation process cannot be directly optimized by gradient descent. To this end, researchers proposed SeqGAN based on reinforcement learning, which models the sequence generation problem as a reinforcement learning task, uses the generator as a policy network, and uses Monte Carlo search to estimate the rewards of future sequences. The discriminator is used as a reward function to distinguish between generated and real sequences. Although this method solves the problem that traditional GAN ​​cannot generate discrete sequences, it still has the following limitations in practical applications:

[0004] 1. The pass rate of generated test cases is low: Since SeqGAN’s generation strategy is highly random in the early stages of training, the pass rate of the generated test cases in the protocol environment is often not ideal, and it is impossible to stably trigger the boundary behavior or vulnerabilities of the protocol.

[0005] 2. Insufficient diversity: SeqGAN may tend to generate similar sequences during training, resulting in poor diversity of generated use cases and difficulty in effectively covering various situations of the protocol. This pattern collapse phenomenon limits its ability to explore complex protocols. Summary of the invention

[0006] In view of this, the object of the present invention is to provide a protocol fuzzy testing method and system based on improved SeqGAN.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A protocol fuzz testing method based on improved SeqGAN, the method comprising the following steps:

[0009] S1. Collect basic data and preprocess the data, where preprocessing includes data cleaning, data set division, data encoding, and data set filling;

[0010] S2. Design a generator G to generate test cases α , where the generator G α The long short-term memory network LSTM is used to learn the spatiotemporal characteristics of the sequence and capture the long-distance dependencies in the time series;

[0011] S3. Design the discriminators Dγ and Dλ for evaluating the pass rate and diversity of test cases respectively. The discriminator captures multi-scale local features through multi-scale convolution kernels and uses LSTM layer to capture global temporal dependencies. The two are fused to discriminate the test cases.

[0012] S4. Design a hybrid reward and use two discriminators to evaluate the test cases based on the hybrid reward;

[0013] S5. Design corresponding loss functions for the generator and the two discriminators, and perform gradient updates respectively to complete the training.

[0014] Further, in step S1, data collection refers to collecting real protocol messages in an actual communication environment through network packet capture or an existing protocol message library;

[0015] Data cleaning refers to removing noise and irrelevant information from the training data, fixing errors in the data, and handling missing values;

[0016] Dataset partitioning refers to the re-division of the data set based on the entropy value of the message. The process includes: calculating the entropy value index for each message to measure the complexity and diversity of the message content; then, according to the entropy value, the data set is divided into two categories of messages: high entropy and low entropy. The high entropy message will be used as a positive sample of diversified messages, and the low entropy message will be used as a negative sample of non-diversified messages;

[0017] Data encoding refers to converting discrete fields in protocol messages into high-dimensional vectors through embedding layers or pre-trained embedding models;

[0018] Data padding refers to the padding operation on the message sequence. Let L maxis the maximum length after padding, the actual length of message Y is L Y , then the filling operation P is expressed as:

[0019]

[0020] Here, <PAD> represents a special character used to fill in a message shorter than the maximum length.

[0021] Further, in step S2, the long short-term memory network LSTM is used as the generator G of the adversarial network. α , which includes the following processes:

[0022] The real data samples are input into the embedding layer to map the discrete characters into a continuous low-dimensional vector space. The process is expressed as:

[0023] e(x i )=W[x i ]

[0024] Among them, x i represents a discrete symbol; e(x i ) is the continuous variable after mapping; W is the lookup matrix of the embedding layer, whose size is |V|×d, |V| is the vocabulary size, d is the vector dimension, and each row of W i The embedding vector corresponding to the i-th symbol in the vocabulary;

[0025] The mapped continuous variable e(x i ) is passed to the LSTM layer to learn the time series features, where forward propagation is first performed in the LSTM layer:

[0026] f t =σ(W f *[h t-1 ,X t ]+b f )

[0027] i t =σ(W i *[h t-1 ,X t ]+b i )

[0028] o t =σ(W o *[h t-1 ,X t ]+b o )

[0029] Among them, f t 、i t , o tRepresent the forget gate, input gate and output gate respectively, through the hidden state h of the previous time step t-1 and the input X at the current time step t The linear transformation and nonlinear activation are obtained, * represents element multiplication, σ represents the sigmoid activation function, W f , W i , W o represents the weight matrix, b f 、b i 、b o represents bias;

[0030] The corresponding gate is related to the previous cell state C t-1 and the current new cell state C t By multiplying each element, the unit state update process is expressed as:

[0031]

[0032] h t =o t *tanh(C t )

[0033] The above process determines the cell state C at the current moment through the cell state controlled by the output gate. t The amount of information output h t ; where tanh(·) represents the activation function, W C represents the weight matrix, b C represents the bias term, Represents the candidate memory cell value;

[0034] After the LSTM network, a fully connected layer and a sigmoid activation function are connected to convert the output h of the LSTM network into t Convert to the probability distribution of the sequence and get the test case.

[0035] Further, in step S2, for each generation step, the generator G α It can only describe the first t generated elements, and uses Monte Carlo search to simulate the generated elements from the t+1th to the Tth step to obtain the complete sequence Y 1:T , the process is expressed as:

[0036] Given the current partial sequence Y 1:t , use the generator Gα to generate multiple complete sequence samples Y through Monte Carlo search based on the currently generated characters (1) 1:T ,Y (2) 1:T ,...,Y (M) 1:T .

[0037] Furthermore, in step S3, the model structures of the two discriminators Dγ and Dλ are the same, and both sequentially include an embedding layer that introduces multi-size convolution kernels, a feature fusion layer, an LSTM layer, a Highway layer, a Dense layer, and a sigmoid layer;

[0038] In the embedding layer, the input sequence is represented as X∈R T×d , T is the time step, d is the feature dimension, and multiple convolution kernel sizes are used to extract features:

[0039] F1=Conv(X,W1)+b1

[0040] F2=Conv(X,W2)+b2

[0041] F3=Conv(X,W3)+b3

[0042] Among them, W1, W2, and W3 are convolution kernels of different scales, and b1, b2, and b3 are bias items of convolution kernels of different scales. In the feature fusion layer, multi-scale convolution features are fused by splicing:

[0043] F=[F1;F2;F3]

[0044] Among them, [;] represents the splicing operation;

[0045] In the LSTM layer, the multi-scale features extracted by the convolution kernel are concatenated as sequence features, and the dependencies between the multi-scale features are learned through temporal modeling.

[0046] There are two key gates in the Highway layer: the transformation gate and the carrying gate, and their formulas are:

[0047] y=transform(H)·T(H)+H·(1-T(H))

[0048] T(H) is the carrying gate, which determines the degree of input information retention and is defined as:

[0049] T(H)=σ(W g H+b g )

[0050] Where W g is another set of weight matrices, b g is the bias, σ is the sigmoid function; 1-T(H) is the complement of the transformation gate, which controls the degree of information preservation from the input H;

[0051] After the LSTM and Highway layers, the final output passes through a fully connected layer and a sigmoid activation function to obtain the target output probability distribution.

[0052] Further, in step S4, the discriminator Dγ is used to analyze each complete sequence Y generated by Monte Carlo. (M) 1:T Determine the pass rate and output the probability D that the test case meets the requirements of the application scenario γ (Y); Use the discriminator Dλ to analyze each complete sequence Y generated by Monte Carlo (M) 1:T (M in total) perform diversity determination and output the diversity probability D of the test case and other generated samples λ (Y);

[0053] Among them, the mixed reward is expressed as:

[0054] Rtotal(Y)=βR γ (Y 1:t )+θR λ (Y 1:t )

[0055] where β and θ are the reward weights of the discriminators Dγ and Dλ, respectively, and R γ (Y 1:t ) and R λ (Y 1:t ) represent the rewards of the discriminator Dγ and Dλ, which are respectively expressed as:

[0056]

[0057] Wherein, m represents the mth sequence among a total of M sequences.

[0058] Further, in step S4, the weights β and θ are dynamically adjusted by combining the simulated annealing algorithm and the sigmoid function. In the simulated annealing, the temperature is dynamically adjusted according to the time step. The following sigmoid temperature adjustment formula is used to obtain the corresponding weight β at a certain time step t: t and θ t :

[0059]

[0060] θ t =1-β t

[0061] Among them, T maxis the initial temperature, which controls the maximum temperature of simulated annealing; k controls the steepness of the sigmoid function. The larger the k value, the faster the temperature changes; x0 is the midpoint of the sigmoid function, which indicates the time step where the "inflection point" of the temperature change is located.

[0062] Further, in step S5, for the update of the generator, the loss function of the generator is:

[0063] loss G(α) =-E Y～G(α) [Rtotal(Y)logG (α) (Y)]

[0064] Among them, Rtotal(Y) is the mixed reward function, G (α) (Y) is the probability of generating sequence Y, and the negative sign indicates minimizing the loss, that is, maximizing the reward of the generator; E Y～G(α) Represents the expectation of generating sequence Y;

[0065] The parameter update of the generator is performed by gradient descent, and the gradient calculation formula is as follows:

[0066]

[0067] Among them, Rtotal(Y 1:t ) is the mixed reward of the current partial sequence obtained by Monte Carlo search, G (α) (Y t |Y 1:t-1 ) is the generator that generates Y given t-1 sequences t The conditional probability of .

[0068] Further, in step S5, for the gradient update of the discriminator, the loss function of the discriminator is:

[0069] loss D =-E Y～Pdata [logD(Y)]-E Y～Gα [log(1-D(Y))]

[0070] Among them, -E Y～Pdata [logD(Y)] represents the loss of the discriminator on the real sample, which aims to increase the output probability D(Y) of the real sample to close to 1; E Y～Gα [log(1-D(Y))] represents the loss of the discriminator on the generated samples, which aims to reduce the output probability D(Y) of the generated samples to close to 0.

[0071] On the other hand, the present invention also proposes a protocol fuzzy testing system based on improved SeqGAN, the system comprising: a test case generation module, a fuzzy test execution module and a log module, wherein the test case generation module is used to execute the aforementioned protocol fuzzy testing method based on improved SeqGAN, and is used to generate test cases after training is completed;

[0072] The fuzz test execution module includes a client unit, a network connection unit, and a listener. The client is used to simulate the device in the actual environment to send the generated test cases and receive responses; the network connection unit is used for the network connection between the client and the MQTT server and agent; the listener is used to monitor the network response from the agent in real time;

[0073] The log module is responsible for recording and analyzing the results of test execution, including pass rate, unique messages, and coverage.

[0074] The beneficial effects of the present invention are:

[0075] The present invention proposes an improved SeqGAN method, which introduces convolution kernels of multiple sizes to extract information at different levels in the discriminator, and fuses these features to improve the accuracy of generator training. At the same time, a new diversity discriminator is added to evaluate the diversity of generated use cases. At each time step, the discriminator provides a diversity reward and weightedly sums it with the original reward to generate a mixed reward. This design not only ensures the accuracy of the generated sequence, but also significantly improves the diversity of the generated use cases, thereby more effectively covering the boundary conditions of the protocol and enhancing the efficiency and coverage of the protocol fuzz testing.

[0076] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0078] Figure 1 Schematic diagram of the overall structure of the improved SeqGAN of the present invention;

[0079] Figure 2 It is a schematic diagram of the LSTM layer network structure of the present invention;

[0080] Figure 3 Schematic diagram of the improved SeqGAN training process of the present invention;

[0081] Figure 4 It is a schematic diagram of the framework of the fuzzy testing system based on the improved SeqGAN model of the present invention. DETAILED DESCRIPTION

[0082] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0083] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0084] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0085] See also Figure 1 to Figure 4 , which is a protocol fuzz testing method and system based on improved SeqGAN.

[0086] This embodiment proposes a specific implementation method of a protocol fuzz testing method and system based on an improved SeqGAN, which extracts information at different levels in the discriminator by introducing convolution kernels of multiple sizes, and fuses these features to improve the accuracy of generator training. At the same time, a new diversity discriminator is added to evaluate the diversity of generated use cases. At each time step, the discriminator provides a diversity reward and weightedly sums it with the original reward to generate a mixed reward. This design not only ensures the accuracy of the generated sequence, but also significantly improves the diversity of the generated use cases, thereby more effectively covering the boundary conditions of the protocol and enhancing the efficiency and coverage of the protocol fuzz testing.

[0087] Specifically, in this embodiment, the method includes the following steps:

[0088] S1: Collect basic data and preprocess the data, where preprocessing includes data cleaning, data set division, data encoding, and data set filling.

[0089] (1) Data Collection

[0090] First, the dataset is derived from real protocol messages in actual communication environments. These messages are collected through network packet capture or from existing protocol message libraries to ensure the authenticity and diversity of the data. The real data of the target protocol provides sufficient samples for the generative adversarial network, which can not only improve the accuracy of the messages generated by the generator, but also ensure the threat and coverage of the test samples.

[0091] (2) Data cleaning

[0092] In this stage, we remove noise and irrelevant information from the training data, fix errors in the data, and handle missing values ​​to ensure data integrity. This step makes the cleaned data more accurately reflect the characteristics of actual messages, thereby improving the quality of model training.

[0093] (3) Dataset Division

[0094] In order to realize the function of the diversity discriminator, the data set needs to be re-divided based on the entropy value of the message. First, the entropy value of each message will be calculated as an indicator to measure the complexity and diversity of the message content. Then, according to the size of the entropy value, the data set is divided into two types of messages: high entropy (diversity) and low entropy (low diversity). High entropy messages will be used as positive samples of diverse messages, and low entropy messages will be used as non-diverse negative samples to provide to the diversity discriminator for training. In this way, the discriminator can distinguish the diversity level of the generated messages, thereby providing reward feedback to the generator and promoting the generation of more diverse MQTT messages.

[0095] (4) Data encoding

[0096] Word embedding: The discrete fields in the protocol message are converted into high-dimensional vectors through an embedding layer or a pre-trained embedding model (such as Word2Vec). Word embedding can capture the contextual relationship in the protocol message and improve the learning ability of the generator.

[0097] (5) Data filling

[0098] Since the length of protocol messages is usually inconsistent, in order to ensure that the model can handle sequences of different lengths, the message sequence is padded. Padding means that messages shorter than the maximum length are replaced with special symbols (such as <pad>) to make all messages of the same length, which is convenient for batch processing. At the same time, by introducing a mask mechanism in the model, it is ensured that the generator and discriminator only process the actual data part and ignore the padding part. Let L max is the maximum length after padding, the actual length of message Y is L Y , then we can define a filling operation P:

[0099]

[0100] SeqGAN is an improved architecture that combines deep learning and reinforcement learning for the generation of sequence data, such as message data. Message data can actually be regarded as a series of characters. A message sequence Y 1:T It can be expressed as: 1:T =(e1,e2,e3,····,e T ), where e i ∈E, E is a set of hexadecimal characters, n is the number of characters. Each message is an unknown distribution function Q m Therefore, the goal of learning is to approximate the distribution function Q m .

[0101] The improved SeqGAN model of the present invention is as follows Figure 1 As shown, it includes a generator G α , authenticity discriminator D γ and diversity discriminator D λ . G α A time series deep model is used to gradually generate outputs for each time step, ensuring that the generated sequence conforms to a specific format and semantics. γ Classify the input sequence, calculate the probability of real data and generated data, and provide feedback to the generator. λ A diversity evaluation mechanism is introduced to improve the diversity of generated sequences.

[0102] S2: Design a generator G to generate test cases α :

[0103] The present invention uses a long short-term memory network LSTM as the generator G of the adversarial network α , learning the spatiotemporal features of the sequence, which can effectively capture the long-distance dependencies in the time series. Discriminator D γ , D λ The improved CNN-LSTM structure can combine the local features extracted by the convolutional layer with the global temporal features learned by the LSTM layer. This feature fusion significantly improves the discriminator's overall understanding of the sequence, enabling it to more accurately distinguish between generated sequences and real sequences.

[0104] Constructing a LSTM-based generator model includes the following steps:

[0105] The input data is real sample data, which is input to the embedding layer to map discrete characters to continuous low-dimensional vector space. The embedding layer maps discrete symbols x i Mapped to a continuous variable e(x i ) is:

[0106] e(x i )=W[x i ]

[0107] W is the lookup matrix of the embedding layer, with a size of |V|×d, where |V| is the vocabulary size and d is the dimension of the vector. i The embedding vector corresponding to the i-th symbol in the vocabulary.

[0108] The continuous vector processed by the embedding layer can be passed as input to the LSTM layer for learning time series features.

[0109] In the LSTM layer, the network structure is as follows Figure 2 As shown:

[0110] The forward propagation process of the LSTM model involves a series of mathematical operations, which can be expressed in the following equations. These formulas describe the state update and output generation process of the LSTM unit, enabling it to process input sequences and generate meaningful outputs:

[0111] f t =σ(W f *[h t-1 ,X t ]+b f )

[0112] i t =σ(W i *[h t-1 ,X t ]+b i )

[0113] o t =σ(W o *[h t-1 ,X t ]+b o )

[0114] where f t 、i t , o t Represent the forget gate, input gate and output gate respectively, through the hidden state h of the previous time step t-1 and the input x at the current time step t W is obtained by linear transformation and nonlinear activation. * represents element multiplication, and σ represents the sigmoid activation function, which is used to limit the output between 0 and 1. f , W i , W o represents the weight matrix, b f 、b i 、b o Represents bias.

[0115] The corresponding gate is related to the previous cell state C t-1 and the current new cell state C t Element-wise multiplication is performed to preserve historical information and filter current information. In LSTM, the cell state update is defined in the following formula, which is decomposed into historical information preservation and input information selection.

[0116]

[0117] h t =o t *tanh(C t )

[0118] Among them, tanh(·) represents the activation function, W C represents the weight matrix, b C represents the bias term, Represents the candidate memory cell value; the formula expresses the cell state controlled by the output gate, which determines the cell state C at the current moment t How much information should be output.

[0119] After the LSTM network, a fully connected layer (Dense layer) and a sigmoid activation function are usually connected to convert the output h of the LSTM network into t Convert to the probability distribution of the sequence and then get the false samples.

[0120] Generator generation test case: In the process of generator Gα generating sequences, for each generation step, the generator can only see the current partial sequence Y 1:t (i.e. the first t generated elements). And the complete sequence Y 1:T It is necessary to wait until the generator completes all steps to obtain it. Therefore, Monte Carlo search is used to simulate the subsequent part of the partial sequence (i.e., the generation from the t+1th to the Tth step).

[0121] The specific implementation is:

[0122] Given the current partial sequence Y 1:t , use the generator Gα to generate multiple complete sequence samples Y through Monte Carlo search MCTS based on the currently generated characters (1) 1:T ,Y (2) 1:T ,...,Y (M) 1:T .

[0123] S3. Design two discriminators:

[0124] Discriminator Dγ: used to evaluate the pass rate of the generated test cases. The discriminator Dγ compares the generated test cases with the real test cases to determine whether the generated test cases can pass the actual application scenario requirements.

[0125] Discriminator Dλ: It is used to evaluate the diversity of the generated test cases. The discriminator Dλ is designed to ensure that the generated test cases are sufficiently different from each other to avoid the generation of repetitive test cases.

[0126] The discriminator model constructed by the present invention includes an embedding layer, a feature fusion layer introducing multi-size convolution kernels, an LSTM layer, a Highway layer, a Dense layer, and a sigmoid layer in sequence. Its input data is divided into two parts: one part is the false samples generated by the generator as negative sample input, and the other part is the real data as positive sample input.

[0127] This paper proposes an improved discriminator structure, which applies multi-size convolution kernels (1x1, 3x3, 5x5) to feature extraction in the embedding layer and LSTM layer. The multi-size convolution kernel captures local features of different scales, and the LSTM layer is responsible for global temporal dependencies. The discriminator's ability to understand and distinguish sequences is enhanced through feature fusion. This design not only improves the accuracy of the discriminator, but also improves the generalization ability of the generated model, making it perform better in complex sequence generation tasks.

[0128] The input sequence can be represented as X∈R T×d Where T is the time step and d is the feature dimension. Using multiple different convolution kernel sizes (for example, 1, 3, 5), the mathematical expression of the features generated by different convolution kernels is:

[0129] F1=Conv(X,W1)+b1

[0130] F2=Conv(X,W2)+b2

[0131] F3=Conv(X,W3)+b3

[0132] Among them, W1, W2, and W3 are convolution kernels of different scales 1×1, 3×3, and 5×5, respectively, and b1, b2, and b3 are bias items of convolution kernels of different scales;

[0133] Feature fusion: These multi-scale convolutional features are fused by splicing.

[0134] F=[F1;F2;F3]

[0135] Where [;] represents the splicing operation.

[0136] The multi-scale features extracted by the convolution kernel are concatenated and regarded as sequence features and input into the LSTM layer. The LSTM layer learns the dependencies between these features through time series modeling, thereby generating richer expressions.

[0137] Introducing the Highway Network after the LSTM layer can help the model better selectively transfer or transform features. The Highway Network allows some information to pass directly through the network in an unchanged form without undergoing nonlinear transformation, thereby retaining important low-level features.

[0138] The Highway layer is similar to a gated network layer, allowing the model to choose between transforming and keeping input information. It introduces two key gates: the transform gate and the carry gate. Its formula is:

[0139] y=transform(H)·T(H)+H·(1-T(H))

[0140] T(H) is the carrying gate, which determines the degree of input information retention and is defined as:

[0141] T(H)=σ(W g H+b g )

[0142] Where W g is another set of weight matrices, b g is the bias, σ is the sigmoid function, which is used to limit the output to the range of [0,1]. 1-T(H) is the complement of the transformation gate, which controls how much information is retained from the input H.

[0143] After the LSTM and Highway layers, the final output goes through a fully connected layer and a sigmoid activation function to obtain the target output probability distribution.

[0144] S4. Use the discriminator to evaluate the generated test cases:

[0145] The discriminator Dγ is used for each complete sequence Y generated by Monte Carlo (M) 1:T (M in total) to determine the pass rate and output the probability D that the test case meets the requirements of the application scenario γ (Y).

[0146] The discriminator Dλ is used for each complete sequence Y generated by Monte Carlo (M) 1:T (M in total) perform diversity determination and output the diversity probability D of the test case and other generated samples λ (Y).

[0147] Then the mixed reward can be expressed as:

[0148] Rtotal(Y)=βR γ (Y 1:t )+θR λ (Y 1:t )

[0149] Where β and θ are the reward weights of the discriminators Dγ and Dλ respectively. The simulated annealing algorithm and the sigmoid function are used to dynamically adjust the weights β and θ. If we want to dynamically adjust the temperature according to the time step in simulated annealing, we can use the following sigmoid temperature adjustment formula to get the corresponding weight β at a certain time step t: t and θ t

[0150]

[0151] θ t =1-β t

[0152] Among them, T max is the initial temperature, which controls the maximum temperature of simulated annealing. k controls the steepness of the sigmoid function. A larger k value will cause the temperature to drop quickly, while a smaller k value will cause the temperature to change more slowly. x0 is the midpoint of the sigmoid function, which indicates the time step where the "inflection point" of the temperature change is located.

[0153] The rewards of the discriminators Dγ and Dλ can be expressed as follows:

[0154]

[0155] The mixed reward output is obtained through the above formula.

[0156] S5. Design corresponding loss functions for the generator and the two discriminators, and perform gradient updates respectively to complete the training.

[0157] Generator gradient update:

[0158] In SeqGAN, the gradient update of the generator is based on the policy gradient method, the reward is obtained through the Monte Carlo search result above, and the loss function of the generator is:

[0159] loss G(α) =-E Y～G(α) [Rtotal(Y)logG (α) (Y)]

[0160] where Rtotal(Y) is the hybrid reward function defined above, G (α) (Y) is the probability of generating sequence Y. The negative sign indicates minimizing the loss, that is, maximizing the reward of the generator; E Y～G(α) Represents the expectation for generating the sequence Y.

[0161] The parameter update of the generator is performed by gradient descent, and the gradient calculation formula is as follows:

[0162]

[0163] Where Rtotal(Y 1:t ) is the mixed reward of the current partial sequence obtained by Monte Carlo search, G (α) (Y t |Y 1:t-1 ) is the generator that generates Y given t-1 sequences t The conditional probability of .

[0164] Through this loss function, the generator can adjust the generation strategy according to the mixed reward, which not only improves the pass rate of generated test cases but also increases the diversity of test cases.

[0165] Gradient update of the discriminator:

[0166] The parameters of the discriminator are trained by gradient updates, usually using the back-propagation algorithm to optimize its loss function. The loss function of the discriminator is:

[0167] loss D =-E Y～Pdata [logD(Y)]-E Y～Gα [log(1-D(Y))]

[0168] Among them, -E Y～Pdata [logD(Y)] represents the loss of the discriminator on the real sample. It is expected that the discriminator can increase the output probability D(Y) of the real sample to as close to 1 as possible. Y～Gα [log(1-D(Y))] represents the loss of the discriminator on the generated samples, and it is expected that the discriminator can reduce the output probability D(Y) of the generated samples to close to 0. By minimizing this loss function, the discriminator can better learn how to distinguish between real samples and generated samples, thereby improving the quality of the samples generated by the generator.

[0169] This embodiment also proposes a specific implementation of a protocol fuzzy testing system based on an improved SeqGAN, wherein: Figure 4 As shown, the overall framework of the system is divided into three main parts: test case generation module, fuzz test execution module and log module.

[0170] (1) Test case generation module

[0171] This part is responsible for generating test cases for fuzz testing, which includes the following steps:

[0172] Data collection: Collect real protocol messages from actual communication environments to ensure the authenticity and diversity of the generated data. This data can be obtained through network packet capture or from an existing protocol message library.

[0173] Data preprocessing: Encode and fill the collected data to improve the model's ability to capture contextual relationships in protocol messages and improve the generator's learning ability.

[0174] Model training: Use the cleaned data to train the generative model. The improved SeqGAN model is used for training to generate effective test cases.

[0175] Model running: After training is completed, run the trained model to generate actual fuzz test cases and prepare to send these cases to the fuzz test execution system.

[0176] (2) Fuzz test execution module

[0177] This part is responsible for executing the generated test cases and performing actual fuzz testing, and specifically includes the following components:

[0178] Client construction: Build a client to simulate the devices in the actual environment to send the generated test cases and receive responses.

[0179] Network connectivity: Ensure that the network connection between the client and the MQTT server and broker is stable so that test cases can be transmitted quickly and feedback can be received.

[0180] Listener: A listener is used to monitor the network response from the agent in real time, helping the execution system to understand the progress and results of the test in a timely manner. During the fuzzing process, all communication messages between the client and the agent will be tracked. The client will record the corresponding response after sending an MQTT packet to the agent, thereby monitoring the status of the agent.

[0181] (3) Log module

[0182] This section is responsible for recording and analyzing the results of test execution, including:

[0183] Pass rate: By analyzing the return code fed back by the agent, the pass rate of the sent test cases is calculated to evaluate the effectiveness of fuzz testing.

[0184] Unique messages: Record the unique messages fed back by the agent. These messages represent the diversity of test cases, thus helping to evaluate the comprehensiveness of the test.

[0185] Coverage: Measures the coverage of the code and provides a basis for subsequent test optimization.

[0186] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.< / pad>

Claims

1. A protocol fuzz testing method based on improved SeqGAN, characterized by: The method comprises the following steps: S1. Collect basic data and preprocess the data, where preprocessing includes data cleaning, data set division, data encoding, and data set filling; S2. Design a generator G to generate test cases α , where the generator G α The long short-term memory network LSTM is used to learn the spatiotemporal characteristics of the sequence and capture the long-distance dependencies in the time series; S3. Design the discriminators Dγ and Dλ for evaluating the pass rate and diversity of test cases respectively. The discriminator captures multi-scale local features through multi-scale convolution kernels and uses LSTM layer to capture global temporal dependencies. The two are fused to discriminate the test cases. S4. Design a hybrid reward and use two discriminators to evaluate the test cases based on the hybrid reward; S5. Design corresponding loss functions for the generator and the two discriminators, and perform gradient updates respectively to complete the training.

2. According to a protocol fuzzy testing method based on improved SeqGAN according to claim 1, it is characterized in that: In step S1, data collection refers to collecting real protocol messages in the actual communication environment through network packet capture or existing protocol message library; Data cleaning refers to removing noise and irrelevant information from the training data, fixing errors in the data, and handling missing values; Dataset partitioning refers to the re-division of the data set based on the entropy value of the message. The process includes: calculating the entropy value index for each message to measure the complexity and diversity of the message content; then, according to the entropy value, the data set is divided into two categories of messages: high entropy and low entropy. The high entropy message will be used as a positive sample of diversified messages, and the low entropy message will be used as a negative sample of non-diversified messages; Data encoding refers to converting discrete fields in protocol messages into high-dimensional vectors through embedding layers or pre-trained embedding models; Data padding refers to the padding operation on the message sequence. Let L max is the maximum length after padding, the actual length of message Y is L Y , then the filling operation P is expressed as: Here, <PAD> represents a special character used to fill in a message shorter than the maximum length.

3. According to a protocol fuzzy testing method based on improved SeqGAN according to claim 1, it is characterized in that: In step S2, the long short-term memory network LSTM is used as the generator G of the adversarial network α ,That The process includes: The real data samples are input into the embedding layer to map the discrete characters into a continuous low-dimensional vector space. The process is expressed as: e(x i )=W[x i ] Among them, x i represents a discrete symbol; e(x i ) is the continuous variable after mapping; W is the lookup matrix of the embedding layer, whose size is |V|×d, |V| is the vocabulary size, d is the vector dimension, and each row of W i The embedding vector corresponding to the i-th symbol in the vocabulary; The mapped continuous variable e(x i ) is passed to the LSTM layer to learn the time series features, where forward propagation is first performed in the LSTM layer: f t =σ(W f *[h t-1 ,X t ]+b f ) i t =σ(W i *[h t-1 ,X t ]+b i ) the t =σ(W o *[h t-1 ,X t ]+b o ) Among them, f t 、i t , o t Represent the forget gate, input gate and output gate respectively, through the hidden state h of the previous time step t-1 and the input X at the current time step t The linear transformation and nonlinear activation are obtained, * represents element multiplication, σ represents the sigmoid activation function, W f , W i , W o represents the weight matrix, b f , b i , b o represents bias; The corresponding gate is related to the previous cell state C t-1 and the current new cell state C t By element-by-element multiplication, the unit state update process is expressed as: h t =o t *tanh(C t ) The above process determines the cell state C at the current moment through the cell state controlled by the output gate. t The amount of information output h t ; where tanh(·) represents the activation function, W C represents the weight matrix, b C represents the bias term, Represents the candidate memory cell value; After the LSTM network, a fully connected layer and a sigmoid activation function are connected to convert the output h of the LSTM network into t Convert to the probability distribution of the sequence and get the test case.

4. According to claim 3, a protocol fuzzy testing method based on improved SeqGAN is characterized in that: In step S2, for each generation step, the generator G α It can only describe the first t generated elements, and uses Monte Carlo search to simulate the generated elements from the t+1th to the Tth step to obtain the complete sequence Y 1:T , the process is expressed as: Given the current partial sequence Y 1:t , use the generator Gα to generate multiple complete sequence samples Y through Monte Carlo search based on the currently generated characters (1) 1:T ,Y (2) 1:T ,...,Y (M) 1:T .

5. According to claim 4, a protocol fuzzy testing method based on improved SeqGAN is characterized in that: In step S3, the model structures of the two discriminators Dγ and Dλ are the same, and both include an embedding layer that introduces multi-size convolution kernels, a feature fusion layer, an LSTM layer, a Highway layer, a Dense layer, and a sigmoid layer in sequence; Among them, in the embedding layer, the input sequence is represented as XvR T×d , T is the time step, d is the feature dimension, and multiple convolution kernel sizes are used to extract features: F1=Conv(X,W1)+b1 F2=Conv(X,W2)+b2 F3=Conv(X,W3)+b3 Among them, W1, W2, and W3 are convolution kernels of different scales, and b1, b2, and b3 are bias items of convolution kernels of different scales; In the feature fusion layer, multi-scale convolutional features are fused by splicing: F=[F1;F2;F3] Among them, [;] represents the splicing operation; In the LSTM layer, the multi-scale features extracted by the convolution kernel are concatenated as sequence features, and the dependencies between the multi-scale features are learned through temporal modeling. There are two key gates in the Highway layer: the transformation gate and the carrying gate, and their formulas are: y=transform(H)·T(H)+H·(1-T(H)) T(H) is the carrying gate, which determines the degree of input information retention and is defined as: T(H)=σ(W g H+b g ) Where W g is another set of weight matrices, b g is the bias, σ is the sigmoid function; 1-T(H) is the complement of the transformation gate, which controls the degree of information preservation from the input H; After the LSTM and Highway layers, the final output passes through a fully connected layer and a sigmoid activation function to obtain the target output probability distribution.

6. According to claim 5, a protocol fuzzy testing method based on improved SeqGAN is characterized in that: In step S4, the discriminator Dγ is used to analyze each complete sequence Y generated by Monte Carlo. (M) 1:T Determine the pass rate and output the probability D that the test case meets the requirements of the application scenario γ (Y); Use the discriminator Dλ to analyze each complete sequence Y generated by Monte Carlo (M) 1:T (M in total) perform diversity determination and output the diversity probability D of the test case and other generated samples λ (Y); Among them, the mixed reward is expressed as: Rtotal(Y)=βR γ (AND 1:t )+θR λ (AND 1:t ) where β and θ are the reward weights of the discriminators Dγ and Dλ, respectively, and R γ (Y 1:t ) and R λ (Y 1:t ) represent the rewards of the discriminator Dγ and Dλ, which are respectively expressed as: Wherein, m represents the mth sequence among a total of M sequences.

7. The protocol fuzzy testing method based on improved SeqGAN according to claim 6, characterized in that: In step S4, the simulated annealing algorithm and the sigmoid function are combined to dynamically adjust the weights β and θ. In simulated annealing, the temperature is dynamically adjusted according to the time step. The following sigmoid temperature adjustment formula is used to obtain the corresponding weight β at a certain time step t: t and θ t : i t =1-β t Among them, T max is the initial temperature, which controls the maximum temperature of simulated annealing; k controls the steepness of the sigmoid function. The larger the k value, the faster the temperature changes; x0 is the midpoint of the sigmoid function, which indicates the time step where the "inflection point" of the temperature change is located.

8. The protocol fuzzy testing method based on improved SeqGAN according to claim 6, characterized in that: In step S5, for the update of the generator, the loss function of the generator is: loose G(α) =-E Y～G(α) [Rtotal(Y)logG (α) (Y)] Among them, Rtotal(Y) is the mixed reward function, G (α) (Y) is the probability of generating sequence Y, and the negative sign indicates minimizing the loss, that is, maximizing the reward of the generator; E Y～G(α) Represents the expectation for the generated sequence Y; The parameter update of the generator is performed by gradient descent, and the gradient calculation formula is as follows: Among them, Rtotal(Y 1:t ) is the mixed reward of the current partial sequence obtained by Monte Carlo search, G (α) (Y t |Y 1:t-1 ) is the generator that generates Y given t-1 sequences t The conditional probability of .

9. The protocol fuzz testing method based on improved SeqGAN according to claim 6, characterized in that: In step S5, for the gradient update of the discriminator, the loss function of the discriminator is: loss D =-E Y～Pdata [logD(Y)]-E Y～Gα [log(1-D(Y))] Among them, -E Y～Pdata [logD(Y)] represents the loss of the discriminator on the real sample, which aims to increase the output probability D(Y) of the real sample to close to 1; E Y～Gα [log(1-D(Y))] represents the loss of the discriminator on the generated samples, which aims to reduce the output probability D(Y) of the generated samples to close to 0.

10. A protocol fuzzy testing system based on improved SeqGAN, characterized by: The system comprises: a test case generation module, a fuzz test execution module and a log module, wherein the test case generation module is used to execute the protocol fuzz test method based on the improved SeqGAN according to any one of claims 1 to 9, and is used to generate test cases after the training is completed; The fuzz test execution module includes a client unit, a network connection unit, and a listener. The client is used to simulate the device in the actual environment to send the generated test cases and receive responses; the network connection unit is used for the network connection between the client and the MQTT server and agent; the listener is used to monitor the network response from the agent in real time; The log module is responsible for recording and analyzing the results of test execution, including pass rate, unique messages, and coverage.

Images