Protocol fuzzy test optimization method based on deep learning

Abstract

The invention relates to a protocol fuzzy test optimization method based on deep learning, and belongs to the technical field of networks. The method comprises the following steps: acquiring data acquired by Boofuzz in a fuzzy test process before optimization, and classifying message data with different responses according to server response types; a dynamic field, a static field and a length field of a protocol are preliminarily identified through a protocol reverse module, a request is constructed through identified information in a Boofuzz framework, and a session message is set; a variation rule in the Boofuzz framework is used, and a test case is generated according to a field deduced by a protocol reverse module; constructing an optimal selector model, and initializing model parameters; model hyper-parameters are adjusted and optimized through an SSA algorithm, and then model training is carried out based on the optimal hyper-parameters; performing feature extraction on the generated test case to obtain input features; and performing validity prediction on the test case by using the optimizer model.

Description

Technical Field

[0001] The present invention belongs to the field of network technology and relates to a protocol fuzzy testing optimization method based on deep learning. Background Art

[0002] Among the methods for discovering network protocol vulnerabilities, fuzz testing is currently the most commonly used and effective technology. Tools such as SPIKE, Peach, and Boofuzz ​​are all developed based on this technology. The core of fuzz testing is to input different test cases into the target and monitor its responses to discover potential vulnerabilities. Therefore, the quality of the test cases directly affects the test effect and efficiency. Many protocol fuzz testing tools, such as Boofuzz ​​and Peach, require users to define the protocol format. Without defining the protocol format, these tools can only perform mutations based on bytes, but this method will produce many invalid mutations, especially when there is a lack of context information, and may not trigger effective vulnerabilities. However, byte-by-byte mutation consumes computing resources and is prone to generate a large number of redundant test cases, which is particularly prominent when dealing with complex protocols and will lead to an explosion of test paths. Fuzz testing also faces a large number of constraints, which come from complex logic such as dependency of protocol fields and state transitions, which increases the difficulty and time-consuming of solving.

[0003] In summary, protocol reverse engineering plays an important role in fuzz testing tools. It extracts protocol structure and field information through packet analysis, which helps improve the accuracy and effectiveness of testing, especially when dealing with complex or closed-source protocols. Current sequence alignment algorithms are mostly used for DNA sequence or text string alignment, and usually do not consider the deviation of character positions, which is also a limitation.

[0004] In addition, the main role of artificial intelligence technology in fuzz testing is to generate and filter test cases. Through genetic algorithms and neural networks, invalid or low-priority test cases can be automatically filtered out to improve test efficiency and coverage. However, the current deep learning models in fuzz testing are usually simple and the accuracy needs to be improved. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a protocol fuzz testing optimization method based on deep learning to solve the problems of difficulty in fuzz testing tools to identify protocol formats, explosion of test case generation paths, and complex constraint solving.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A protocol fuzz testing optimization method based on deep learning, the method comprising:

[0008] S1. Collect the data collected by Boofuzz ​​during the fuzz testing process before optimization, and classify the message data with different responses according to the server response type;

[0009] Among them, the classification of message data includes normal test cases, invalid test cases and abnormal test cases. Normal test cases are test cases that can respond normally; invalid test cases are test cases with no response, no reply or incorrect response message notification format; abnormal test cases are test cases with incorrect response;

[0010] S2. Preliminarily identify the dynamic fields, static fields, and length fields of the protocol through the protocol reversal module, and construct requests and set session messages in the Boofuzz ​​framework based on the identified information; use the mutation rules in the Boofuzz ​​framework to generate test cases based on the fields inferred by the protocol reversal module;

[0011] S3, build a selector model and initialize model parameters; adjust and optimize model hyperparameters through the SSA algorithm, and then train the model based on the optimal hyperparameters;

[0012] S4. Extract features from the generated test cases to obtain input features; and use the optimizer model to predict the effectiveness of the test cases.

[0013] Further, in step S2, preliminarily identifying the dynamic field, static field and length field of the protocol by the protocol reverse module includes: sorting the messages in ascending order of length, calling the improved SmithWaterman algorithm to extract the format string for two adjacent messages each time, until only one string is left, and obtaining the static field of the protocol;

[0014] Randomly select several messages, match them with the extracted format string, split the format string, and retain the part with shorter subfields;

[0015] Infer the length field by exhaustive enumeration;

[0016] According to the above steps, the format fields of the remaining categories of messages are obtained. Taking the normal response category messages as the standard, the message data of other categories are used for sequence comparison, and the fields divided into different categories are compared to make the field segmentation more delicate.

[0017] Among them, the improved Smith Waterman algorithm is expressed as:

[0018]

[0019]

[0020] In the formula, Q ij Indicates the total score, Sij represents the byte matching score, o represents the gap penalty, P represents the relative position penalty function, distance represents the distance between two different characters, and lenth represents the total length of the message.

[0021] Furthermore, in step S3, the optimizer model is a model combining multiple deep learning networks, wherein the deep learning network includes a convolutional neural network, a bidirectional long short-term memory network, an attention mechanism, and a Kolmogorov-Arnold network. By combining different network structures and adopting a dual-channel structure, while combining the advantages of CNN and LSTM, the accuracy and convergence speed of the model are increased, and the model can process complex temporal and spatial feature data and uses fewer parameters than MLP.

[0022] Furthermore, in step S3, the hyperparameters of the optimizer model are adjusted by the SSA algorithm, including:

[0023] 1) Collect message data of different response types during the communication process, and clean, format and normalize the data;

[0024] 2) Adjust the model weights by minimizing the loss function, and adjust the model parameters according to the loss value through the Adam optimizer to improve the classification accuracy of the model;

[0025] 3) Use SSA algorithm to optimize hyperparameters.

[0026] Wherein, the loss function is expressed as:

[0027]

[0028] In the formula, N represents the number of training samples, C represents the number of categories; ic represents the true label, which represents the true category of sample i. If sample i belongs to category c, then y ic =1, otherwise y ic =0;o ic Represents the probability output of category c predicted by the model, that is, the output of the Softmax layer.

[0029] In step 3), the SSA algorithm is used to optimize the hyperparameters, including:

[0030] ① Initialize the population and generate a set of random solutions, where each solution is a hyperparameter vector X i , represents the initial position of each individual;

[0031] ②Evaluate the performance of each solution through model training and testing, and the fitness of each individual is evaluated by the error rate of the model:

[0032]

[0033] In the formula, M represents the number of test set samples, y m represents the true label of the mth sample, o m represents the model prediction label of the mth sample; II(y m ≠o m ) represents the indicator function, when y m ≠o m When , it takes 1, indicating that the prediction is wrong;

[0034] ③ Update the best solution through iteration, and finally output the hyperparameter combination with the lowest error rate; perform explorer update, follower update and reconnaissance warning operations during the iteration process, so that the followers can move closer to the current optimal solution for local development and optimization; the end condition of the iteration is to reach the preset maximum number of iterations t max Or the fitness change is no longer significant.

[0035] The beneficial effects of the present invention are as follows: the present invention adds a protocol reversal module and a selector module on the basis of the Boofuzz ​​framework, wherein the protocol reversal module automatically identifies the protocol format, and then generates test cases through the identified format template, and the selector module uses the SSA-KAN-CNN-BiLSTM-Attention deep learning model to predict the validity of the test cases, eliminates the test cases predicted to be invalid, and gives priority to the valid test cases that may find vulnerabilities for testing, thereby improving the pass rate of the test cases and the efficiency of vulnerability mining. The present invention can solve the problems of difficult protocol identification, path explosion and complex constraint solving, and the present invention is applicable to other fuzz testing tools that require pre-defined protocol formats.

[0036] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0038] Figure 1 is a structural block diagram of the method of the present invention;

[0039] Figure 2 This is a schematic diagram of the KAN-CNN-BI-LSTM-Attention model structure;

[0040] Figure 3 This is a schematic diagram of the LSTM network structure;

[0041] Figure 4 Schematic diagram of Bi-LSTM network structure;

[0042] Figure 5 It is a schematic diagram of the SSA algorithm flow;

[0043] Figure 6 This is a schematic diagram of the fuzz testing process. DETAILED DESCRIPTION

[0044] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0045] like Figure 1 As shown, an embodiment of the present invention provides a protocol fuzzy testing optimization method based on deep learning, and the method is specifically described as follows:

[0046] Step 1: Collect the data captured during the Boofuzz ​​fuzz testing process before optimization, classify it according to its server response type, and use it for protocol reversal and deep learning modules.

[0047] The captured data is classified according to its server response type. Under normal circumstances, it is divided into normal test cases, invalid test cases, and abnormal test cases, which are defined as follows:

[0048] Invalid test case: no response, no reply, or the response message notification format is incorrect.

[0049] Normal test cases: test cases that can respond normally.

[0050] Abnormal test case: error response but the response message does not contain the format error flag information.

[0051] Step 2: Design a protocol reverse analysis module that can preliminarily identify the dynamic fields, static fields, and length fields of the protocol, and construct requests in the Boofuzz ​​framework based on the identified information, set the session message template, and use the mutation rules in the Boofuzz ​​framework to generate test cases based on the fields inferred by the protocol reverse analysis module.

[0052] Construct a protocol reverse module and use the improved multiple sequence comparison method to extract the protocol format of each type of message obtained in step 1. The following steps are included:

[0053] (1) Sort the messages in ascending order of length, and call the improved Smith Waterman algorithm to extract the format string for two adjacent messages each time until only one string remains, which is the string of the same format strings merged, namely the protocol static field.

[0054] Among them, the Smith Waterman algorithm is a dynamic programming algorithm based on matrix operations. It first gives a function to calculate the similarity score of two strings. After the match is successful, the optimal matching result is found by backtracking. Its state transition function is defined as follows:

[0055]

[0056] Among them, S ij is the byte match score, and o is the gap penalty.

[0057] The disadvantage of the Smith Waterman algorithm is that when there is an indefinite length message, it will cause a false match because the relative position difference of the match is too large. Therefore, the present invention adjusts the distance score function and adds a relative position penalty function P. The higher the position offset, the stronger the penalty. The improved Smith Waterman algorithm is as follows:

[0058]

[0059]

[0060] In the formula, distance represents the distance between two different characters, and lenth refers to the total length of the message. When the distance is less than 10% of the sequence length, the penalty value is 0.1 times the distance, that is, a slight penalty. When the distance is between 10% and 30%, the penalty value is the distance itself. When the distance is greater than 30% of the sequence length, the penalty value is the square of the distance, that is, a heavy penalty.

[0061] (2) Splitting the format string: Randomly select several messages, match them with the extracted format string, and split the format string, retaining the shorter subfield part because the shorter field is more representative.

[0062] (3) Use the exhaustive method to infer the length field.

[0063] The specific process of determining the protocol length field is as follows: First, all sequences with fixed lengths need to be excluded. In this case, the static field will be considered by default. Therefore, the present invention discusses the method of obtaining variable-length messages, and generally, the length field refers to the length from the length field to the end field. Therefore, first, all aligned one- to three-byte field values ​​are exhaustively enumerated, and then the length len(v) of all sequences is calculated. In the process of inferring the length field, the empty space needs to be ignored. Finally, determine whether the following equation is satisfied:

[0064] len(v)=v+C

[0065] Where v represents a candidate value of the length field, and C is a constant, which is usually used to adjust the offset of the message header or other fixed fields. If the total length of the message matches the length field plus the constant, it can be determined that the field is the length field.

[0066] (4) The format fields of the other two types of messages obtained by the method in the first three steps are taken as the standard for the normal response category, and then the sequences are compared with the data of other categories respectively, so as to make the field segmentation more delicate.

[0067] Step 3: Design the optimizer module, build the KAN-CNN-BI-LSTM-Attention model, and initialize the model parameters.

[0068] like Figure 2 As shown in the figure, the KAN-CNN-BI-LSTM-Attention model is a hybrid model architecture that combines multiple deep learning technologies, including convolutional neural network (CNN), bidirectional long short-term memory network (Bi-LSTM), attention mechanism (Attention) and Kolmogorov-Arnold network (KAN). Through the combination of different network structures and the use of a dual-channel structure, while combining the advantages of CNN and LSTM, the accuracy and convergence speed of the model are increased. The model can process complex temporal and spatial feature data and uses fewer parameters than MLP.

[0069] like Figure 3 As shown, in the LSTM network, its forget gate determines how much information is retained in the cell state:

[0070] f t =sigm(x t W f +U f h t-1 +b f )

[0071] Among them, f t represents the output of the forget gate, x tIndicates the current state input, W f represents the current weight matrix, h t-1 Indicates the previous state, U f Represents the weight matrix of the previous state, b f Represents the bias term.

[0072] The input gate controls how much of the input information is written into the cell state:

[0073] i t =sigm(x t W i +U i h t-1 +b i )

[0074] g t =tanh(x t W g +U g h t-1 +b g )

[0075] Among them, i t , g t All are input gate inputs.

[0076] Candidate cell state: Calculate the candidate cell state at the current moment. This information will determine the new cell state together with the input gate:

[0077] c t =(f t *c t-1 +i t *g t )

[0078] where c t is the output of the current cell state.

[0079] The output gate controls the output at the current moment, that is, the hidden state of the LSTM unit:

[0080] o t =sigm(x t W o +U o h t-1 +b o )

[0081] Among them t is the output of the output gate.

[0082] Hidden State Update: Calculate the new hidden state:

[0083] h t=sigm(o t *tanh(c t ))

[0084] where h t In hidden state.

[0085] Similarly, the bidirectional long short-term memory network is an improved recurrent neural network (RNN), which is particularly suitable for processing sequence data. Unlike the traditional LSTM, Bi-LSTM processes input data in two directions to capture richer contextual information.

[0086] Bidirectional processing: Bi-LSTM processes the input sequence forward and backward through two independent LSTM layers. The forward LSTM layer processes data from the beginning to the end of the sequence, and the backward LSTM layer processes data from the end to the beginning of the sequence. This enables Bi-LSTM to utilize past and future information of the input data and capture more comprehensive features, such as Figure 4 shown.

[0087] The Bi-LSTM network is represented as follows

[0088]

[0089]

[0090]

[0091] in, For the forward LSTM output result, is the reverse LSTM output result, y t is the final output.

[0092] CNN can automatically extract local patterns or feature combinations between fields in a message through convolution operations. These patterns may appear repeatedly in multiple messages. The attention mechanism is a technology developed in the field of deep learning in recent years to improve the model's ability to process sequence data. KAN proposes a new AI architecture that is different from MLP. It provides a more flexible activation function setting than MLP, has higher accuracy and interpretability, and is especially suitable for small-scale scientific tasks. The advantage of KAN is that it can achieve higher accuracy with fewer parameters and has very good interpretability.

[0093] Step 4: Introduce the sparrow optimization algorithm (SSA), simulate the foraging behavior of sparrows, adjust the hyperparameters to optimize the model performance, find the optimal parameter set and the optimal model, and then conduct model training. Specifically include:

[0094] 1. Collect message data of different response types during the communication process, and clean, format and normalize the data, mainly including the following key operations:

[0095] (1) Data cleaning is a key step in data preprocessing, which aims to remove invalid or inaccurate data that affects model training. First, by checking and deleting duplicate data in the dataset, the same information can be avoided from appearing multiple times, which helps prevent the model from being biased.

[0096] (2) Formatting: All collected data are normalized to a fixed length, zero-padded for rows that are shorter than the length, and rows that exceed the length are filtered out. Then, labels are added to each piece of data.

[0097] (3) Normalization: Save the processed data into a unified CSV file, which contains all processed feature data and label information to ensure consistent data format for subsequent model training and testing. The normalization formula is as follows:

[0098]

[0099] Among them, X is the original data value, X min is the minimum value of the data, X max is the maximum value of the data, X nomal is the normalized data value, usually in the range of [-1, 1].

[0100] This step ensures the format consistency, integrity, and traceability of the input data, providing high-quality standardized data for subsequent model training and testing.

[0101] 2. Conduct model training.

[0102] For the KAN-CNN-BI-LSTM-Attention model, the objective function of the classification model is the loss function, and its formula is as follows:

[0103]

[0104] Where N is the number of training samples, C is the number of categories; ic is the true label, indicating the true category of sample i. If sample i belongs to category c, then y ic =1, otherwise y ic =0;o ic The probability output of category c predicted by the model is the output of the Softmax layer. The Softmax function is used for multi-classification problems to convert the output of the model into a probability distribution. The formula is as follows:

[0105]

[0106] Among them, z is the score of the i-th category, and Softmax converts these scores into probability values ​​so that the sum of the probabilities of all categories is 1. During the training process, the model gradually adjusts the weights by minimizing the loss value so that the predicted value is as close to the true label as possible. The Adam optimizer adjusts the model parameters according to the loss value to improve the classification accuracy of the model.

[0107] 3. Use SSA for hyperparameter optimization. The core steps of SSA include population initialization, fitness evaluation, and iterative optimization, such as Figure 5 As shown, the SSA algorithm includes the following steps:

[0108] (1) Population initialization: In the population initialization step, a set of random solutions is generated. Each solution is a hyperparameter vector X i , represents the initial position of each individual:

[0109] X i =θ1,θ2,…,θ d

[0110] Where d represents the number of hyperparameters to be optimized, θ j represents the value of the jth hyperparameter. The initial value of each hyperparameter is randomly generated to meet its upper and lower limits:

[0111]

[0112] (2) Fitness evaluation: The performance of each solution is evaluated through model training and testing. The fitness of each individual is evaluated by the error rate of the model. The formula is:

[0113]

[0114] Among them, M is the number of test set samples, y m is the true label of the mth sample, o m is the model predicted label of the mth sample, II(y m ≠o m ) is the indicator function, when y m ≠o m When , it takes 1, indicating that the prediction is wrong.

[0115] 3. Iterative optimization: Continuously update the best solution in several rounds of iterations, and finally output the hyperparameter combination with the lowest error rate, including:

[0116] Explorer update,The update method of the explorer in the group is based on the global search of its current state. The formula is as follows:

[0117]

[0118] where \(t\) represents the current iteration number; \(j\) represents the dimension of the input variable; \(t\) max is set as a constant, representing the maximum iteration number of the algorithm; represents the position of the \(i\)-th sparrow in the \(j\)-th dimension during the \(t\)-th iteration process; \(\alpha\) is a random number in \([0, 1)\); \(Q\) follows a normal distribution. \(R_2\) and \(ST\) represent the warning value and the safety value respectively. When \(R_2 < ST\), it means that the foraging environment is in a safe area, and the discoverer can search globally; if \(R_2\geq ST\), it means that the foraging environment is no longer safe, and all individuals must take actions to change their positions to avoid being preyed on.

[0119] Update of followers. The joiners in the foraging state will continuously observe the movements of the discoverers. Once they obtain a better foraging position, they will immediately go to compete for food. If they win, they will increase their energy value; otherwise, they will repeat the process. The position update method of the joiners is as follows:

[0120]

[0121] where, is the current globally worst position; is the current optimal position of the discoverer; \(A\) + is a one-dimensional matrix, and the elements in it are randomly assigned as 1 or -1. In the above formula, when , it indicates that the current joiner does not obtain food due to a lower fitness value, and thus needs to go to other areas to forage for more energy; in other cases, it indicates that the sparrow randomly finds a position near the current optimal position.

[0122] Reconnaissance and warning. When the foraging environment where the sparrows are located is no longer safe, sparrows in different positions will adopt different movement strategies, and its mathematical expression is:

[0123]

[0124] where \(\beta\) is a random number conforming to a normal distribution, \(K\) is a random number between \([-1, 1]\), and \(\varepsilon\) is a relatively small number to prevent the denominator from being 0. \(f\) worst is the worst fitness value of the current population. \(f\) g is the global optimal fitness value. When \(f\) i \(\neq f\) g , the sparrow is at the edge of the population and is vulnerable to attacks by foragers. At this time, it flies towards the optimal sparrow; when \(f\) i \(= f\) g , the sparrow is in a dangerous position and moves away from the worst sparrow position.

[0125] Through these update formulas, the followers approach the current optimal solution to conduct local development and optimization.

[0126] The iteration ends when one of the following conditions is met:

[0127] 1) Reach the preset maximum number of iterations t max .

[0128] 2. The fitness change is no longer significant.

[0129] The optimal hyperparameters of the model are obtained through the SSA optimization algorithm, and the optimizer model is trained using the hyperparameters.

[0130] Step 5: Figure 6 As shown in the figure, the server is tested, and the generated test cases are feature extracted to obtain input features. The KAN-CNN-BI-LSTM-Attention model trained with the optimal parameters obtained in step 4 is used to predict the validity of the test cases. Test cases predicted as valid are filtered out, and test cases predicted as invalid are sent to the server. At the same time, valid test cases are sent to the server, and monitors are added to evaluate the performance of the framework by detecting the response of the object under test, the pass rate of the calculated test cases, the total number of states found, and the number of vulnerabilities found.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A protocol fuzz testing optimization method based on deep learning, characterized in that: The method includes: Collect the data collected by Boofuzz ​​during the fuzz test before optimization, and classify the message data with different responses according to the server response type; The protocol reversal module is used to preliminarily identify the dynamic fields, static fields, and length fields of the protocol, and a request is constructed in the Boofuzz ​​framework using the identified information to set the session message; the mutation rules in the Boofuzz ​​framework are used to generate test cases based on the fields inferred by the protocol reversal module; Build a selector model and initialize model parameters; adjust and optimize model hyperparameters through the SSA algorithm, and then train the model based on the optimal hyperparameters; Feature extraction is performed on the generated test cases to obtain input features; and the effectiveness of the test cases is predicted using the optimizer model.

2. The method according to claim 1, characterized in that The preliminary identification of the dynamic field, static field and length field of the protocol by the protocol reverse module includes: sorting the messages in ascending order of length, calling the improved Smith Waterman algorithm to extract the format string for two adjacent messages each time, until only one string is left, and obtaining the static field of the protocol; Randomly select several messages, match them with the extracted format string, split the format string, and retain the shorter part of the subfield; Infer the length field by exhaustive enumeration; According to the above steps, the format fields of the remaining categories of messages are obtained. Taking the normal response category messages as the standard, the message data of other categories are used for sequence comparison, and the fields divided into different categories are compared to make the field segmentation more delicate.

3. The method according to claim 2, characterized in that The improved Smith Waterman algorithm is expressed as: In the formula, Q ij Indicates the total score, S ij represents the byte matching score, o represents the gap penalty, P represents the relative position penalty function, distance represents the distance between two different characters, and lenth represents the total length of the message.

4. The method according to claim 1, characterized in that The optimizer model is a model that combines multiple deep learning networks, wherein the deep learning network includes a convolutional neural network, a bidirectional long short-term memory network, an attention mechanism, and a Kolmogorov-Arnold network.

5. The method according to claim 4, characterized in that The hyperparameters of the optimizer model are adjusted and optimized by the SSA algorithm, including: 1) Collect message data of different response types during the communication process, and clean, format and normalize the data; 2) Adjust the model weights by minimizing the loss function, and adjust the model parameters according to the loss value through the Adam optimizer to improve the classification accuracy of the model; 3) Use SSA algorithm to optimize hyperparameters.

6. The method according to claim 5, characterized in that The loss function is expressed as: In the formula, N represents the number of training samples, C represents the number of categories; ic represents the true label, which represents the true category of sample i. If sample i belongs to category c, then y ic =1, otherwise y ic =0;o ic Represents the probability output of category c predicted by the model, that is, the output of the Softmax layer.

7. The method according to claim 5, characterized in that The use of the SSA algorithm to optimize hyperparameters includes: 1) Initialize the population and generate a set of random solutions, where each solution is a hyperparameter vector X i , represents the initial position of each individual; 2) The performance of each solution is evaluated through model training and testing, and the fitness of each individual is evaluated by the error rate of the model: In the formula, M represents the number of test set samples, y m represents the true label of the mth sample, o m represents the model prediction label of the mth sample; II(y m ≠o m ) represents the indicator function, when y m ≠o m When , it takes 1, indicating that the prediction is wrong; 3) Update the best solution through iteration, and finally output the hyperparameter combination with the lowest error rate; perform explorer update, follower update and reconnaissance warning operations during the iteration process, so that the followers can move closer to the current optimal solution for local development and optimization; the end condition of the iteration is to reach the preset maximum number of iterations t max Or the fitness change is no longer significant.

8. The method according to claim 1, characterized in that The classification of the message data includes normal test cases, invalid test cases and abnormal test cases; The normal test case is a test case that can respond normally; the invalid test case is a test case with no response, no reply or incorrect response message notification format; the abnormal test case is a test case with an incorrect response.

Images