AI-Based Random Forest Malicious Traffic Detection Method and System

Abstract

The present invention provides an AI-based random forest malicious traffic detection method and system. By dimensionality reduction sampling of the data stream, a discretized data stream is obtained, which not only reduces the subsequent required computing speed but also greatly reduces the amount of computation. By invoking the syntactic model and semantic analysis model, the data stream can be automatically segmented and redundantly filtered to obtain a feature vector matrix, realizing the artificial intelligence and automation of feature extraction. Through convolutional neural network and random forest classification, the required feature vectors can be further highlighted, and the classification function with different classification integration capabilities can be realized, thus overcoming the problems of difficult detection of attacks that change at any time and huge amount of computation in the prior art.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular, to an AI random forest-based malicious traffic detection method and system. Background Art

[0002] With the rapid development of the network, the network has gradually penetrated into people's daily lives. However, malicious codes have also grown and developed accordingly. Network security issues have become increasingly prominent and presented an industrial chain trend. Currently, the known malicious code attacks pose relatively small threats. Instead, potential malicious codes can cause great damage, and their means and attack forms are constantly changing, making them difficult to detect.

[0003] At the same time, the scale of the sample data volume of malicious codes is getting larger and larger. It has also become a technical problem for machine learning to complete the training of a huge data set, and an improvement measure needs to be proposed to speed up the progress of machine learning.

[0004] Therefore, there is an urgent need for a targeted AI random forest-based malicious traffic detection method and system. Summary of the Invention

[0005] The purpose of the present invention is to provide an AI random forest-based malicious traffic detection method and system, which solves the problems that it is difficult to detect attacks with constantly changing means and forms in the prior art, and improves the detection method to speed up the model recognition speed.

[0006] In the first aspect, this application provides an AI random forest-based malicious traffic detection method, and the method includes:

[0007] Receiving the data stream sent by the acquisition terminal, extracting the content of the packet header field from the data stream, identifying different clients, and generating a separate identifier for each client;

[0008] Performing discretization processing on the data stream, sampling the data stream according to time-domain continuity, and obtaining a dimension-reduced discrete data stream;

[0009] Respectively establishing different containers according to the identifier for storing feature vectors corresponding to different clients;

[0010] Obtaining the discrete data stream, calling the syntactic model of the server, performing sentence segmentation, automatically querying the dictionary to obtain the first word component, and storing the first word component in the container corresponding to the identifier;

[0011] Extracting the first word component from the corresponding container according to the identifier corresponding to the required client, and inputting it into the semantic analysis model of the server one by one to receive the word meaning corresponding to the first word component returned;

[0012] Filter redundant information from the word meanings according to the first rule to obtain the corresponding second word components after filtering, and form the first word component matrix;

[0013] Input the first word component matrix into the input layer of the recognition model, calculate the standard deviations of different word classes, and the standard deviations are used to determine the width of the sliding window of the subsequent convolutional layer; the recognition model is a model architecture based on random forest and convolutional neural network;

[0014] The output of the input layer is sent into the convolutional layer of the recognition model. Different-sized sliding windows are used to select local word components in the text, and the local word components are spliced to obtain the second word component matrix, and the second word component matrix is sent into the pooling layer of the recognition model;

[0015] The pooling layer selects and distinguishes the effective eigenvalue of the word meaning by choosing a pooling function, and splices again to obtain the third word component matrix;

[0016] Transmit the third word component matrix that has completed the above processing to the random forest of the recognition model for classification. The random forest extracts the third word component matrix for n rounds to obtain n training sets, and uses the n extracted training sets to randomly use a specified amount of eigenvalues by column sampling to train n decision trees, and the n decision trees obtain the classification result by voting;

[0017] Judge whether the data stream sent by the acquisition terminal includes an attack vector according to the classification result. If it includes an attack vector, block the data stream, otherwise allow the data stream.

[0018] In a second aspect, the present application provides an AI random forest-based malicious traffic detection system, and the system includes:

[0019] A preprocessing module, configured to receive a data stream sent by an acquisition terminal, extract the content of the packet header field from the data stream, identify different clients, and generate a separate identifier for each client; perform discretization processing on the data stream, and sample the data stream according to time-domain continuity to obtain a discretized data stream with reduced dimensions;

[0020] A container module, configured to establish different containers according to the identifier for storing feature vectors corresponding to different clients;

[0021] An AI module is used to obtain the discrete data stream, call the syntactic model of the server to perform sentence segmentation, automatically query the dictionary to obtain the first word components, and store the first word components in the container corresponding to the identifier; according to the identifier corresponding to the required client, extract the first word components from the corresponding container, input them one by one into the semantic analysis model of the server, and receive the word meanings corresponding to the first word components returned; filter redundant information from the word meanings according to the first rule to obtain the corresponding second word components after filtering, and form a first word component matrix.

[0022] The recognition module includes a recognition model. The recognition model is a model architecture based on random forest and convolutional neural network, and is used to receive the first word component matrix output by the AI module, input it into the input layer of the recognition model, calculate the standard deviation of different word classes, and the standard deviation is used to determine the width size of the sliding window of the subsequent convolutional layer; the output of the input layer is sent into the convolutional layer of the recognition model, and different-sized sliding windows are used to select local word components in the text, splice the local word components to obtain a second word component matrix, and send the second word component matrix into the pooling layer of the recognition model; the pooling layer selects and distinguishes the effective eigenvalue of the word meaning by selecting a pooling function, and splices again to obtain a third word component matrix.

[0023] Transmit the third word component matrix that has completed the above processing to the random forest of the recognition model for classification. The random forest extracts the third word component matrix for n rounds to obtain n training sets, and uses the n extracted training sets to randomly use a specified number of eigenvalues for training by column sampling to obtain n decision trees. The n decision trees obtain the classification result by voting.

[0024] The execution module is used to determine whether the data stream sent by the acquisition terminal includes an attack vector according to the classification result. If it includes an attack vector, block the data stream, otherwise allow the data stream.

[0025] In a third aspect, the present application provides an AI random forest-based malicious traffic detection system, and the system includes a processor and a memory:

[0026] The memory is used to store program code and transmit the program code to the processor;

[0027] The processor is used to execute the method according to any one of the four possibilities in the first aspect according to the instructions in the program code.

[0028] In a fourth aspect, the present application provides a computer-readable storage medium, and the computer-readable storage medium is used to store program code, and the program code is used to execute the method according to any one of the four possibilities in the first aspect.

[0029] Beneficial effects

[0030] The present invention provides an AI-based random forest malicious traffic detection method and system. By dimensionality reduction sampling of the data stream, a discretized data stream is obtained, which not only reduces the subsequent required computing speed but also greatly reduces the amount of computation. By invoking the syntactic model and semantic analysis model, the data stream can be automatically segmented and redundant filtered to obtain a feature vector matrix, realizing the artificial intelligence and automation of feature extraction. Through convolutional neural network and random forest classification, the required feature vectors can be further highlighted and the classification function with different classification integration capabilities can be realized, thus overcoming the problems of difficult detection of attacks that change over time and huge amount of computation in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0032] Figure 1 It is a general flowchart of the AI-based random forest malicious traffic detection method of the present invention;

[0033] Figure 2 It is an architecture diagram of the AI-based random forest malicious traffic detection system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The following will elaborate on the preferred embodiments of the present invention in conjunction with the drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0035] Figure 1 It is a general flowchart of the AI-based random forest malicious traffic detection method provided by this application. The method includes:

[0036] Receiving the data stream sent by the acquisition terminal, extracting the content of the message header field from the data stream, identifying different clients, and generating a separate identifier for each client;

[0037] Performing discretization processing on the data stream, sampling the data stream according to time domain continuity to obtain a dimension-reduced discrete data stream;

[0038] Establishing different containers according to the identifiers for storing feature vectors corresponding to different clients;

[0039] Obtaining the discrete data stream, invoking the syntactic model of the server to perform sentence segmentation, automatically querying the dictionary to obtain the first word component, and storing the first word component in the container corresponding to the identifier.

[0040] Extract the first word component from the corresponding container according to the identifier corresponding to the required client, and input it into the semantic analysis model of the server one by one, and receive the word meaning corresponding to the first word component returned.

[0041] Filter redundant information from the word meaning according to the first rule to obtain the corresponding second word component after filtering, and form the first word component matrix.

[0042] Input the first word component matrix into the input layer of the recognition model, calculate the standard deviation of different word classes, and the standard deviation is used to determine the width size of the sliding window of the subsequent convolutional layer; the recognition model is a model architecture based on random forest and convolutional neural network.

[0043] The larger the standard deviation, the more significant the role of this word class in recognition (such as attack).

[0044] The output of the input layer is sent into the convolutional layer of the recognition model. Use sliding windows of different sizes to select local word components in the text, splice the local word components to obtain the second word component matrix, and send the second word component matrix into the pooling layer of the recognition model.

[0045] The pooling layer selects effective eigenvalue to distinguish the word meaning by choosing a pooling function, and splices again to obtain the third word component matrix.

[0046] Transmit the third word component matrix that has completed the above processing to the random forest of the recognition model for classification. The random forest resolves the feature matrix by decision trees, classifies it into different root directories, and makes inferences and judgments according to the knowledge and experience provided by experts, simulating the decision-making process of human experts.

[0047] The random forest extracts the third word component matrix for n rounds to obtain n training sets. Use the n extracted training sets to train n decision trees by randomly using a specified amount of eigenvalues by column sampling. The n decision trees obtain the classification result by voting.

[0048] Judge whether the data stream sent by the acquisition terminal includes an attack vector according to the classification result. If it includes an attack vector, block the data stream; otherwise, allow the data stream.

[0049] In some preferred embodiments, when the recognition model is trained, the entropy loss function is minimized by the backpropagation method to avoid oversaturation. When the accuracy of the recognition model meets the threshold requirement, it indicates that the recognition model training is completed. Then it can be used for data verification.

[0050] In some preferred embodiments, the classification ability of each decision tree is targeted, and the specified quantity of feature values is obtained according to different classifications. By classifying the same feature vector matrix through decision trees from different perspectives, the integration function for different classification abilities is completed. Its classification performance is higher than that of a single classifier.

[0051] The average generalization error of a decision tree in a random forest is related to the regression function.

[0052] In some preferred embodiments, the voting method includes weighted accumulation of the output results of each decision tree.

[0053] Figure 2 The following is an architecture diagram of the AI random forest malicious traffic detection system provided by this application. The system includes:

[0054] A preprocessing module, which is used to receive the data stream sent by the acquisition terminal, extract the content of the packet header field from the data stream, identify different clients, and generate a separate identifier for each client; perform discretization processing on the data stream, sample the data stream according to time-domain continuity, and obtain the dimension-reduced discrete data stream;

[0055] A container module, which is used to establish different containers according to the identifier, and is used to store the feature vectors corresponding to different clients;

[0056] An AI module, which is used to obtain the discrete data stream, call the syntactic model of the server, perform sentence segmentation, automatically query the dictionary to obtain the first word component, and store the first word component in the container corresponding to the identifier; according to the identifier corresponding to the required client, extract the first word component from the corresponding container, and input it into the semantic analysis model of the server one by one, and receive the word meaning corresponding to the first word component returned; filter redundant information from the word meaning according to the first rule to obtain the corresponding second word component after filtering, and form the first word component matrix;

[0057] An identification module, including an identification model. The identification model is a model architecture based on a random forest and a convolutional neural network. It is used to receive the first word component matrix output by the AI module, input it into the input layer of the identification model, calculate the standard deviation of different word classes, and the standard deviation is used to determine the width size of the sliding window of the subsequent convolutional layer; the output of the input layer is sent into the convolutional layer of the identification model, and different-sized sliding windows are used to select local word components in the text, splice the local word components to obtain the second word component matrix, and send the second word component matrix into the pooling layer of the identification model; the pooling layer selects and distinguishes the effective eigenvalue of the word meaning by selecting a pooling function, and splices it again to obtain the third word component matrix;

[0058] The third word component matrix after the above processing is transmitted to the random forest of the recognition model for classification. The random forest extracts the third word component matrix for n rounds to obtain n training sets. Using the n extracted training sets, n decision trees are trained by randomly using a specified number of eigenvalues through column sampling. The n decision trees obtain a classification result by voting;

[0059] An execution module, configured to determine whether the data stream sent by the collection terminal includes an attack vector according to the classification result. If it includes an attack vector, the data stream is blocked; otherwise, the data stream is allowed.

[0060] This application provides an AI-based random forest malicious traffic detection system, and the system includes: a processor and a memory in the system:

[0061] The memory is used to store program codes and transmit the program codes to the processor;

[0062] The processor is configured to execute the method described in any one of all the embodiments of the first aspect according to the instructions in the program codes.

[0063] This application provides a computer-readable storage medium, and the computer-readable storage medium is used to store program codes, and the program codes are used to execute the method described in any one of all the embodiments of the first aspect.

[0064] In specific implementation, the present invention further provides a computer storage medium. Among them, the computer storage medium can store a program, and when the program is executed, it may include some or all of the steps in each embodiment of the present invention. The storage medium may be a magnetic disk, an optical disc, a read-only memory (abbreviation: ROM) or a random access memory (abbreviation: RAM), etc.

[0065] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions in the embodiments of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments of the present invention.

[0066] For the same and similar parts between the embodiments of this specification, reference can be made to each other. In particular, for the embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the description in the method embodiments.

[0067] The above-described embodiments of the present invention do not limit the protection scope of the present invention.

Claims

1. An AI-based random forest malicious traffic detection method, characterized in that, The method includes: Receiving a data stream sent by a collection terminal, extracting the content of the message header field from the data stream, identifying different clients, and generating a separate identifier for each client; Performing discretization processing on the data stream, sampling the data stream according to time-domain continuity to obtain a discrete data stream with reduced dimensions; Respectively establishing different containers according to the identifier for storing feature vectors corresponding to different clients; Obtaining the discrete data stream, calling the syntactic model of the server to perform sentence segmentation, automatically querying the dictionary to obtain the first word component, and storing the first word component in the container corresponding to the identifier; Extracting the first word component from the corresponding container according to the identifier corresponding to the required client, inputting it into the semantic analysis model of the server one by one, and receiving the word meaning corresponding to the first word component returned; Filtering redundant information from the word meaning according to the first rule to obtain the corresponding second word component after filtering, and forming a first word component matrix; Inputting the first word component matrix into the input layer of the recognition model, calculating the standard deviation of different word classes, and the standard deviation is used to determine the width size of the sliding window of the subsequent convolutional layer; the recognition model is a model architecture based on random forest and convolutional neural network; The output of the input layer is sent into the convolutional layer of the recognition model, using sliding windows of different sizes to select local word components in the text, splicing the local word components to obtain a second word component matrix, and sending the second word component matrix into the pooling layer of the recognition model; The pooling layer selects and distinguishes effective eigenvalue of the word meaning by choosing a pooling function, and splices again to obtain a third word component matrix; Transmitting the third word component matrix after the above processing to the random forest of the recognition model for classification. The random forest extracts the third word component matrix for n rounds to obtain n training sets, and uses the n extracted training sets to train n decision trees by randomly using a specified amount of eigenvalues by column sampling. The n decision trees obtain a classification result by voting; Judging whether the data stream sent by the collection terminal includes an attack vector according to the classification result. If it includes an attack vector, block the data stream, otherwise allow the data stream.

2. The method according to claim 1, wherein: When the recognition model is trained, it minimizes the entropy loss function through the backpropagation method to avoid oversaturation. When the accuracy of the recognition model meets the threshold requirement, it indicates that the recognition model training is completed.

3. The method according to claim 1, wherein: The classification ability of each decision tree is targeted. The specified amount of eigenvalues is obtained according to different classifications. Classifying the same feature vector matrix by the decision tree from different angles can complete the integration function for different classification abilities.

4. The method according to any one of claims 2 or 3, characterized in that: The voting method includes weighted accumulation of the output results of each decision tree.

5. An AI-based random forest malicious traffic detection system, characterized in that, The system includes: A preprocessing module, configured to receive a data stream sent by a collection terminal, extract the content of the message header field from the data stream, identify different clients, and generate a separate identifier for each client; perform discretization processing on the data stream, sample the data stream according to time-domain continuity to obtain a discrete data stream with reduced dimensions; A container module for respectively creating different containers according to the identifiers, which are used to store the feature vectors corresponding to different clients; An AI module for obtaining the discrete data stream, calling the syntactic model of the server to perform sentence segmentation, automatically querying the dictionary to obtain the first word component, and storing the first word component into the container corresponding to the identifier; extracting the first word component from the corresponding container according to the identifier corresponding to the required client, inputting it into the semantic analysis model of the server one by one, and receiving the word meaning corresponding to the first word component returned; filtering redundant information from the word meaning according to the first rule to obtain the corresponding second word component after filtering, and forming the first word component matrix; An identification module, including an identification model, the identification model is a model architecture based on a random forest and a convolutional neural network, which is used to receive the first word component matrix output by the AI module, input it into the input layer of the identification model, calculate the standard deviation of different word classes, and the standard deviation is used to determine the width size of the sliding window of the subsequent convolutional layer; the output of the input layer is sent into the convolutional layer of the identification model, and different sizes of sliding windows are used to select the local word components in the text, and the local word components are spliced to obtain the second word component matrix, and the second word component matrix is sent into the pooling layer of the identification model; the pooling layer selects and distinguishes the effective eigenvalue of the word meaning by selecting a pooling function, and splices it again to obtain the third word component matrix; Transmitting the third word component matrix that has completed the above processing to the random forest of the identification model for classification. The random forest extracts the third word component matrix for n rounds to obtain n training sets, and uses the n extracted training sets to randomly use a specified amount of eigenvalues by column sampling to train n decision trees, and the n decision trees obtain the classification result by voting; An execution module for judging whether the data stream sent by the acquisition terminal includes an attack vector according to the classification result. If it includes an attack vector, the data stream is blocked, otherwise the data stream is allowed.

6. An AI-based random forest malicious traffic detection system, characterized in that, The system includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the method according to any one of claims 1-4 according to the instructions in the program code.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code, and the program code is used to execute the method according to any one of claims 1-4.

Images