Malicious software detection method and system

Abstract

The invention provides a malicious software detection method and system. Belongs to the field of software detection and comprises the steps that a data set is obtained and divided into a training set and a test set, and the training set and the test set are each composed of a plurality of. Asm and. Bytes files; pairing the. Asm file and the. Bytes file in pairs to synthesize a binary file, and converting the binary file into a grey-scale map; inputting the grey-scale map of the training set into a pre-established malicious software detection model for training to obtain a trained malicious software detection model; performing five-fold crossing on the grey-scale map of the test set to inspect the trained malicious software detection model, and obtaining a final malicious software detection model after the verification is passed; and detecting the software by using the final malicious software detection model and outputting a result. The system comprises a data set acquisition module, a grey-scale map conversion module, a training module, an inspection module and an output module. According to the method and the system, the potential malicious software can be effectively identified.

Description

Technical Field

[0001] The present application relates to the field of software detection, and in particular, to a malware detection method and system. Background Art

[0002] With the rapid development of the Internet and the exponential growth of data, network security has become a major challenge facing the world. Enterprises and individuals generate a large amount of data every day, which may contain sensitive information, and the systems storing this sensitive information will become targets of hacker attacks. At the same time, network threats are becoming increasingly complex and changeable, and traditional security protection measures are difficult to cope with new types of network attacks. In this context, how to effectively identify potential malware and protect data security and the stability of the network environment has become an urgent problem to be solved. Summary of the invention

[0003] The purpose of the present application is to provide a malware detection method and system, which can effectively identify potential malware.

[0004] This application is implemented as follows: In a first aspect, the present application provides a malware detection method, comprising: Get the data set and divide it into a training set and a test set. The training set and the test set are composed of several .asm and .bytes files respectively. Pair the .asm file and the .bytes file into binary files, and convert the binary files into grayscale images; Inputting the grayscale image of the training set into the pre-selected malware detection model for training to obtain a trained malware detection model; Perform a five-fold cross-check on the grayscale image of the test set to test the trained malware detection model. After verification, the final malware detection model is obtained. Use the final malware detection model to detect the software and output the results.

[0005] Based on the first aspect, the specific steps of converting the binary file into a grayscale image include: Cut the binary file into small sequences to obtain the cut binary sequence, and the cut binary sequence is regarded as a grayscale image in the range of [0,255]; The cut binary sequence is decompressed and unpacked, and then divided into groups of 8 bits to obtain a binary sequence group; The binary sequence group is converted into corresponding unsigned decimal values, each decimal value represents a gray level, ranging from 0 to 255, with a total of 256 gray values, corresponding to different gray pigment levels.

[0006] Based on the first aspect, the above malware detection model adopts a convolutional neural network, which includes 3 convolutional layers, 3 pooling layers, 1 BiLSTM layer and 1 fully connected layer.

[0007] Based on the first aspect, the nonlinear mapping of convolutional neural network features is realized through the convolution layer. The convolution layer is responsible for feature extraction. Through continuous convolution operations, the input grayscale image is converted into an output feature vector. In the convolution process, when the convolution kernel completes a convolution operation with the current receptive field, it will slide to the next window position according to the set convolution step size to continue the convolution. The formula for this convolution operation is: in, represents the convolution output of the jth channel, is the kernel used for convolution, refers to the convolution operation, is the bias value, and It is calculated The desired subset of input feature maps, It is the output feature map of the previous layer.

[0008] Based on the first aspect, the pooling operation of the pooling layer divides the grayscale image into multiple non-overlapping areas and simplifies the nodes in these areas; the formula of the pooling operation is: in, Represents the net activation value of the jth channel of the pooling layer l, which is the output feature map of the previous layer Obtained by downsampling and adding bias , the pooling operation is performed by the function Execute, which is responsible for reducing the size of the feature map while retaining important feature information.

[0009] Based on the first aspect, the number of filters in the three convolutional layers in the malware detection model are 30, 50, and 90 respectively, and each convolutional layer is followed by a pooling layer to reduce the data dimension and computational complexity. After the pooling layer, a BiLSTM layer with 128 neurons is connected, and finally a fully connected layer is connected for output.

[0010] Based on the first aspect, after the grayscale image is processed by the convolution layer, pooling layer and LSTM layer, the obtained feature vector is sent to the fully connected layer, and the feature vector is processed by the softmax function to classify and output the final prediction result. The output calculation of the fully connected layer can be expressed by the following formula: , in, represents the activation function of the fully connected layer, is the net activation of the fully connected layer; is the output feature map of the previous layer, is the bias of the fully connected layer, is the weight of the layer, represents the output of the fully connected layer.

[0011] In a second aspect, the present application provides a malware detection system, comprising: Get Dataset Module: Get the dataset and divide it into training set and test set. The training set and test set are composed of several .asm and .bytes files respectively. Grayscale image conversion module: Combine .asm files and .bytes files into binary files, and convert binary files into grayscale images; Training module: inputting the grayscale image of the training set into the pre-selected malware detection model for training to obtain a trained malware detection model; Verification module: The grayscale image of the test set is cross-checked by five folds to verify the trained malware detection model. After verification, the final malware detection model is obtained; Output module: detects software using the final malware detection model and outputs the results.

[0012] In a third aspect, the present application provides an electronic device, comprising: a memory for storing one or more programs; a processor; when the one or more programs are executed by the processor, the method is implemented.

[0013] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above method when executed by a processor.

[0014] Compared with the prior art, the present invention has at least the following advantages or beneficial effects: Get the data set and divide it into a training set and a test set. The training set and the test set are composed of several .asm and .bytes files respectively. Pair the .asm file and the .bytes file into binary files, and convert the binary files into grayscale images; Inputting the grayscale image of the training set into the pre-selected malware detection model for training to obtain a trained malware detection model; Perform a five-fold cross-check on the grayscale image of the test set to test the trained malware detection model. After verification, the final malware detection model is obtained. Use the final malware detection model to detect the software and output the results.

[0015] In this application, the data set is divided into a training set and a test set, both of which are composed of multiple .asm and .bytes files. Each .asm file is associated with its corresponding .bytes file to represent a specific data point. This segmentation method maintains the consistency of the data set, that is, the training set and the test set both contain the same type of files, ensuring the consistency of model training and evaluation. And the data set consists of .asm and .bytes files, which are used to analyze malware. In addition, dividing the data set into a training set and a test set ensures the generalization ability of the model, that is, the ability of the model to process new data, rather than just remembering the training data. The grayscale image of this application contains features related to malware, such as specific byte patterns, instruction sequences, and code structures. During training, the malware detection model will learn to extract useful features from the grayscale image and learn how to associate these features with malware and benign software. After multiple iterations, the trained model can be used to classify new samples to determine whether they are malware or benign software. Furthermore, the present application also sets a test set. If the model shows good performance in the five-fold cross validation, that is, its average performance on all five subsets reaches a predetermined threshold or standard, then the model is considered to have passed the verification. After passing the verification, it is regarded as the final malware detection model. This final model can be used to detect malware on new, unseen data, thereby improving the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of an embodiment of a malware detection method of the present application; Figure 2 A schematic diagram of converting a binary file into a grayscale image to train a model in an embodiment of a malware detection method of the present application; Figure 3 This is a schematic diagram of the process of generating a grayscale image in an embodiment of a malware detection method of the present application; Figure 4 This is a schematic diagram of grayscale color levels in an embodiment of a malware detection method of the present application; Figure 5 A convolution operation flow chart of an embodiment of a malware detection method of the present application; Figure 6A schematic diagram of an LSTM layer in an embodiment of a malware detection method of the present application; Figure 7 A schematic diagram of the structure of a malware detection model in an embodiment of a malware detection method of the present application; Figure 8 The present application provides a structural block diagram of a malware detection system; Fig. 9 The present application provides a structural block diagram of an electronic device. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0019] In conjunction with the accompanying drawings, some implementation methods of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] Example In a first aspect, an embodiment of the present application provides a malware detection method, which can improve the accuracy of malware detection.

[0021] Please refer to Figure 1 , a malware detection method, comprising the following steps: S1: Get a data set and divide it into a training set and a test set. The training set and the test set are composed of several .asm and .bytes files respectively. Specifically, the dataset comes from Kaggle (2015 Microsoft Malware Classification Challenge). The dataset is divided into two parts: training set and test set. Each part consists of more than 20,000 .asm and .bytes files. These two files contain assembly code (.asm) and binary data (.bytes), which are used to analyze malware.

[0022] S2: Pair the .asm file and the .bytes file into binary files, and convert the binary files into grayscale images; Please refer to Figure 2Specifically, the assembly code described in the .asm file is combined with the binary data in the .bytes file to form a new binary file (.bin file). This includes compiling the assembly code described in the asm file into machine code (binary format) and then merging it with the data in the .bytes file. Convert the binary file to a grayscale image, which is an image constructed by dividing black and white into multiple levels in a logarithmic relationship. Grayscale values ​​are usually represented by a range of numbers from 0 to 255, where 0 represents black, 255 represents white, and the middle numbers represent different levels of grayscale values. By setting this up, the binary data can be represented in a visual form, that is, the data is converted into a grayscale image, in which the brightness (or grayscale level) of each pixel corresponds to one or more bits in the binary data. This conversion allows the malware detection model to process and understand the data. Through visualization, the malware detection model can learn to extract features from the image and then perform recognition tasks.

[0023] Please refer to Figure 3 , the specific steps to convert binary files to grayscale images include: S2-1: Cut the binary file into small sequences to obtain the cut binary sequence, and the cut binary sequence is regarded as a grayscale image in the range of [0,255]; Specifically, in the malicious code identification task, we cannot directly input the raw binary data of exe (executable file) or apk (Android application package) files into the neural network for calculation. The reason is that neural networks usually process structured, fixed-length data (such as images, text vectorized data, etc.), while exe or apk files are complex binary files containing various codes, resources, metadata, etc., and their structure and length are irregular. To solve this problem, we preprocess the binary files by cutting the files into small, fixed-length binary sequences (or "fragments", "blocks", etc.). These binary sequences are easy for neural networks to process and contain enough context information to identify the characteristics of malicious code. Each binary sequence can be regarded as a row or column of a grayscale image. Since each byte (8 bits) of a binary file can be represented as an integer between 0 and 255 (i.e., a grayscale level), the entire binary file can be converted into a grayscale image. This conversion allows us to use image processing technology and neural network models to process and analyze malicious code.

[0024] S2-2: decompressing and unpacking the cut binary sequence, and then dividing it into groups of 8 bits to obtain a binary sequence group; Specifically, malicious files exist in the form of binary code, which contains various characteristic information used to identify malicious behavior. Malicious files may be compressed or encrypted, so they need to be decompressed and unpacked first. The processed binary files are divided into groups of 8 bits (i.e., one byte), and each group is converted into an unsigned decimal number. Each byte of binary data can be represented as an integer between 0 and 255 (the maximum value that an 8-bit binary number can represent is 255).

[0025] S2-3: Convert the binary sequence group into corresponding unsigned decimal values, each decimal value represents a gray level, ranging from 0 to 255, with a total of 256 gray values, corresponding to different gray pigment levels.

[0026] Please refer to Figure 4 Specifically, each unsigned decimal number is mapped to a grayscale level, which ranges from 0 to 255, with a total of 256 possible values. These values ​​correspond to different gray pigment levels, which are used to represent different brightness in grayscale images. The value range of each byte (code segment) in the original binary file (0x00 to 0xFF, i.e., 0 to 255) is mapped to 256 possible grayscale values. This mapping provides image-based data representation for subsequent analysis and processing, so that image processing technology and algorithms can be used to analyze the characteristics of malicious code. Specifically, the B2M algorithm can be used to set a fixed row width to generate a two-dimensional array. This embodiment uses (10000,1), and its height varies according to the length of the binary data. The B2M algorithm (Binary to Matrix, i.e., binary to matrix conversion algorithm) is used to convert binary data into a two-dimensional array (matrix). The number of columns (or row width) of the two-dimensional array is fixed to 10000, while the number of rows (or height) varies according to the length of the binary data. Each binary file will be converted into a grayscale image with a width of 10,000 pixels and a height that varies with the length of the data. In this way, the compressed code segment data is effectively converted into a grayscale image. Through this process, the binary code of the malicious file is effectively converted into the form of a grayscale image.

[0027] S3: inputting the grayscale image of the training set into the pre-selected malware detection model for training to obtain a trained malware detection model; Specifically, the grayscale image dataset converted in the training set is input into the malware detection model for model training. Through training, the model can learn the features in the data and make predictions or classifications based on these features.

[0028] S4: Perform a five-fold cross-check on the grayscale image of the test set to verify the trained malware detection model. After verification, the final malware detection model is obtained; Specifically, the trained malware detection model is tested using the five-fold cross-validation method to evaluate the performance of the model. The five-fold cross-validation method is to divide the test set into five non-overlapping subsets, and then use four subsets as training data and the remaining subset as test data to train and test the model five times. Each training and test will get a performance indicator (such as accuracy, recall, etc.). Finally, the average of these five performance indicators is used as the final performance indicator of the model. This method can effectively reduce the risk of model overfitting and improve the reliability of the evaluation results. If the performance indicators of the model in the five-fold cross-validation (such as accuracy, recall, etc.) meet the expected requirements or standards, we can consider that the model has passed the verification.

[0029] S5: Use the final malware detection model to detect the software and output the results.

[0030] Specifically, it uses trained and verified malware detection models to perform malware detection on unknown binary files, helping users identify and respond to potential malware threats.

[0031] In some embodiments of the present invention, the malware detection model adopts a convolutional neural network, which includes 3 convolutional layers, 3 pooling layers, 1 BiLSTM layer and 1 fully connected layer.

[0032] Please refer to Figure 7 Specifically, in malware detection, 3 convolutional layers (CNN) can extract malicious features from binary data (or converted grayscale images). 3 pooling layers are used to reduce the dimensionality of the data (i.e., reduce the number and size of features) while retaining important feature information. This helps reduce the computational complexity of the model and the risk of overfitting. 1 BiLSTM layer is a bidirectional long short-term memory (BiLSTM) layer. BiLSTM is a special recurrent neural network (RNN) structure that can capture long-term dependencies in sequence data. In malware detection, the BiLSTM layer can analyze sequence features in binary data (or converted grayscale images), such as the structure and logic of the code. In malware detection, the fully connected layer can determine whether a given binary file is malware or normal software based on the extracted features.

[0033] Furthermore, the nonlinear mapping of convolutional neural network features is realized through the convolution layer. The convolution layer is responsible for feature extraction. Through continuous convolution operations, the input grayscale image is converted into an output feature vector. During the convolution process, when the convolution kernel completes a convolution operation with the current receptive field, it will slide to the next window position according to the set convolution step size to continue the convolution. The formula for this convolution operation is: in, represents the convolution output of the jth channel, is the kernel used for convolution, refers to the convolution operation, is the bias value, and It is calculated The desired subset of input feature maps, It is the output feature map of the previous layer.

[0034] Specifically, the convolution kernel (also called a filter) slides over the input data and performs a convolution operation with the receptive field of the current position (i.e., the input data area covered by the convolution kernel). After completing a convolution, the convolution kernel will move to the next position according to the set stride and continue to convolve until the entire input data is covered. For example, if the size of the input image is 3×3, and a 2×2 convolution kernel is used, and the stride is set to 1, then the specific convolution operation process is as follows: Figure 5 As shown, this ensures that the convolution kernel can gradually cover and process all areas of the input image, that is, by gradually sliding the convolution kernel and covering the entire input image, it can ensure that all areas are processed.

[0035] Furthermore, the pooling operation of the pooling layer divides the grayscale image into multiple non-overlapping areas and simplifies the nodes in these areas; the formula for the pooling operation is: in, Represents the net activation value of the jth channel of the pooling layer l, which is the output feature map of the previous layer Obtained by downsampling and adding bias , the pooling operation is performed by the function Execute, which is responsible for reducing the size of the feature map while retaining important feature information.

[0036] Specifically, the pooling operation divides the feature map into multiple non-overlapping regions and simplifies the nodes in these regions. This simplification usually includes operations such as taking the maximum value (max pooling) or the average value (average pooling), aiming to reduce the size of the feature map (i.e., dimensionality reduction) and the amount of computation while retaining important feature information.

[0037] Furthermore, in this embodiment, the number of filters of the three convolutional layers in the malware detection model are 30, 50, and 90 respectively, and each convolutional layer is followed by a pooling layer for reducing the dimension of the data and the amount of computation, and a BiLSTM layer with 128 neurons is connected after the pooling layer, and finally a fully connected layer for output is connected.

[0038] For details, please refer to BiLSTM layer Figure 6 This structure combines the advantages of convolutional neural networks (CNN) and recurrent neural networks (RNN, especially LSTM), which can extract and integrate feature information in the input data and output the final detection results.

[0039] Furthermore, after the grayscale image is processed by the convolution layer, the pooling layer, and the LSTM layer, the obtained feature vector is sent to the fully connected layer, and the feature vector is processed by the softmax function to classify and output the final prediction result. The output calculation of the fully connected layer can be expressed by the following formula: , in, represents the activation function of the fully connected layer, is the net activation of the fully connected layer; is the output feature map of the previous layer, is the bias of the fully connected layer, is the weight of the layer, represents the output of the fully connected layer.

[0040] Specifically, after the fully connected layer, the feature vector is processed using the softmax function. The softmax function is a commonly used multi-classification function that maps each component of the feature vector to the interval (0,1), and the sum of all components is 1, thereby representing the probability of each category. By comparing these probabilities, the final predicted category can be determined.

[0041] Please refer to Figure 8In a second aspect, an embodiment of the present application provides a malware detection system, including: a data set acquisition module 1: acquiring a data set, and dividing the data set into a training set and a test set, wherein the training set and the test set are respectively composed of a plurality of .asm and .bytes files; a grayscale image conversion module 2: pairing .asm files and .bytes files in pairs to synthesize binary files, and converting the binary files into grayscale images; a training module 3: inputting the grayscale image of the training set into a pre-selected malware detection model for training to obtain a trained malware detection model; a verification module 4: performing a five-fold crossover on the grayscale image of the test set to verify the trained malware detection model, and obtaining a final malware detection model after verification; an output module 5: using the final malware detection model to detect software and output the results.

[0042] Specifically, the specific implementation of the malware detection system can refer to the implementation of the malware detection method, which will not be described in detail here.

[0043] Please refer to Fig. 9 In a third aspect, an embodiment of the present application provides an electronic device, including: a memory 7 for storing one or more programs; a processor 6; when the one or more programs are executed by the processor 6, the above malware detection method is implemented. The processor 6 communicates with the memory 7 via a communication interface 8.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned malware detection method when executed by a processor.

[0045] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A malware detection method, characterized in that: include: Get the data set and divide it into a training set and a test set. The training set and the test set are composed of several .asm and .bytes files respectively. Pair the .asm file and the .bytes file into binary files, and convert the binary files into grayscale images; Inputting the grayscale image of the training set into the pre-selected malware detection model for training to obtain a trained malware detection model; Perform a five-fold cross-check on the grayscale image of the test set to test the trained malware detection model. After verification, the final malware detection model is obtained. Use the final malware detection model to detect the software and output the results.

2. A malware detection method according to claim 1, characterized in that: The specific steps of converting the binary file into a grayscale image include: Cut the binary file into small sequences to obtain the cut binary sequence, and the cut binary sequence is regarded as a grayscale image in the range of [0,255]; The cut binary sequence is decompressed and unpacked, and then divided into groups of 8 bits to obtain a binary sequence group; The binary sequence group is converted into corresponding unsigned decimal values, each decimal value represents a gray level, ranging from 0 to 255, with a total of 256 gray values, corresponding to different gray pigment levels.

3. A malware detection method according to claim 2, characterized in that: The malware detection model adopts a convolutional neural network, which includes 3 convolutional layers, 3 pooling layers, 1 BiLSTM layer and 1 fully connected layer.

4. A malware detection method according to claim 3, characterized in that: The nonlinear mapping of the convolutional neural network features is realized through the convolution layer. The convolution layer is responsible for feature extraction. Through continuous convolution operations, the input grayscale image is converted into an output feature vector. During the convolution process, when the convolution kernel completes a convolution operation with the current receptive field, it will slide to the next window position according to the set convolution step size to continue the convolution. The formula for this convolution operation is: in, represents the convolution output of the jth channel, is the kernel used for convolution, refers to the convolution operation, is the bias value, and It is calculated The desired subset of input feature maps, It is the output feature map of the previous layer.

5. A malware detection method according to claim 4, characterized in that: The pooling operation of the pooling layer divides the grayscale image into multiple non-overlapping areas and simplifies the nodes in these areas; the formula for the pooling operation is: in, Represents the net activation value of the jth channel of the pooling layer l, which is the output feature map of the previous layer Obtained by downsampling and adding bias , the pooling operation is performed by the function Execute, which is responsible for reducing the size of the feature map while retaining important feature information.

6. A malware detection method according to claim 5, characterized in that: The number of filters in the three convolutional layers in the malware detection model are 30, 50, and 90 respectively. Each convolutional layer is followed by a pooling layer to reduce the data dimension and computational complexity. After the pooling layer, a BiLSTM layer with 128 neurons is connected, and finally a fully connected layer is connected for output.

7. A malware detection method according to claim 6, characterized in that: After the grayscale image is processed by the convolution layer, pooling layer and LSTM layer, the obtained feature vector is sent to the fully connected layer, and the feature vector is processed by the softmax function to classify and output the final prediction result. The output calculation of the fully connected layer can be expressed by the following formula: , in, represents the activation function of the fully connected layer, is the net activation of the fully connected layer; is the output feature map of the previous layer, is the bias of the fully connected layer, is the weight of the layer, represents the output of the fully connected layer.

8. A malware detection system, characterized in that: include: Get Dataset Module: Get the dataset and divide it into training set and test set. The training set and test set are composed of several .asm and .bytes files respectively. Grayscale image conversion module: Combine .asm files and .bytes files into binary files, and convert binary files into grayscale images; Training module: inputting the grayscale image of the training set into the pre-selected malware detection model for training to obtain a trained malware detection model; Verification module: The grayscale image of the test set is cross-checked by five folds to verify the trained malware detection model. After verification, the final malware detection model is obtained; Output module: detects software using the final malware detection model and outputs the results.

9. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images