File detection method, device and equipment and readable storage medium

Abstract

The invention discloses a file detection method, device and equipment and a readable storage medium, and the method comprises the steps: monitoring a file system, and obtaining a first access behavior feature of an accessed file in the file system; determining suspicious files from the accessed files based on the first file access features by using a first classifier; obtaining a second access behavior feature and a file content feature of the suspicious file; and determining a submerging file controlled by the ransomware from the suspicious files by using a second classifier based on the second access behavior feature and the file content feature. According to the method, the ransomware detection is carried out based on the access behavior, the ransomware detection can be carried out on unknown ransomware based on file content features, the ransomware detection precision can be improved, rapid screening and deep screening are combined, and the ransomware detection can be rapidly and accurately completed.

Description

Technical Field

[0001] The present invention relates to the field of security technologies, and in particular, to a file detection method, device, equipment, and readable storage medium. Background Art

[0002] Ransomware usually encrypts various files such as documents, emails, databases, source codes, pictures, and compressed files on the user system in a certain form to make them unavailable, or reduces the usability of the system by modifying the system configuration file or interfering with the normal use of the system by the user, and then sends a ransom notice to the user by means of a pop-up window, dialog box, or generating a text file, etc., asking the user to remit money to a specified account to obtain the password for decrypting the files or to restore the system to normal operation.

[0003] The storage system, like the user system, is also threatened by ransomware attacks. And because the number of files stored in the storage system is much larger than that of the user system, the loss is greater after being invaded by ransomware. However, there are few anti-ransomware technologies for the file system based on the storage system at present. That is, the current file system is in a completely open form for ransomware. To achieve effective anti-ransomware, an effective detection scheme is needed to determine whether the file system is being or has been invaded by ransomware.

[0004] In summary, how to effectively solve problems such as file ransom detection is a technical problem that those skilled in the art need to solve urgently at present. Summary of the Invention

[0005] The purpose of the present invention is to provide a file detection method, device, equipment, and readable storage medium, which can efficiently and accurately identify whether the files in the file system are invaded by ransomware.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] A file detection method includes:

[0008] Monitoring the file system to obtain the first access behavior characteristics of the accessed files in the file system; the first access behavior characteristics are used to distinguish normal I / O from suspicious I / O;

[0009] Using a first classifier to determine suspicious files from the accessed files based on the first file access characteristics;

[0010] Obtaining the second access behavior characteristics and file content characteristics of the suspicious files; the second access behavior characteristics are used to identify ransom I / O;

[0011] Using a second classifier, determine the compromised files controlled by the ransomware from the suspicious files based on the second access behavior characteristics and the file content characteristics.

[0012] Preferably, monitoring the file system to obtain the first access behavior characteristics of the accessed files in the file system includes:

[0013] Performing real-time monitoring during the interaction between the software and the file system to obtain file I / O log information;

[0014] Based on a sliding window, extract from the file I / O log information whether the accessed file has been renamed or deleted, the same proportion of file name suffixes, whether the file type has changed, and the file type;

[0015] Determine whether the accessed file has been renamed or deleted, the same proportion of file name suffixes, whether the file type has changed, and the file type as the first access behavior characteristics.

[0016] Preferably, obtaining the second access behavior characteristics and file content characteristics of the suspicious files includes:

[0017] Based on a sliding window, extract from the file I / O log information the number of times the suspicious file is read, written, deleted, renamed, the number of file type changes, and the entropy of the write buffer;

[0018] Determine the number of times the suspicious file is read, written, deleted, and renamed as the second access behavior characteristics;

[0019] Determine the number of file type changes and the entropy of the write buffer as the file content characteristics.

[0020] Preferably, performing real-time monitoring during the interaction between the software and the file system to obtain file I / O log information includes:

[0021] During the interaction between the software and the file system, use a hook to obtain the file I / O log information;

[0022] Or, during the interaction between the software and the file system, communicate with the file system or the monitoring software monitoring the file system to obtain file I / O log information.

[0023] Preferably, the process of training the first classifier includes:

[0024] Obtain a first training sample; the first training sample has label information on whether it belongs to a suspicious file;

[0025] Obtain the first file access feature of the first training sample;

[0026] Based on the decision tree algorithm, support vector machine algorithm, multi-layer perceptron algorithm, adaptive boosting algorithm or nearest neighbor algorithm, input the first file access feature into the first preset classifier for training;

[0027] After the training is completed, determine the first preset classifier as the first classifier;

[0028] And / or, the process of training the second classifier includes:

[0029] Obtain a second training sample; the second training sample has label information on whether it belongs to a compromised file;

[0030] Obtain the second file access feature and file content feature of the second training sample;

[0031] Based on the random forest algorithm, long short-term memory network, gated recurrent unit structure, convolutional neural network or deep residual network, input the second file access feature and file content feature of the second training sample into the second preset classifier for training;

[0032] After the training is completed, determine the second preset classifier as the second classifier.

[0033] Preferably, after determining the compromised file, it further includes:

[0034] Obtain the I / O log information of the compromised file;

[0035] Use the I / O log information to determine the suspicious software that has accessed the compromised file;

[0036] Prohibit the suspicious software from accessing the file system.

[0037] Preferably, after obtaining at least one type of detection feature among the first file access feature, the second file access feature and the file content feature, it further includes:

[0038] Calculate the average value and standard deviation of the feature values of each dimension in the detection feature;

[0039] Subtract the average value of the dimension to which the feature value belongs to obtain a difference;

[0040] Determine the ratio of the difference to the standard deviation of the dimension to which it belongs as the standardized value corresponding to the feature value;

[0041] Replace all the feature values in the detection feature with the standardized values.

[0042] A file detection device, comprising:

[0043] File system activity monitoring is used to monitor the file system and obtain the first access behavior characteristics of the accessed files in the file system; the first access behavior characteristics are used to distinguish normal I / O from suspicious I / O;

[0044] A quick screening module is used to use a first classifier to determine suspicious files from the accessed files based on the first file access characteristics;

[0045] A deep screening module is used to obtain the second access behavior characteristics and file content characteristics of the suspicious files; the second access behavior characteristics are used to identify ransomware I / O; use a second classifier to determine the compromised files controlled by the ransomware from the suspicious files based on the second access behavior characteristics and the file content characteristics.

[0046] An electronic device includes:

[0047] A memory for storing a computer program;

[0048] A processor for implementing the steps of the above file detection method when executing the computer program.

[0049] A readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above file detection method are implemented.

[0050] Applying the method provided by the embodiments of the present invention, monitor the file system to obtain the first access behavior characteristics of the accessed files in the file system; the first access behavior characteristics are used to distinguish normal I / O from suspicious I / O; use a first classifier to determine suspicious files from the accessed files based on the first file access characteristics; obtain the second access behavior characteristics and file content characteristics of the suspicious files; the second access behavior characteristics are used to identify ransomware I / O; use a second classifier to determine the compromised files controlled by the ransomware from the suspicious files based on the second access behavior characteristics and the file content characteristics.

[0051] Since the detection is based on file access characteristics and is faster than detecting the file content itself, in the present invention, in order to effectively determine whether there is an attack by ransomware in the file system, it is first necessary to monitor the file system and obtain the first access behavior characteristics of the accessed files in the file system. Then, the first file access characteristics are input into the first classifier to quickly screen the accessed files, so as to quickly screen out suspicious files from the accessed files. Since the characteristics based on the file content itself can more accurately detect whether there is a ransomware attack, in the present invention, after screening out the suspicious files, the second access behavior characteristics can be combined with the file content characteristics, so as to use the second classifier to determine the compromised files controlled by the ransomware from the suspicious files, thus completing the ransomware detection. That is to say, the present invention performs ransomware detection based on access behavior, can face unknown ransomware, performs ransomware detection based on file content characteristics, can improve the accuracy of ransomware detection, and combines quick screening and in-depth screening, so as to quickly and accurately complete ransomware detection.

[0052] In addition, during the process of quick screening and in-depth screening, classifiers are used for screening. Since the classifiers themselves can be trained and optimized, therefore, the file detection of the present invention can also be more effective along with the training and optimization of the classifiers.

[0053] Correspondingly, the embodiments of the present invention also provide a file detection device, a device and a readable storage medium corresponding to the above file detection method, which have the above technical effects and will not be elaborated herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0055] Figure 1 is the implementation flowchart of a file detection method in an embodiment of the present invention;

[0056] Figure 2 is a schematic diagram of a file detection system based on machine learning in an embodiment of the present invention;

[0057] Figure 3 is a schematic diagram of file system activity monitoring in an embodiment of the present invention;

[0058] Figure 4 is a schematic diagram of a quick screening module in an embodiment of the present invention;

[0059] Figure 5Schematic diagram of a depth screening module in an embodiment of the present invention;

[0060] Figure 6 Schematic diagram of the structure of a file detection device in an embodiment of the present invention;

[0061] Figure 7 Schematic diagram of the structure of an electronic device in an embodiment of the present invention;

[0062] Figure 8 Specific structural schematic diagram of an electronic device in an embodiment of the present invention. Detailed implementation manners

[0063] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0064] Please refer to Figure 1 , Figure 1 which is a flowchart of a file detection method in an embodiment of the present invention. This method can also be implemented on other devices that have a communication connection with the storage device at the storage end. Implementing anti-ransomware technology at the user host end or the storage end of the storage system is a challenge. Although the host end can obtain more information, anti-ransomware detection needs to be embedded in each user's terminal, which affects the performance of the user system, has a high cost, and the detection software is easily attacked by ransomware to elevate privileges and uninstall. On the contrary, at the storage end, although less available information can be obtained, it does not affect user use, has a low cost, and the detection software is separated from the ransomware and will not be attacked by the ransomware. Therefore, the file detection method provided by the present invention can preferably be applied to the storage device at the storage end. Taking the application of the storage device as an example, this method will be described in detail below.

[0065] Specifically, this method includes the following steps:

[0066] S101. Monitor the file system to obtain the first access behavior characteristics of the files accessed in the file system.

[0067] Among them, the first access behavior characteristics are used to distinguish normal I / O from suspicious I / O.

[0068] The file system in the present invention can be any file system that can be implemented in a storage system. For example, the file system can be Ceph (a distributed file system), NTFS (New Technology File System, a Microsoft standard file system), an EXT (a JavaScript library) file system, etc. Hereinafter, the case where the file system is specifically the Ceph file system will be used as an example for explanation.

[0069] In the present invention, the storage device where the file system is located can monitor the access behavior of external software to itself, so as to obtain the first access behavior characteristics of the accessed files in the file system.

[0070] Alternatively, other devices can monitor the file system to obtain the first access behavior characteristics.

[0071] In the present invention, the first access behavior characteristics can specifically be used to screen out a large number of normal I / O (Input / Output) accesses. That is to say, the first access behavior characteristics can be used to determine which are normally accessed, so as to find out suspicious files.

[0072] The I / O mode of ransomware encrypting files follows the following 4 situations:

[0073] (1) The ransomware directly writes the encrypted data into the original file, and then renames the original file.

[0074] (2) The ransomware first renames the original file, and then writes the encrypted data into the renamed file.

[0075] (3) The ransomware creates a new file, writes the encrypted data of the original data into the new file, and then deletes the original file.

[0076] (4) The ransomware creates a new file, writes the encrypted data of the original data into the new file, then renames the new file, and deletes the original file.

[0077] For these 4 situations, the I / O operations all have renaming or deletion operations. Therefore, in practical applications, the first access behavior characteristics can include whether renaming and deletion operations occur.

[0078] When the ransomware renames most files, it will modify the file suffix, and modify it to the same suffix name. Or, the ransomware creates new files with the same prefix as the original file name according to the original file, and the suffix names of the numerous newly created files are all the same, and then deletes the original file. This situation rarely occurs during the use of benign software. Therefore, in practical applications, the first access behavior characteristics can include the same ratio of file name suffixes.

[0079] Since ransomware usually causes file type changes by encrypting file contents, while file type changes rarely occur during the use of benign software, in practical applications, the first access behavior feature can include whether a file type change occurs.

[0080] Ransomware usually reads a large number of different types of files, writes a single type of file or writes multiple types of files, while benign software usually only reads a small number of types of files and writes a single type. Therefore, in practical applications, the first access behavior feature can include the file type or the file type range.

[0081] In a specific implementation manner of the present invention, the file system is monitored to obtain the first access behavior feature of the accessed file in the file system, including:

[0082] Step 1: Real-time monitoring is performed during the interaction between the software and the file system to obtain file I / O log information;

[0083] Step 2: Based on a sliding window, whether the accessed file has been renamed or deleted, the same proportion of the file name suffix, whether a file type change has occurred, and the file type are extracted from the file I / O log information;

[0084] Step 3: Whether the file has been renamed or deleted, the same proportion of the file name suffix, whether a file type change has occurred, and the file type are determined as the first access behavior feature.

[0085] Among them, in Step 1, real-time monitoring is performed during the interaction between the software and the file system to obtain file I / O log information, including: during the interaction between the software and the file system, file I / O log information is obtained by using a hook;

[0086] Or, during the interaction between the software and the file system, communicate with the file system or the monitoring software that monitors the file system to obtain file I / O log information.

[0087] That is to say, during the process of software and file exchange, the software can access files in the file system, thus generating relevant I / O information. In practical applications, file I / O log information can be directly obtained by using a hook, or the file system itself can feedback the file I / O log information, or the file I / O log information can be recorded by the monitoring software and then communicate with the monitoring software to obtain the file I / O log information.

[0088] The following takes obtaining file I / O log information by using a hook as an example for detailed description.

[0089] When the host accesses a directory or file in the mounted file system, the network file sharing service calls the file I / O operation interface of the storage system, allowing the application to perform various operations such as creating, reading, writing, deleting files, etc., and then communicates with the storage system cluster through the file I / O operation, thus realizing the sharing of the file system.

[0090] During the file sharing (i.e., the file mounting process is called sharing and the operation directory is visible remotely) / access operation, real-time monitoring is carried out by hooking the file operation interface, collecting information such as reading, writing, opening, deleting, renaming files, etc. and writing them into the global log, and thus file I / O log information can be obtained.

[0091] After obtaining the file I / O log information, it is possible to extract from the file I / O log information whether the accessed file has been renamed or deleted, the same proportion of file name suffixes, whether the file type has been changed and the file type. Considering that in actual applications, whether the file access is normal access or abnormal access, the time factor also needs to be considered. Therefore, in the present invention, when extracting features, based on a sliding window, it is possible to extract whether the accessed file has been renamed or deleted, the same proportion of file name suffixes, whether the file type has been changed and the file type within a period of time, and then determine these information / feature values as the first file access features.

[0092] S102. Use the first classifier to determine suspicious files from the accessed files based on the first file access features.

[0093] In the present invention, among the accessed files, there are normal files that are normally accessed and suspicious files that are abnormally accessed. Since there are a large number of normal files, in order to avoid time-consuming content detection for all files, fast screening is proposed in the present invention. Specifically, after extracting the first file access features, the first file access features can be input into the first classifier, and thus based on the first file access features, suspicious files can be determined from the accessed files. In this way, a large number of normal files can be filtered out, and a small number of suspicious files for further in-depth screening can be found.

[0094] Among them, the first classifier can specifically be a normal file that can effectively filter normal I / O access behaviors trained based on a large number of training samples.

[0095] In a specific embodiment of the present invention, the process of training the first classifier includes:

[0096] Obtain the first training samples; the first training samples have label information on whether they belong to suspicious files;

[0097] Obtain the first file access features of the first training samples;

[0098] Based on the decision tree algorithm, support vector machine algorithm, multi-layer perceptron algorithm, adaptive boosting algorithm or nearest neighbor algorithm, input the first file access feature into the first preset classifier for training;

[0099] After the training is completed, determine the first preset classifier as the first classifier.

[0100] That is, the first file access feature can be used to train the first preset classifier with algorithms such as the decision tree algorithm, support vector machine algorithm, multi-layer perceptron algorithm, adaptive boosting algorithm or nearest neighbor algorithm, so as to determine whether the I / O data is suspicious based on the trained first classifier.

[0101] Among them, the decision tree algorithm is a supervised learning algorithm that can perform classification and regression. The decision tree selects the optimal feature by using the Gini index and generates a binary tree to distinguish between benign I / O (corresponding to normal files) and suspicious I / O (corresponding to suspicious files).

[0102] S103. Obtain the second access behavior feature and file content feature of the suspicious file.

[0103] Among them, the second access behavior feature is used to identify ransomware I / O.

[0104] After a quick screening based on the first access behavior feature, suspicious files are identified. Although they are suspicious files, not all of them are compromised files infected by ransomware. Since compromised files often have certain file content features and there are also certain distinguishing features in the I / O access of compromised files, in the present invention, the second access behavior feature and the file content feature can be combined to further judge the suspicious files.

[0105] That is, after the suspicious files are identified, the second access behavior feature and the file content feature of the suspicious files can be obtained.

[0106] Among them, the second access behavior feature is used to judge whether the suspicious file conforms to the access behavior feature infected by ransomware, and the file content feature is used to judge whether it conforms to the file content feature infected by ransomware.

[0107] Ransomware, while maximizing the destruction of files, also tries to reduce the risk of being detected. It usually performs read-encrypt-write, rename or delete operations quickly within a short period of time. Taking advantage of this feature, the following features are extracted: the number of times of reading files, the number of times of writing files, the number of times of deleting files, the number of times of renaming files.

[0108] Entropy (Shannon entropy) is used to represent the uncertainty or randomness of data; its calculation formula: Among them, is the probability of the byte value i appearing in all bytes, e represents the entropy value, and 0 - 255 represents 256 possible characters (an 8-bit binary byte can represent 256 different characters, numbers, and symbols from 0 to 255). represents the summation over all 256 characters to calculate the information content of each character. B i represents the i-th character, which is a specific character among the 256 characters. Then it represents the character B i The probability of occurrence.

[0109] represents taking the logarithm of to represent the information content of each character, that is, how much information is provided by the occurrence of a character. When the probability of a certain character appearing is low, the information it provides is high because it is relatively rare.

[0110] The range of entropy is from 0 to 8. When ransomware encrypts data, due to the randomness of the content, the entropy value will be relatively high, close to 8, while the entropy value of benign software is generally relatively low. Therefore, the entropy value feature of the write buffer can be added in the in-depth screening.

[0111] Ransomware reads a large number of different file types in a short time and writes one type. Therefore, the feature of the number of file type changes is added to the in-depth screening module.

[0112] In a specific implementation manner of the present invention, obtaining the second access behavior feature and file content feature of the suspicious file includes:

[0113] Step 1: Based on a sliding window, extract the number of read files, number of written files, number of deleted files, number of renamed files, number of file type changes, and entropy of the write buffer of the suspicious file from the file I / O log information;

[0114] Step 2: Determine the number of read files, number of written files, number of deleted files, and number of renamed files as the second access behavior feature;

[0115] Step 3: Determine the number of file type changes and entropy of the write buffer as the file content feature.

[0116] In practical applications, after determining the suspicious file, the number of read files, number of written files, number of deleted files, number of renamed files, number of file type changes, and entropy of the write buffer of the suspicious file can be extracted from the file I / O log information. Among them, the number of read files, number of written files, number of deleted files, and number of renamed files are the second access behavior features, and the number of file type changes and entropy of the write buffer are the file content features.

[0117] Since the judgment of compromised files is also time-related, when extracting the second access behavior features and file content features, feature values within a period of time can be extracted based on a sliding window. The time length of this sliding window can be set and adjusted according to actual needs or experience, and will not be elaborated here one by one.

[0118] S104. Use the second classifier to determine the compromised files controlled by ransomware from the suspicious files based on the second access behavior features and file content features.

[0119] In the present invention, after the second access behavior features and file content features are extracted, the second access behavior features and file content features can be input into the trained second classifier for in-depth screening, so as to determine the compromised files controlled by ransomware from the suspicious files.

[0120] In practical applications, the first access behavior features and the second access behavior features can overlap or be recombined with each other.

[0121] Among them, the process of training the second classifier includes:

[0122] Obtain the second training samples; the second training samples have label information on whether they belong to compromised files;

[0123] Obtain the second file access features and file content features of the second training samples;

[0124] Based on the random forest algorithm, long short-term memory network, gated recurrent unit structure, convolutional neural network or deep residual network, input the second file access features and file content features of the second training samples into the second preset classifier for training;

[0125] After the training is completed, determine the second preset classifier as the second classifier.

[0126] That is, the second access behavior features and file content features can be used to train a classifier using the random forest algorithm, long short-term memory network, gated recurrent unit structure, convolutional neural network or deep residual network to determine whether a ransomware attack has occurred.

[0127] Among them, the random forest algorithm is a supervised learning algorithm, an ensemble algorithm composed of a group of decision trees, which can perform classification and regression. When performing binary classification on ransomware and benign software, each decision tree in the random forest judges the input samples respectively to obtain their respective classification results, and the random forest takes the class with the most decision tree classification results as the final result.

[0128] The compromised file is the file controlled by the ransomware. After identifying the compromised file, it can be determined that the file system has been attacked by the ransomware. At this time, a ransomware attack alert can be issued, and relevant defense and data recovery processes can be initiated.

[0129] For example, an alert can be issued to remind users to take corresponding protective measures to prevent the further spread of the ransomware.

[0130] Apply the method provided by the embodiment of the present invention to monitor the file system and obtain the first access behavior characteristics of the accessed files in the file system; the first access behavior characteristics are used to distinguish normal I / O from suspicious I / O; use the first classifier to determine suspicious files from the accessed files based on the first file access characteristics; obtain the second access behavior characteristics and file content characteristics of the suspicious files; the second access behavior characteristics are used to identify ransom I / O; use the second classifier to determine the compromised files controlled by the ransomware from the suspicious files based on the second access behavior characteristics and file content characteristics.

[0131] Since the detection is based on file access characteristics, which is faster than detecting the file content itself, in the present invention, in order to effectively determine whether there is a ransomware attack in the file system, it is first necessary to monitor the file system and obtain the first access behavior characteristics of the accessed files in the file system. Then, input the first file access characteristics into the first classifier to quickly screen the accessed files, so as to quickly screen out suspicious files from the accessed files. Since the characteristics of the file content itself can more accurately detect whether there is a ransom attack, in the present invention, after screening out the suspicious files, the second access behavior characteristics can be combined with the file content characteristics, so as to use the second classifier to determine the compromised files controlled by the ransomware from the suspicious files, thus completing the ransom detection. That is to say, the present invention performs ransom detection based on access behavior, which can face unknown ransomware. Ransom detection based on file content characteristics can improve the accuracy of ransom detection. The combination of quick screening and in-depth screening can quickly and accurately complete ransom detection.

[0132] In addition, during the quick screening and in-depth screening processes, classifiers are used for screening. Since the classifier itself can be trained and optimized, the file detection of the present invention can also be more effective with the training and optimization of the classifier.

[0133] It should be noted that based on the above embodiments, the embodiments of the present invention also provide corresponding improvement solutions. In the preferred / improved embodiments, the same steps or corresponding steps as those in the above embodiments can be referred to each other, and the corresponding beneficial effects can also be referred to each other. They will not be repeated one by one in the preferred / improved embodiments of this article.

[0134] In a specific embodiment of the present invention, considering that when a compromised file is detected, ransomware may be attacking the file system. To prevent more files from being attacked / controlled, after determining the compromised file, the following steps are further included:

[0135] Obtain the I / O log information of the compromised file;

[0136] Use the I / O log information to identify the suspicious software that has accessed the compromised file;

[0137] Prohibit the suspicious software from accessing the file system.

[0138] That is to say, after determining the compromised file, the I / O log information of the compromised file can be queried again, and based on this I / O log information, the suspicious software that has attacked the compromised file can be found. To prevent further compromise of other files, the suspicious software can be prohibited from accessing the file system.

[0139] In a specific embodiment of the present invention, after obtaining at least one type of detection feature among the first file access feature, the second file access feature, and the file content feature, the following steps are further included:

[0140] Calculate the mean and standard deviation of the feature values of each dimension in the detection feature;

[0141] Subtract the mean of the dimension to which the feature value belongs from the feature value to obtain a difference;

[0142] Determine the ratio of the difference to the standard deviation of the dimension to which it belongs as the standardized value corresponding to the feature value;

[0143] Replace all the feature values in the detection feature with the standardized values.

[0144] That is to say, the numerical differences between different features are relatively large. To eliminate this influence, preprocessing of data normalization is performed. Specifically, after at least one type of detection feature among the first file access feature, the second file access feature, and the file content feature, a normalization operation can be adopted.

[0145] Specifically, Z-score normalization can be used, that is, the data is distributed into a normal distribution with a mean of 0 and a standard deviation of 1. The steps include:

[0146] A. Calculate the mean (mean) and standard deviation (std) of the data set.

[0147] B. For each data point x in the data set, apply the following formula to standardize it:

[0148] Standardized value (z_score_x) = (x - mean) / std.

[0149] For example, when normalizing the dataset data = np.array([10, 20, 30, 40, 50]), the processing process includes:

[0150] Find the mean of these data: mean = np.mean(data);

[0151] Find the standard deviation of the dataset: std = np.std(data);

[0152] Calculate the normalized values: z_scores = (data - mean) / std;

[0153] The normalized database set is [-1.41421356, 70710678, 0, 0.70710678, 1.41421356].

[0154] In this way, the differences between data can be eliminated, facilitating subsequent classification and recognition.

[0155] To facilitate those skilled in the art to better understand and implement the file detection method provided by the embodiments of the present invention, the following takes a specific application scenario as an example to elaborate on this method in detail.

[0156] As can be seen from the above, the file detection method provided by the embodiments of the present invention can detect unknown ransomware, and can deal with ransomware using code obfuscation and code injection techniques; it can improve the detection accuracy and speed; it can reduce the resource overhead and performance impact on the system; it can perform real-time detection; it can solve the anti-ransomware technology problems in the file systems of few storage systems.

[0157] For the specific implementation process, please refer to Figure 2 , Figure 2 is a schematic diagram of a file detection system based on machine learning in the embodiments of the present invention. The system includes three modules: file system activity monitoring, quick screening, and in-depth screening.

[0158] There are differences in the file access behaviors of ransomware and benign software during their interaction with the file system, so the differences in this I / O information are utilized to detect ransomware. File system activity monitoring is to collect the I / O characteristics of file behaviors and perform real-time monitoring during the interaction between software and the file system. The hook method can be used to obtain file I / O log information such as read, write, open, delete, and rename.

[0159] The quick screening module aims to quickly filter a large number of benign sample I / Os through anomaly detection, reduce resource overhead, and achieve a low false negative rate. Therefore, it does not use features that require calculation of file content, but instead uses four features: whether rename and delete operations occur, the same proportion of file name suffixes, whether file type changes occur, and file type. The decision tree algorithm is used for preliminary judgment.

[0160] Deep screening. To more accurately detect ransomware attacks, further identification is performed on the suspicious behavior samples detected by the quick screening module. Therefore, file I / O access behavior features and file content features are combined. Specifically, six features are used: the number of times of reading files, the number of times of writing files, the number of times of deleting files, the number of times of renaming files, the number of times of file type changes, and the entropy of the write buffer. The random forest algorithm is used to complete the final detection.

[0161] Furthermore, please refer to Figure 3 , Figure 3 which is the schematic diagram of file system activity monitoring in the embodiment of the present invention. File system activity monitoring is realized when the host accesses a directory or file in the mounted file system. The network file sharing service will call the file I / O operation interface of the storage system, allowing the application program to perform various operations such as creating, reading, writing, deleting files, etc., and then communicate with the storage system cluster through file I / O operations, thus realizing file system sharing. During the file sharing operation process, real-time monitoring is carried out by hooking the file operation interface, and five access behavior information of reading, writing, opening, deleting, and renaming files are collected and written into the global log.

[0162] Please refer to Figure 4 , Figure 4 which is the schematic diagram of a quick screening module in the embodiment of the present invention. Quick screening includes feature extraction: whether rename and delete operations occur, detection of the same proportion of file name suffixes, and file type. Among them, the file type filters out the abnormal suffix names of the current user. Normal suffix names are, for example, file suffix name tables such as pptx, Png, log, ini, pub, docx, file, jpg, lnk, 7z, etc.

[0163] Data preprocessing: Specifically, by using a sliding window, features 1-4 within a window are processed. The numerical differences between different features are relatively large. To eliminate this influence, data normalization preprocessing is performed on the data.

[0164] Training the classifier: Use the above 4 features to train the decision tree algorithm to generate a classifier to determine whether the I / O data is suspicious. The decision tree algorithm is a supervised learning algorithm that can perform classification and regression. The decision tree selects the optimal feature by using the Gini index and generates a binary tree to distinguish between benign I / O and suspicious I / O.

[0165] Please refer to Figure 5 , Figure 5 which is a schematic diagram of a deep screening module in an embodiment of the present invention. Deep screening includes: Feature extraction: While maximizing the damage to files, ransomware also needs to reduce the risk of being detected. It usually quickly performs read-encrypt-write, rename, or delete operations within a short period of time. Utilizing this feature, the following features are extracted: the number of file reads, the number of file writes, the number of file deletions, and the number of file renames.

[0166] In addition, the file content features include the entropy value feature of the write buffer and the number of file type changes.

[0167] Data preprocessing: Specifically, by using a sliding window, the number of file reads, the number of file writes, the number of file deletions, the number of file renames, the average entropy of the write buffer, and the number of file type changes within a sliding window time are counted as features. Since the numerical differences between different features are relatively large, in order to eliminate this influence, the data is preprocessed by normalization.

[0168] Training the classifier: Using the above features to train a random forest algorithm to generate a classifier to determine whether a ransomware attack has occurred. The random forest algorithm is a supervised learning algorithm, an ensemble algorithm composed of a group of decision trees, capable of classification and regression. When performing binary classification on ransomware and benign software, each decision tree in the random forest separately judges the input samples to obtain their respective classification results, and the random forest takes the most frequent class among the decision tree classification results as the final result.

[0169] Alarm publishing: When the above random forest classifier identifies a ransomware attack, the system will quickly publish an alarm to remind the user to take corresponding protective measures to prevent the further spread of ransomware.

[0170] Thus, the present invention has the following technical effects:

[0171] 1. Implementing anti-ransomware detection technology for the file system based on the distributed storage Ceph system, which solves the problem that the storage system is severely threatened by ransomware.

[0172] 2. By real-time monitoring and analyzing the file access behavior information in the storage system to identify potential ransomware threats, it can solve the deficiencies of static detection in being unable to detect unknown ransomware and being difficult to cope with ransomware code obfuscation and code injection techniques.

[0173] 3. By combining the quick screening module and the in-depth screening module, integrating the file I / O access behavior characteristics and the file content characteristics, and adopting machine learning algorithms, the attack of ransomware can be detected quickly and accurately, while reducing the resource occupancy and performance impact on the system.

[0174] 4. When a ransomware attack is detected, this aspect can quickly issue an alarm to remind users to take corresponding protection measures, which helps to prevent the further spread of ransomware and protect the security of other systems and data.

[0175] Corresponding to the above method embodiments, the embodiment of the present invention also provides a file detection device, and the file detection device described below can be correspondingly referred to the file detection method described above.

[0176] See Figure 6 As shown, the device includes the following modules:

[0177] The file system activity monitor 101 is used to monitor the file system to obtain the first access behavior characteristics of the accessed files in the file system; the first access behavior characteristics are used to distinguish normal I / O from suspicious I / O;

[0178] The quick screening module 102 is used to use the first classifier to determine suspicious files from the accessed files based on the first file access characteristics;

[0179] The in-depth screening module 103 is used to obtain the second access behavior characteristics and file content characteristics of the suspicious files; the second access behavior characteristics are used to identify ransom I / O; use the second classifier to determine the compromised files controlled by the ransomware from the suspicious files based on the second access behavior characteristics and file content characteristics.

[0180] Applying the device provided by the embodiment of the present invention, monitor the file system to obtain the first access behavior characteristics of the accessed files in the file system; the first access behavior characteristics are used to distinguish normal I / O from suspicious I / O; use the first classifier to determine suspicious files from the accessed files based on the first file access characteristics; obtain the second access behavior characteristics and file content characteristics of the suspicious files; the second access behavior characteristics are used to identify ransom I / O; use the second classifier to determine the compromised files controlled by the ransomware from the suspicious files based on the second access behavior characteristics and file content characteristics.

[0181] Since detection is based on file access characteristics, it is faster than detecting the file content itself. Therefore, in the present invention, in order to effectively determine whether there is an attack by ransomware in the file system, it is first necessary to monitor the file system and obtain the first access behavior characteristics of the accessed files in the file system. Then, the first file access characteristics are input into the first classifier to quickly screen the accessed files, so as to quickly screen out suspicious files from the accessed files. Since the characteristics based on the file content itself can more accurately detect whether there is a ransom attack, therefore, in the present invention, after the suspicious files are screened out, the second access behavior characteristics can be combined with the file content characteristics, so as to use the second classifier to determine the compromised files controlled by the ransomware from the suspicious files, thereby completing the ransom detection. That is to say, the present invention performs ransom detection based on access behavior, can face unknown ransomware, performs ransom detection based on file content characteristics, can improve the accuracy of ransom detection, and combines quick screening and in-depth screening, so as to quickly and accurately complete ransom detection.

[0182] In addition, during the processes of quick screening and in-depth screening, classifiers are used for screening. Since the classifier itself can be trained and optimized, therefore, the file detection of the present invention can also be more effective along with the training and optimization of the classifier.

[0183] In a specific embodiment of the present invention, the file system activity monitoring is specifically used to perform real-time monitoring during the interaction between the software and the file system to obtain file I / O log information;

[0184] Based on a sliding window, extract from the file I / O log information whether the accessed file has been renamed or deleted, the same proportion of file name suffixes, whether the file type has changed, and the file type;

[0185] Determine whether the file has been renamed or deleted, the same proportion of file name suffixes, whether the file type has changed, and the file type as the first access behavior characteristics.

[0186] In a specific embodiment of the present invention, the in-depth screening module is specifically used to extract, based on a sliding window, the number of times a suspicious file is read, the number of times a file is written, the number of times a file is deleted, the number of times a file is renamed, the number of times the file type changes, and the entropy of the write buffer from the file I / O log information;

[0187] Determine the number of times a file is read, the number of times a file is written, the number of times a file is deleted, and the number of times a file is renamed as the second access behavior characteristics;

[0188] Determine the number of times the file type changes and the entropy of the write buffer as the file content characteristics.

[0189] In a specific embodiment of the present invention, the file system activity monitoring is specifically used to obtain file I / O log information by using hooks during the interaction between the software and the file system;

[0190] Or, during the interaction between the software and the file system, communicate with the file system or the monitoring software that monitors the file system to obtain file I / O log information.

[0191] In a specific embodiment of the present invention, the training module is used to train the first classifier and the second classifier. The process of training the first classifier includes:

[0192] Obtain the first training sample; the first training sample has label information indicating whether it belongs to a suspicious file;

[0193] Obtain the first file access feature of the first training sample;

[0194] Based on the decision tree algorithm, support vector machine algorithm, multi-layer perceptron algorithm, adaptive boosting algorithm or nearest neighbor algorithm, input the first file access feature into the first preset classifier for training;

[0195] After the training is completed, determine the first preset classifier as the first classifier;

[0196] And / or, the process of training the second classifier includes:

[0197] Obtain the second training sample; the second training sample has label information indicating whether it belongs to a compromised file;

[0198] Obtain the second file access feature and file content feature of the second training sample;

[0199] Based on the random forest algorithm, long short-term memory network, gated recurrent unit structure, convolutional neural network or deep residual network, input the second file access feature and file content feature of the second training sample into the second preset classifier for training;

[0200] After the training is completed, determine the second preset classifier as the second classifier.

[0201] In a specific embodiment of the present invention, the ransomware defense module is used to obtain the I / O log information of the compromised file after determining the compromised file;

[0202] Use the I / O log information to determine the suspicious software that has accessed the compromised file;

[0203] Prohibit the suspicious software from accessing the file system.

[0204] In a specific embodiment of the present invention, a normalization processing module is configured to calculate the average value and standard deviation of the feature values of each dimension in the detection features after obtaining at least one type of detection feature among the first file access feature, the second file access feature, and the file content feature;

[0205] Subtract the average value of the dimension to which the feature value belongs from the feature value to obtain a difference value;

[0206] Determine the ratio of the difference value to the standard deviation of the dimension to which it belongs as the standardized value corresponding to the feature value;

[0207] Replace all the feature values in the detection features with the standardized values.

[0208] Corresponding to the above method embodiment, the embodiment of the present invention further provides an electronic device, and an electronic device described below can be correspondingly referred to with a file detection method described above.

[0209] See Figure 7 As shown, the electronic device includes:

[0210] A memory 332 for storing a computer program;

[0211] A processor 322 for implementing the steps of the file detection method in the above method embodiment when executing the computer program.

[0212] Specifically, please refer to Figure 8 , Figure 8 , which is a schematic structural diagram of an electronic device provided in this embodiment. The electronic device may vary greatly due to configuration or performance differences, and may include one or more processors (central processing units, CPUs) 322 (for example, one or more processors) and a memory 332. The memory 332 stores one or more computer programs 342 or data 344. Among them, the memory 332 can be short-term storage or persistent storage. The program stored in the memory 332 may include one or more modules (not marked in the figure), and each module may include a series of instruction operations on the data processing device. Further, the processor 322 may be configured to communicate with the memory 332 and execute a series of instruction operations in the memory 332 on the electronic device 301.

[0213] The electronic device 301 may further include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341.

[0214] The steps in the file detection method described above can be implemented by the structure of the electronic device.

[0215] Corresponding to the above method embodiments, an embodiment of the present invention further provides a readable storage medium. A readable storage medium described below can be correspondingly referred to with a file detection method described above.

[0216] A readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the file detection method in the above method embodiments are implemented.

[0217] The readable storage medium can specifically be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disc, or any other readable storage medium capable of storing program codes.

[0218] In this specification, each embodiment is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple. For the relevant parts, refer to the descriptions in the method part.

[0219] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in the form of hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0220] The steps of the method or algorithm described in combination with the embodiments disclosed in this article can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium well-known in the technical field.

[0221] Finally, it should also be noted that in this text, relationships such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms including, containing or any other variant are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device.

[0222] In this article, specific examples are used to elaborate on the principles and implementation manners of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A file detection method, characterized in that, Including: Monitoring the file system to obtain the first access behavior characteristics of the accessed files in the file system; The first access behavior characteristics are used to distinguish normal I / O from suspicious I / O; Using a first classifier, determining suspicious files from the accessed files based on the first file access characteristics; Obtaining the second access behavior characteristics and file content characteristics of the suspicious files; The second access behavior characteristics are used to identify ransomware I / O; Using a second classifier, determining compromised files controlled by ransomware from the suspicious files based on the second access behavior characteristics and the file content characteristics.

2. The method according to claim 1, characterized in that, The monitoring the file system to obtain the first access behavior characteristics of the accessed files in the file system includes: Performing real-time monitoring during the interaction between the software and the file system to obtain file I / O log information; Based on a sliding window, extracting from the file I / O log information whether the accessed files have been renamed or deleted, the same proportion of file name suffixes, whether the file type has changed, and the file type; Determining whether the accessed files have been renamed or deleted, the same proportion of file name suffixes, whether the file type has changed, and the file type as the first access behavior characteristics.

3. The method according to claim 2, characterized in that, The obtaining the second access behavior characteristics and file content characteristics of the suspicious files includes: Based on a sliding window, extracting from the file I / O log information the number of times the suspicious files are read, written, deleted, renamed, the number of times the file type changes, and the entropy of the write buffer; Determining the number of times the suspicious files are read, written, deleted, and renamed as the second access behavior characteristics; Determining the number of times the file type changes and the entropy of the write buffer as the file content characteristics.

4. The method according to claim 2, characterized in that, The performing real-time monitoring during the interaction between the software and the file system to obtain file I / O log information includes: During the interaction between the software and the file system, using a hook to obtain the file I / O log information; Or, during the interaction between the software and the file system, communicating with the file system or the monitoring software monitoring the file system to obtain file I / O log information.

5. The method according to claim 1, characterized in that, The process of training the first classifier includes: Obtaining a first training sample; the first training sample has label information indicating whether it belongs to a suspicious file; Obtaining the first file access characteristics of the first training sample; Based on a decision tree algorithm, a support vector machine algorithm, a multi-layer perceptron algorithm, an adaptive boosting algorithm, or a nearest neighbor algorithm, inputting the first file access characteristics into a first preset classifier for training; After the training is completed, determining the first preset classifier as the first classifier; And / or, the process of training the second classifier includes: Obtaining a second training sample; the second training sample has label information indicating whether it belongs to a compromised file; Obtaining the second file access characteristics and file content characteristics of the second training sample; Based on the random forest algorithm, long short-term memory network, gated recurrent unit structure, convolutional neural network, or deep residual network, input the second file access feature and file content feature of the second training sample into a second preset classifier for training; After the training is completed, determine the second preset classifier as the second classifier.

6. The method according to claim 1, characterized in that, After determining the compromised file, it further includes: Obtain the I / O log information of the compromised file; Use the I / O log information to determine the suspicious software that has accessed the compromised file; Prohibit the suspicious software from accessing the file system.

7. The method according to any one of claims 1 to 6, characterized in that, After obtaining at least one type of detection feature among the first file access feature, the second file access feature, and the file content feature, it further includes: Calculate the average value and standard deviation of the feature values of each dimension in the detection feature; Subtract the average value of the dimension to which the feature value belongs from the feature value to obtain a difference; Determine the ratio of the difference to the standard deviation of the dimension to which the difference belongs as the standardized value corresponding to the feature value; Replace all the feature values in the detection feature with the standardized values.

8. A file detection device, characterized in that, It includes: File system activity monitoring, which is used to monitor the file system and obtain the first access behavior feature of the files accessed in the file system; The first access behavior feature is used to distinguish normal I / O from suspicious I / O; A quick screening module, which is used to use a first classifier to determine suspicious files from the accessed files based on the first file access feature; A deep screening module, which is used to obtain the second access behavior feature and file content feature of the suspicious files; The second access behavior feature is used to identify ransomware I / O; use a second classifier to determine the compromised files controlled by ransomware from the suspicious files based on the second access behavior feature and the file content feature.

9. An electronic device, characterized in that, It includes: A memory, which is used to store computer programs; A processor, which is used to implement the steps of the file detection method according to any one of claims 1 to 7 when executing the computer program.

10. A readable storage medium, characterized in that, A computer program is stored on the readable storage medium, and when the computer program is executed by the processor, it implements the steps of the file detection method according to any one of claims 1 to 7.

Images