System and method for detecting malicious files

Abstract

The present invention relates to a system and method for detecting malicious files. The method comprises: training a mapping model for mapping files in a probability space; determining multiple characteristics of an analyzed file based on a set of rules; generating a mapping of the analyzed file in the probability space based on the determined multiple characteristics; using the generated mapping of the analyzed file to search a first database to determine whether the analyzed file is associated with a malicious file family, the first database storing mappings associated with one or more malicious file families; and in response to determining that the analyzed file is associated with a malicious file family, selecting one or more malware detection methods from a second database, the second database storing multiple malware detection methods, the selected methods being used to detect the associated malicious file family.

Description

Technical Field

[0001] The present invention relates to the field of information technology (IT) security, and more particularly to detection of malicious files. Background Art

[0002] With the rapid development of computer technology, various electronic devices (such as personal computers) and various portable devices (such as laptops, tablet computers, mobile phones, etc.) are now widely used. Today, such devices are the main communication devices in business, industry, government, and social life. For example, mobile phones provide people with various functions, such as short message services, communications, games, calendars, music, etc. With the growth of the number of computing devices and the number of software running on these devices, the number of malicious programs is also growing rapidly.

[0003] There are a large number of malicious programs of various types. Some types of malicious programs are used to steal personal and confidential data (such as usernames and passwords, bank details, electronic documents) from various user devices. Other types of malicious programs form so-called botnets from user devices for use in attacks such as distributed denial of service (DDoS) against other computers or for cracking user passwords using so-called brute force methods. Still other types of malicious programs recommend paid content to users through intrusive advertising, spam, sending text messages (SMS) to premium numbers, etc.

[0004] Typically, specialized programs called antivirus programs are used to combat malicious programs, including detecting malicious programs, preventing infection, and restoring the working capacity of computing devices infected by malicious programs. Antivirus programs use a variety of techniques to detect various malicious programs, such as:

[0005] Static analysis (including signature analysis and whitelist / blacklist analysis) - analysis of a program for maliciousness based on data contained in the files that make up the program being analyzed, not including startup or

[0006] Simulate the execution of the analyzed program and use statistical analysis:

[0007] o Signature analysis - searches for correspondence between any part of the analyzed program's code and known code (signatures) from a database of malware signatures;

[0008] o Whitelist and blacklist - search the checksum calculated from the analyzed program (or its part) in the checksum database of malware (blacklist) or in the checksum database of security software (whitelist);

[0009] Dynamic analysis (including heuristic analysis and active defense) - analysis of malware programs based on data obtained during the execution of the analyzed program. During dynamic analysis, the following data can be used:

[0010] o Heuristic analysis - emulates the work of the analyzed program, creates emulation logs (containing data about calls to Application Programming Interface (API) functions, passed transmission parameters, code segments of the analyzed program, etc.), and searches the created logs for data that matches data from the malware signature database;

[0011] o Active defense - intercepts calls to API functions of the running analyzed program, creates a behavior log of the analyzed program (containing data about calls to API functions, passed parameters, code segments of the analyzed program, etc.), and searches the created log for data that matches data from the database of malicious call data.

[0012] Both static and dynamic analysis methods have their advantages and disadvantages. Static analysis has lower resource requirements for the computing device on which the analysis is performed, and because static analysis does not require the execution or simulation of the program being analyzed, static analysis is faster, but less effective. For example, static analysis may have a lower malware detection percentage and a higher false positive percentage (i.e., the analyzed file is determined to be malicious by the antivirus program even though the analyzed file is safe). Due to the fact that dynamic analysis uses data obtained during the execution or simulation of the analyzed program, dynamic analysis is slower and has higher resource requirements for the computing device on which the analysis is performed, but is also more effective.

[0013] Currently, in order to reliably detect most malicious programs, it may be necessary to use several different methods of detecting malicious programs in succession, which is often redundant.

[0014] For example, if a malicious program is a well-known program and detection rules have been developed for it, it may not be necessary to employ dynamic detection methods, which are more demanding on the resources of the computing device on which it is analyzed. In other words, such malicious programs can be detected by fast methods (e.g., signature analysis) that are less demanding on computing resources. However, if the malicious program is new, the use of fast methods may not be as effective and may not result in the correct detection of such new malicious programs. In such cases, it may be more appropriate to use more effective malware detection methods, even if such methods are more demanding on the resources of the computing device.

[0015] Therefore, if it is possible to quickly and accurately determine in advance to which category a malicious program being analyzed belongs, it would be desirable to select a method of detecting the malicious program that will generally provide the highest level of effectiveness.

[0016] Although conventional security techniques handle well the detection of malicious files having specific characteristic signatures that are similar to characteristic signatures of known malicious files (i.e., data describing specific characteristics of files from a particular group of files, such as the presence of a graphical interface, data encryption, data transmission over a computer network, etc.), conventional security techniques often fail to detect malicious files having characteristic signatures that are different from characteristic signatures of known malicious files (despite having similar behavior).

[0017] Therefore, it is necessary to choose a method to detect malicious files in order to use computer resources more efficiently. Summary of the invention

[0018] Aspects of the invention relate to the field of antivirus scanning of files.

[0019] One technical effect of the present invention is to optimize the utilization of computing resources when scanning files for malware.

[0020] Another technical effect of the present invention is to improve the effectiveness of selecting a mode for detecting malicious files.

[0021] Yet another technical effect of the present invention is to change the category of the file being analyzed.

[0022] In one exemplary aspect, a method for detecting malicious files includes: training a mapping model for mapping files in a probability space that characterizes one or more malicious file families; determining multiple characteristics of the analyzed file based on a set of rules; generating a mapping of the analyzed file in the probability space based on the determined multiple characteristics; using the generated mapping of the analyzed file to search a first database to determine whether the analyzed file is associated with a malicious file family, the first database storing mappings associated with one or more malicious file families; in response to determining that the analyzed file is associated with a malicious file family, selecting one or more malware detection methods from a second database storing multiple malware detection methods, the selected methods being used to detect the associated malicious file family.

[0023] In one aspect, the probability space is a multidimensional space. The probability space provides a probability distribution for the plurality of characteristics associated with the analyzed file. In one aspect, a pair of one of the plurality of characteristics of the analyzed file and one of the plurality of attributes associated with a behavior of the analyzed file is matched with a probability that the behavior described by the one of the plurality of attributes is unique to the analyzed file having the one of the plurality of characteristics.

[0024] In one aspect, the generated map comprises an aggregation of a plurality of parameters describing faces associated with the analyzed file.The faces are located in the probability space.

[0025] In one aspect, the first database includes a map of files associated with a particular malicious file family.

[0026] In one aspect, the first database includes a summary map of a plurality of files associated with a particular malicious file family.

[0027] In one aspect, searching the first database includes identifying an association between the analyzed file and one of a plurality of files associated with a malicious file family. A similarity between a mapping of the analyzed file and a mapping of the one of the files associated with the malicious file family exceeds a predetermined threshold.

[0028] On the one hand, the method also includes: retraining the mapping model to increase the corresponding similarities between the corresponding files in the retrained mapping model compared to the similarities between the mapping of the analyzed file and the mapping of one of the multiple files associated with the malicious file family in the mapping model before performing the retraining. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more exemplary aspects of the present invention and, together with the detailed description, serve to explain the principles and implementations of these exemplary aspects.

[0030] Figure 1 A high-level architecture of a system implementing a method for detecting malicious files according to aspects of the present invention is shown.

[0031] Figure 2 A flow chart of a method for detecting malicious files according to aspects of the present invention is shown.

[0032] Figure 3 A schematic diagram illustrating an example of a mapping of malicious file families in a probability space according to aspects of the present invention is shown.

[0033] Figure 4 A high-level architecture of a system for training a mapping model configured to map files in a probabilistic space according to aspects of the present invention is shown.

[0034] Figure 5 A flow chart of a method for training a mapping model according to aspects of the present invention is shown.

[0035] Figure 6 A schematic diagram illustrating an example of selecting a file based on comparison of mappings according to aspects of the present invention is shown.

[0036] Figure 7 A high-level architecture of a system for detecting sources of malicious activity according to aspects of the present invention is shown.

[0037] Figure 8 A flow chart of a method of detecting a source of malicious activity according to aspects of the present invention is shown.

[0038] Fig. 9 An example of a general purpose computer system upon which aspects of the present invention may be implemented is shown. DETAILED DESCRIPTION

[0039] Exemplary aspects are described herein in the context of systems, methods, and computer program products for spam identification. One of ordinary skill in the art will recognize that the following description is merely exemplary and is not intended to be limiting in any way. Other aspects will readily suggest themselves to one of ordinary skill in the art who appreciates the advantages of the present invention. Reference will now be made in detail to implementations of the exemplary aspects as shown in the accompanying drawings. The same reference numerals will be used to refer to the same or similar items as much as possible throughout the drawings and the following description.

[0040] A number of definitions and concepts that will be used in describing various aspects of the present invention will now be introduced.

[0041] Secure file - a file whose execution is known not to result in the unauthorized destruction, blocking, modification, copying of computer information, or to defeat the protection measures used to protect computer information.

[0042] Malicious files - files whose execution is known to cause unauthorized destruction, blocking, modification, copying of computer information, or to render ineffective the protection of computer information.

[0043] Malicious behavior of an executable file - a set of actions that can be performed during the execution of the file and are known to result in the unauthorized destruction, blocking, modification, copying of information, or the defeat of the protection measures of computer information.

[0044] Malicious activities of an executable file - a set of actions performed by that file based on its malicious behavior.

[0045] Suspicious file – a file the execution of which can lead to unauthorized destruction, blocking, modification, copying of computer information, or to the defeat of the means of protection of computer information with a certain probability, wherein such probability can be assessed based on data about the file itself (source of its origin, information about its developer, popularity among users) or based on data about the behavior of the operating system or application when executing the file.

[0046] Conditionally secure file - a file whose execution, according to its stated functions, shall not result in the unauthorized destruction, blocking, modification, duplication, or defeat of the protection of computer information, even though such result could be achieved during unauthorized use of the file or when the file is used by methods other than those stated.

[0047] Various aspects of the present invention enable selection of methods for detecting malicious files.

[0048] Figure 1 A high-level architecture of a system implementing a method for detecting malicious files according to aspects of the present invention is shown.

[0049] The system implementing the method for detecting malicious files includes a file 101, a malicious file detection method 102, a file characteristic determination module 110 configured to determine the characteristics of the file 101, a mapping module 120 configured to generate a mapping of the file 101 based on a mapping model 121, a search module 130, a mapping database 131, a selection module 140, a malicious file detection method database 141, and a retraining module 150.

[0050] According to aspects of the present invention, the file characteristic determination module 110 may be configured to determine the characteristic of the file 101 based on a predetermined rule. In addition, the file characteristic determination module 110 may be configured to send the determined characteristic of the file 101 to the mapping module 120.

[0051] In an exemplary aspect, the characteristics of the file 101 may include, but are not limited to, metadata associated with the file 101, the file structure of the file 101, encapsulation information associated with the file 101, a byte representation of the file 101, a representation of the disassembled binary file 101, an import / export table associated with the file 101, a string constant contained in the file 101, a work result of an artificial expert heuristic algorithm associated with the file 101, etc. The metadata associated with the file 101 may include, but are not limited to, the file name, file size, file type, creation date, modification date, access date, file access rights, etc. The file structure of the file 101 may include, but are not limited to, data written in the file header, information related to the file segment, such as the size and type of the file segment, the name of the file segment, etc. The encapsulation information associated with the file 101 may include data associated with the encapsulator that encapsulates the file 101, such as, but not limited to, the name and version of the encapsulator, encapsulation parameters, etc. The work result of the artificial expert heuristic algorithm associated with the file 101 may include an artificial expert heuristic algorithm that constitutes a set of rules for searching for various dependencies and for calculating various weighting factors, etc. As non-limiting examples, the file characteristic determination module 110 may also be configured to determine the following characteristics of the file 101:

[0052] File size (in bytes);

[0053] The number of segments;

[0054] The presence of a signature;

[0055] Entropy (frequency distribution of characters, lexemes, etc., and deviations of these distributions from a normal distribution);

[0056] The size of the import table;

[0057] The number of assembler jump instructions;

[0058] A flag indicating that file 101 contains the byte sequence 0xABCDEF1313.

[0059] The mapping module 120 may be configured to generate a mapping of the file 101 in the probability space using the trained mapping model 121 based on the determined characteristics of the file 101. The mapping module 120 may also be configured to send the formed mapping of the file 101 to the search module 130. In one aspect, the probability space is a multidimensional space in which each pair of characteristics of the file 101 and the behavioral attributes of the file 101 may be matched with the probability that the behavior described by the given behavioral attributes is specific to the file 101 having those characteristics.

[0060] In another aspect, the mapping of file 101 may include an aggregation of parameters describing faces that lie in the probability space.

[0061] In another aspect of the present invention, the behavior attributes of the file 101 may include but are not limited to:

[0062] The structure and content of network traffic generated by the executable file 101;

[0063] The state of the operating system during and after the completion of execution of file 101 (including the operating system's use of computing resources, such as RAM, processor workload, linked dynamic libraries, started services, etc.);

[0064] Information about user interactions with executable file 101;

[0065] The result of the artificial expert heuristic algorithm working on the file 101.

[0066] On the other hand, the mapping model 121 can be trained using one or more of the following methods: deep learning algorithm, genetic algorithm, mapping (embedding) method, such as but not limited to, latent vector space method, vector distribution method, reinforcement learning method, etc.

[0067] In another aspect, the mapping of file 101 may be generated using at least one of the following algorithms:

[0068] Deep neural networks

[0069] ·convolution;

[0070] Recurrent Neural Networks (RNNs);

[0071] Neural encoders;

[0072] Variational autoencoder;

[0073] Bayesian networks;

[0074] Similarity learning;

[0075] • Representations in latent hyperbolic spaces (as a special case of representations in latent non-Euclidean spaces or hyperbolic embeddings).

[0076] The search module 130 may be configured to search for malicious file families in the mapping database based on the formed mapping of the file 101. The search module 130 may also be configured to send the search results to the selection module 140.

[0077] In a variation of the present aspect, the mapping database 131 may include, but is not limited to: a mapping of files belonging to a given malicious file family, a summary mapping of all files belonging to a given malicious file family, and the like.

[0078] On the other hand, the search module 130 can search the mapping database 131 for the following malicious file families, which include but are not limited to: a family containing the following file, the similarity between the mapping of the file and the mapping of the formed file 101 is greater than a predetermined threshold; a family with the following summary mapping, the similarity between the summary mapping of all files and the mapping of the formed file 101 is greater than a predetermined threshold.

[0079] In another aspect, the search module 130 may employ one or more of the following techniques:

[0080] Nearest neighbor search, such as locality-sensitive hashing, ball trees, etc.

[0081] Density estimation, such as kernel density estimation functions, MinCount sketch.

[0082] It should be noted that the following will be combined with Figure 3 Explain the search results in more detail.

[0083] The selection module 140 may be configured to select a malicious file detection method 102 corresponding to a malicious file family found in the mapping database 131 from the malicious file detection method database 141 .

[0084] The selection module 140 may be further configured to change the category of the analyzed file 101 if a category has been previously assigned to the analyzed file 101. Such categories may include, but are not limited to:

[0085] Malicious file category

[0086] ·Security document category;

[0087] · Conditionally secure document categories;

[0088] Suspicious file categories;

[0089] Unknown file type.

[0090] Retraining module 150 may be configured to retrain mapping model 121 in a manner such that the generated mapping of file 101 is more similar to the mapping of at least one file from a given malicious file family than before retraining mapping model 121 .

[0091] In one aspect, retraining of mapping model 121 may involve at least:

[0092] Start training the mapping model 121;

[0093] ·Continuously train the mapping model 121, correct the mapping of the known file 101, and train the model based on the corrected mapping.

[0094] Advantageously, selecting a method for detecting malicious files solves many of the aforementioned technical problems. For example, selecting the best detection method reduces the utilization of resources of the computing device on which the analysis is performed because the method only uses the dedicated detection means with the highest effectiveness (e.g., detection rate). In addition, this selection reduces the burden on analysts who manually handle malicious files. In other words, the disclosed technology enables the selection of new files that do not belong to a known family of malicious files (e.g., files with little-known or unknown structures or behaviors) from a given file sample. Various aspects of the present invention contemplate the classification of files to be analyzed into files with known behaviors and structures (for which effective detection means are already known) and files with unknown behaviors or structures (for which no effective detection means are known). Conventional malware detection techniques use several detection means in combination to detect malicious files, which is suboptimal. A simplified algorithm for the selection of the best method for malware detection may include the following steps performed by the mapping module 120: For all files 101 previously assigned for dynamic analysis, a vector of behavior parameters is calculated based on the behavior log:

[0095] V i =ML2(log i ),

[0096] ML2 is a trained mapping model 121, which uses the behavior parameters log of file 101. i Work,

[0097] And these vectors are saved in a database;

[0098] 1) For all files 101 that have not been analyzed above, calculate:

[0099] V i =ML1(log i )

[0100] Among them, ML1 is a trained mapping model that uses the characteristics of file 101 t Work;

[0101] 2) If computing resources are available for dynamic analysis of K files, select K files from all previously unprocessed files so that their distance to the most recent previously known file behavior log is minimized (the

[0102] The processing steps are combined below Figure 4 Describe in more detail):

[0103] Novelty(file)=min {i＝1..N} Distance(ML1(file),v i).

[0104] It should be noted that in some cases:

[0105] 1) significantly more complex property descriptions of the file 101 (both characteristics of the file 101 and behavioral parameters of the file 101) may be used;

[0106] 2) For models ML1 and ML2, significantly more complex architectures (EncF, DecF, EncL, DecL) can be used, where

[0107] EncF is a function that maps files to probability space.

[0108] EncL is a function that maps file behavior logs to probability space.

[0109] DecF is a function that maps data from probability space to a file.

[0110] DecL is a function that maps data from probability space to file behavior logs;

[0111] 3) significantly more flexible distance functions can be used (e.g., models ML1, ML2 can return a parameterized distribution into an infinite-dimensional space at the output, and can compute asymmetric similarity measures between these two distributions);

[0112] 4) The construction principle of functions ML1 and ML2 can be modified. For example, instead of training 2 models, 4 models can be trained:

[0113] EncF(file,W f )→v,

[0114] EncL(log,W l )→v,

[0115] DecF(v,V f )→file′,

[0116] DecL(v,V l )→log′,

[0117] And the training can try to minimize the distance between the initial attribute description and the decoded attribute description:

[0118] Distance(file i ,DecF(EncF(file i ))),

[0119] Distance(log i ,DecL(EncF(file i ))),

[0120] Distance(file i ,DecF(EncL(log i ))),

[0121] Distance(log i ,DecL(EncL(log i ))),

[0122] And maximize the corresponding distances for different file behavior log pairs;

[0123] 5) For the same file, multiple behavior logs corresponding to it can be saved (the file behavior may change when running at different times), and these multiple behavior logs can be considered when evaluating similarity;

[0124] 6) In some cases, not all representations from previously processed files are saved, but for example representations of certain representatives or centroids may be saved (to save memory / speed up searching);

[0125] 7) In some cases, priority chains can be used:

[0126] Mild static analysis → Mild simulator → Deep static analysis → Long simulation

[0127] And the model ML1 (file light )、ML2(log light )、ML1(file deep )、ML2(log deep ).

[0128] Therefore, after each processing step, the most familiar object (i.e., the most familiar object) can be picked out based on the known ML1 (file light )、ML2(log light ) and prioritize sending for deep static analysis).

[0129] Figure 2 A flow chart of a method for detecting malicious files according to aspects of the present invention is shown.

[0130] The flowchart of the method for selecting a malicious file detection method includes step 210, in which the file characteristic determination module 110 determines the file characteristic based on a predetermined rule. In step 220, the mapping module 120 generates a mapping of the file 101 in the probability space using the trained mapping model 121 based on the file characteristic determined in step 210. In step 230, the search module 130 searches for malicious file families in the mapping database 131 based on the mapping of the file 101 generated in step 220.

[0131] In step 240, the selection module 140 selects a malicious file detection method 102 from the malicious file detection method database 141. According to one aspect of the present invention, the selected method corresponds to the malicious file family found in the mapping database 131 in step 230. In addition, in step 240, the selection module 140 may change the category of the file 101 being analyzed (if a category has been assigned to the file before) using the selected malicious file detection method. The category may include, but is not limited to:

[0132] Malicious file category

[0133] ·Security document category;

[0134] · Conditionally secure document categories;

[0135] Suspicious file categories;

[0136] Unknown file type.

[0137] In step 250 , the retraining module 150 retrains the mapping model 121 so that the generated mapping of the file 101 is more similar to the mapping of at least one file from a given malicious file family than before the mapping model 121 is retrained.

[0138] Figure 3 A schematic diagram illustrating an example of a mapping of malicious file families in a probability space according to aspects of the present invention is shown.

[0139] The graph of mappings of malicious program families in probability space includes mapping 310 of files of family AdWare.Win32.SmartInstaller, mapping 320 of files of family Worm.Win32.Vobrus, mapping 330 of files of family Backdoor.Win32.DarkKomet, mapping 340 of files of family Trojan.Win32.ShipUp, mapping 350 of files of family Trojan.Win32.AntiAV, and mapping 360 of files of family Trojan.Win32.Bayrod. Figure 3Also shown are a mapping of files in a malicious set 301 built based on the characteristics of these files, a mapping of files in a malicious set 302 built based on the behavioral attributes of these files, a mapping of files in a given malicious file family 303 built based on the characteristics of these files, and a mapping of files in a given malicious file family 304 built based on the behavioral attributes of these files.

[0140] in other words, Figure 3 A graph showing file mappings from multiple malicious file families is shown. Files from various malicious file families have similar functionality (executable file behavior) and similar structure. It should be noted that this similarity means that mappings for files of the same family are grouped in compact regions of the probability space.

[0141] If the mapping of a certain file 101 is also located in the area occupied by the mapping of a file of any malicious file family, it can be determined with a high probability that the file 101 may also belong to the malicious file family.

[0142] If the mapping of a certain file 101 is outside the area occupied by the mappings of files of the malicious file family, it can be determined with a high probability that the file 101 is a new variant that does not belong to the known malicious file family.

[0143] In the first case (when the mapping of a certain file 101 is also located in the area occupied by the mapping of files of any malicious file family), in order to verify that the certain file 101 is malicious, it is usually sufficient to use one of the malicious file detection methods that are specifically used to detect malicious files in a given malicious file family with a high probability. However, in the second case (when the mapping of a certain file 101 is located outside the area occupied by the mapping of files of the malicious file family), a more careful manual analysis by a virus analyst may be required, or multiple malicious file detection methods may need to be used. As mentioned above, at least in some cases, such analysis may require a large waste of computing resources.

[0144] Figure 4 A high-level architecture of a system for training a mapping model configured to map files in a probabilistic space according to aspects of the present invention is shown.

[0145] A system for training a model for mapping files in a probability space includes: a training sample 401 including at least one file 101, a mapping model 121, a mapping database 131, a detection method database 141, a file characteristic determination module 110, an attribute behavior determination module 410, a mapping generation module 420, a detection module 430 and a training module 440.

[0146] The attribute behavior determination module 410 is configured to determine the behavior attributes of the file 101 based on the action log. The log may be obtained based on the analysis of the execution of the file 101. The attribute behavior determination module 410 may also be configured to send the determined behavior attributes of the file 101 to the mapping generation module 420.

[0147] For example, the attribute behavior determination module 410 may be configured to determine the following behavior attributes of the file 101:

[0148] The number of threads running;

[0149] A flag indicating that a record has been made in the registry;

[0150] The number of objects downloaded from a computer network (LAN or WAN);

[0151] A sign indicating that the trap system has been accessed;

[0152] A flag indicating that the program has drawn a graphical interface (using the corresponding GUI function);

[0153] A flag indicating successful completion of the procedure;

[0154] Maximum consumption of RAM.

[0155] The mapping generation module 420 may be configured to generate a first mapping of the file 101 in the probability space based on the determined characteristics of the file 101, and to generate a second mapping of the file 101 in the probability space based on the determined behavioral attributes of the file 101. The mapping generation module 420 may also be configured to send the formed file mapping to the training module 440.

[0156] In one aspect, the probability space may be a multidimensional space in which each pair of characteristics of file 101 and behavioral attributes of file 101 is matched with a probability that the behavior described by a given behavioral attribute is unique to file 101 having those characteristics.

[0157] In another aspect, the mapping of file 101 may be an aggregation of parameters (eg, coefficients in a second order polynomial) that describe a surface located in the multi-dimensional space.

[0158] Detection module 430 may be configured to detect file 101, determine which malicious file family file 101 belongs to, and determine at least one malicious file detection method that gives correct results during detection of file 101. Detection module 430 may also be configured to send the determined information to training module 440.

[0159] The training module 440 may be configured to train the mapping model 121 based on the generated first mapping and second mapping of the file 101 .

[0160] In one aspect, the training module 440 can be further configured to add the generated mappings of the files 101 corresponding to a particular malicious file family to the mapping database 131. In one aspect, the mapping database 131 can include, but is not limited to:

[0161] A map of files belonging to a given malicious file family;

[0162] A summary map of all files belonging to a given malicious file family.

[0163] In another aspect, the training module 440 may be additionally configured to add the detection method determined during the detection of the file 101 to the detection method database 141. The determined detection method corresponds to the determined malicious file family.

[0164] An example of a simple training variant of the mapping model 121 may be:

[0165] ML1(F file ,W f )→{x1,x2},

[0166] ML2(F loe , W l )→{x1,x2},

[0167] in:

[0168] ML1 is a trained mapping model 121 that works using the characteristics of the training samples 401;

[0169] ML2 is a trained mapping model 121 that works with behavioral attributes;

[0170] F file It is a feature of File 101;

[0171] F log is the behavior attribute of file 101;

[0172] W f are adjustable (trainable) parameters of model ML1;

[0173] W1 is an adjustable (trainable) parameter of model ML2.

[0174] For example:

[0175] If W f =W, then

[0176] ML1(F file , W f )

[0177] =ML1({a1,a2,...,a 99},W)==[{W[0]+W[1]×a1+W[2]×a2+...+W

[99] ×a 99 , W

[100] +W

[101] ×a1+W

[102] ×a2+...+W

[199] ×a 99}]

[0178] =[{x1,x2}]

[0179] In one aspect, multiple files (e.g., N files, N=1000) may be sent to a file behavior simulator. After the file behavior simulator has been fine-tuned, an action log may be formed:

[0180] {(file1,log1), (file2,log2),...}

[0181] After generating the logs, the distance between two objects (files or file behavior logs) can be determined:

[0182] Distance({x1, y1}, {x2, y2}) = (x1-x2) 2 +(y1-y2) 2 ,

[0183] Distance(file1,file2)=Distance(ML1(file1,W f ),ML1(file2,W f )),

[0184] Distance(file,log)=Distance(ML1(file,W f ), ML2(log,W l )),

[0185] It should be noted that using the process of training the mapping model 121, the parameter W can be selected f , W1, so that the average distance between the mapping of file 101 and its corresponding action log (or the average distance between the characteristics of file 101 and the behavior parameters of file 101) will be less than a predetermined threshold. f , W1, so that the average distance between the file 101 and the external action log of another file will be greater than a predetermined threshold.

[0186] For example, the magnitude of the error can be minimized numerically as shown in the following formula:

[0187]

[0188] Figure 5 A flow chart of a method for training a mapping model according to aspects of the present invention is shown.

[0189] The flowchart of the method for training a mapping model for mapping a file in a probability space includes step 210, in which the file characteristic determination module 110 determines the file characteristic based on a predetermined rule. In step 510, the attribute behavior determination module 410 determines the file behavior attribute of the file 101 based on the action log obtained by analyzing the execution of the file 101. In step 520, the mapping generation module 420 generates a first mapping of the file in the probability space based on the characteristic of the file determined in step 210, and generates a second mapping of the file 101 in the probability space based on the behavior attribute of the file 101 determined in step 510.

[0190] In step 530, the detection module 430 detects the file 101 by determining the malicious file family to which the file 101 belongs. In this step, the detection module 430 may also determine at least one malicious file detection method that gives a correct result during the detection of the file 101. In step 540, the training module 440 trains the mapping model based on the first mapping and the second mapping generated in step 520.

[0191] Additionally, in step 540, if the detection module 430 detects the file 101 in step 530, the detection module 430 adds the mapping of the file 101 corresponding to the specific malicious file family and generated in step 520 to the mapping database 131. In one aspect, the mapping database 131 may include, but is not limited to:

[0192] A map of files belonging to a given malicious file family;

[0193] A summary map of all files belonging to a given malicious file family.

[0194] Combination Figure 6 Explain the rationale for the training in more detail.

[0195] Figure 6 A schematic diagram illustrating an example of selecting a file based on comparison of mappings according to aspects of the present invention is shown.

[0196] The schematic diagram showing an example of selecting files based on comparison of mappings includes a collection of files 610 to be analyzed (which includes the file 611 being analyzed), a mapping 620 of files, and a collection of malicious files 630 (which includes malicious files 631 similar to the file 611 being analyzed).

[0197] In order to analyze the maliciousness of files, a certain sample of files 610 is formed, from which one file 611 is selected one at a time in sequence during the analysis.

[0198] For each file, its characteristics are determined (step 210 ), and a map of the file is generated in the probability space (step 220 ). Next, the generated map is compared 620 with the maps calculated for a predetermined set of malicious files 630 .

[0199] In this process, each analyzed file 611 may be matched with multiple files 631 in the sample from malicious files 630, and the mappings of these files 631 are similar to the mapping of the analyzed file 611.

[0200] In certain aspects, an algorithm for generating mappings for files may be selected such that files having similar structure or having similar behavior during their execution have similar mappings (ie, mappings that lie in the same region of probability space).

[0201] Advantageously, the disclosed method not only determines whether the file 611 is malicious (ie, similar to other malicious files 631), but also determines whether the file 611 belongs to any family of malicious files.

[0202] In another aspect, the above-described systems and methods ( Figures 1 to 6 ) can be used to classify and cluster malicious sets. This classification can divide the entire set of malicious files into several different malicious file families, which are characterized by similar file composition or behavior.

[0203] Figure 7 A high-level architecture of a system for detecting sources of malicious activity according to aspects of the present invention is shown.

[0204] The system for detecting the source of malicious activity includes: an executable file 701 of an operating system, a mapping model 121, a mapping database 131, an attribute behavior determination module 710, a mapping generation module 720, a search module 730, a file characteristic determination module 740, a malicious file identification module 750, and a retraining module 760.

[0205] The attribute behavior determination module 710 is configured to determine the behavior attributes of the file 701 (a collection of files and a single file) running in the operating system based on the action log. These action logs can be obtained based on the analysis of the operation of the operating system. The attribute behavior determination module 710 can also be configured to send the determined behavior attributes of the file 701 to the mapping generation module 720.

[0206] The mapping generation module 720 is configured to generate a mapping of the file 701 in the probability space using the trained mapping model 121 based on the determined behavior attributes of the file 701. The mapping generation module may also be configured to send the generated mapping of the file 701 to the search module 730.

[0207] In one variation, the probability space may be a multi-dimensional space in which each pair of characteristics of a file and behavioral attributes of the file may be matched with a probability that the behavior described by a given behavioral attribute is characteristic for a file having those characteristics.

[0208] In yet another alternative aspect, the mapping of the file may be an aggregation of parameters describing faces located in the multi-dimensional space.

[0209] The search module 730 is configured to search for malicious file families in the mapping database 131 based on the generated file mappings. The search module 730 may also be configured to send the search results to the file characteristic determination module 740.

[0210] In one aspect, malicious file families are searched in mapping database 131. In one aspect, mapping database 131 may include, but is not limited to:

[0211] Files whose mappings are more similar to the generated mappings than a predetermined threshold established;

[0212] A summary map of all documents whose similarity to the generated map is greater than a predetermined threshold.

[0213] File characteristic determination module 740 is configured to identify characteristics of files corresponding to malicious file families found in mapping database 131. File characteristic determination module 740 may also be configured to send the identified characteristics of the files to malicious file identification module 750.

[0214] Malicious file identification module 750 is configured to search for malicious files that are the source of malicious activity in the operating system. This identification generally corresponds to the identification of file characteristics.

[0215] The retraining module 760 is configured to retrain the mapping model 121 so that the mapping of the formed file is more similar to the mapping of at least one file from a given malicious file family than before the retraining of the mapping model 121 .

[0216] In one aspect, the source of the malicious threat may not be the malicious file itself, but a network resource, which is at least:

[0217] The source of the attack on the computing device (e.g., DDoS attack, exploit deployment, etc.);

[0218] The source of the malicious file's distribution on the computing device (e.g., when using malware downloaders, etc.).

[0219] In other words, the malicious file identification module 750 can be configured to determine that malicious activity has occurred on the computing device based on an analysis of the behavioral attributes of the determined operating system file 701. The malicious file identification module 750 can also determine the source of the determined malicious activity, which can include some network resource (a site from which the malicious file was downloaded, an IP address that conducted a DDoS attack, etc.).

[0220] Thus, the disclosed system is configured to resist and hinder various targeted attacks (APT - Advanced Persistent Threat).

[0221] Figure 8 A flow chart of a method of detecting a source of malicious activity according to aspects of the present invention is shown.

[0222] The flowchart of the method for detecting the source of malicious activity includes step 810, in which the attribute behavior determination module 710 determines the behavior attributes of the file running in the operating system based on the behavior log. In one aspect, such an action log can be obtained based on an analysis of the operation of the operating system. In step 820, the mapping generation module 720 generates a mapping of the file 701 in the probability space based on the behavior attributes of the file 701 determined in step 810 using the trained mapping model 121. In step 830, the search module 730 searches for malicious file families in the mapping database 131 based on the mapping of the file 701 generated in step 820.

[0223] In step 840, the file characteristic determination module 740 identifies characteristics of files corresponding to the malicious file family found in the mapping database 131 in step 830. In step 850, the malicious file identification module 750 determines malicious files that may be the source of malicious activity in the operating system. The identified malicious files may have corresponding characteristics of the files identified in step 840. In step 860, the retraining module 760 retrains the mapping model 121 so that the generated mapping of the file 701 is more similar to the mapping of at least one file from a given malicious file family than before the mapping model 121 was retrained.

[0224] Fig. 9 1 is a block diagram illustrating a computer system 20 on which aspects of the system and method for detecting malicious files may be implemented according to an exemplary aspect. The computer system 20 may represent Figure 1 A computer system 100, Figure 4 Computer system 400 and Figure 7The computer system 700 may be in the form of multiple computing devices or a single computing device, such as a desktop computer, a laptop computer, a handheld computer, a mobile computing device, a smart phone, a tablet computer, a server, a mainframe, an embedded device, and other forms of computing devices.

[0225] As shown, computer system 20 includes a central processing unit (CPU) 21, a system memory 22, and a system bus 23 connecting various system components, including a memory associated with the central processing unit 21. The system bus 23 may include a bus memory or a bus memory controller, a peripheral bus, and a local bus capable of interacting with any other bus architecture. Examples of buses may include PCI, ISA, PCI-Express, HyperTransport, and the like. TM (HyperTransport TM ), unlimited bandwidth TM (InfiniBand TM ), Serial ATA, I2C, and other suitable interconnections. The central processing unit 21 (also referred to as a processor) may include a single group or multiple groups of processors having a single core or multiple cores. The processor 21 may execute one or more computer executable codes that implement the technology of the present invention. The system memory 22 may be any memory for storing data used herein and / or computer programs executable by the processor 21. The system memory 22 may include volatile memory (such as random access memory (Random Access Memory, RAM) 25) and non-volatile memory (such as read-only memory (Read-Only Memory, ROM) 24, flash memory, etc.) or any combination thereof. The basic input / output system (Basic Input / Output System, BIOS) 26 may store basic programs for transmitting information between elements of the computer system 20, such as those basic programs when the operating system is loaded using ROM 24.

[0226] The computer system 20 may include one or more storage devices, such as one or more removable storage devices 27, one or more non-removable storage devices 28, or a combination thereof. The one or more removable storage devices 27 and the one or more non-removable storage devices 28 are connected to the system bus 23 via a memory interface 32. In one aspect, the storage devices and corresponding computer-readable storage media are power-independent modules for storing computer instructions, data structures, program modules, and other data for the computer system 20. The system memory 22, the removable storage devices 27, and the non-removable storage devices 28 may use a variety of computer-readable storage media. Examples of computer-readable storage media include: machine memory, such as cache, SRAM, DRAM, zero-capacitor RAM, two-transistor RAM, eDRAM, EDO RAM, DDR RAM, EEPROM, NRAM, RRAM, SONOS, PRAM; flash memory or other storage technology, such as in a solid state drive (SSD) or a flash drive; magnetic cassettes, tapes, and magnetic disk storage, such as in a hard drive or a floppy disk drive; optical storage, such as in a compact disk (CD-ROM) or a digital versatile disk (DVD); and any other medium that can be used to store the desired data and can be accessed by the computer system 20.

[0227] The system memory 22, removable storage device 27 and non-removable storage device 28 of computer system 20 can be used to store operating system 35, additional program applications 37, other program modules 38 and program data 39. Computer system 20 may include a peripheral interface 46 for transmitting data from input device 40, such as a keyboard, mouse, light pen, game controller, voice input device, touch input device, or other peripheral devices, such as a printer or scanner via one or more I / O ports, such as a serial port, parallel port, Universal Serial Bus (USB), or other peripheral interfaces. Display device 47 (such as one or more monitors, projectors or integrated displays) can also be connected to system bus 23 via output interface 48 (such as a video adapter). In addition to display device 47, computer system 20 can also be equipped with other peripheral output devices (not shown), such as speakers and other audio-visual devices.

[0228] The computer system 20 can work in a network environment using a network connection with one or more remote computers 49. The one or more remote computers 49 can be local computer workstations or servers, which include most or all of the elements described above when describing the properties of the computer system 20. Other devices can also be present in the computer network, such as but not limited to routers, websites, peer devices or other network nodes. The computer system 20 may include one or more network interfaces 51 or network adapters for communicating with the remote computer 49 via one or more networks, such as a local area computer network (LAN) 50, a wide area computer network (WAN), an intranet, and the Internet. Examples of the network interface 51 may include an Ethernet interface, a frame relay interface, a synchronous optical network (SONET) interface, and a wireless interface.

[0229] Various aspects of the present invention may be systems, methods and / or computer program products. The computer program products may include one or more computer-readable storage media having computer-readable program instructions for causing a processor to perform various aspects of the present invention.

[0230] Computer readable storage medium can be a tangible device, and this tangible device can keep and store the program code in the form of instruction or data structure, and this program code can be accessed by the processor (such as computer system 20) of computing device.Computer readable storage medium can be electronic storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or its any suitable combination.By way of example, this type of computer readable storage medium can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), portable optical disc read-only memory (CD-ROM), digital versatile disc (DVD), flash memory, hard disk, portable computer disk, memory stick, floppy disk or even mechanical coding device, such as punch card or convex structure in the groove of recording instruction thereon.As used in this article, computer readable storage medium should not be regarded as temporary signal itself, temporary signal such as radio wave or other free propagation electromagnetic wave, electromagnetic wave propagated by waveguide or transmission medium or electric signal transmitted by wire.

[0231] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network interface in each computing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing device.

[0232] The computer-readable program instructions for performing the operation of the present invention can be assembly instructions, instruction set architecture (Instruction-Set-Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​and traditional programmed programming languages).Computer-readable program instructions can be performed completely on the user's computer, partly on the user's computer, partly on the user's computer and partly on a remote computer or completely on a remote computer or server as an independent software package.In the latter scenario, the remote computer can be connected to the user's computer through any type of network (including LAN or WAN), or can be connected to an external computer (for example, through the Internet).In some embodiments, electronic circuit (including for example programmable logic circuit, field programmable gate array (FPGA) or programmable logic array (Programmable Logic Array, PLA)) can perform computer-readable program instructions by utilizing the state information of computer-readable program instructions to personalize the electronic circuit, thereby performing various aspects of the present invention.

[0233] In various aspects, the system and method described in the present invention can be processed according to modules. As used herein, the term "module" refers to, for example, the arrangement of the components of the equipment, components, or hardware (such as by application-specific integrated circuit (ASIC) or FPGA) of the real world, or refers to a combination of hardware and software, such as by a microprocessor system and an instruction set (the instruction set converts the microprocessor system into a special-purpose device when executed) that implements the module function. A module can also be implemented as a combination of two modules, wherein certain functions are promoted by hardware alone, and other functions are promoted by a combination of hardware and software. In some implementations, at least a portion of a module (and in some cases, all of a module) can be run on a processor of a computer system. Therefore, each module can be implemented with various suitable configurations, and should not be limited to any specific implementation illustrated herein.

[0234] For the sake of clarity, not all routine features of various aspects are disclosed herein. It should be appreciated that in the development of any practical implementation of the present invention, many implementation-specific decisions must be made in order to achieve the developer's specific goals, and these specific goals will vary for different implementations and different developers. It should be understood that such development efforts can be complex and time-consuming, but are still a routine task of engineering for those of ordinary skill in the art who understand the advantages of the present invention.

[0235] In addition, it should be understood that the words or terms used herein are for the purpose of description rather than limitation, so the terms or terms of this specification should be interpreted by those skilled in the art according to the teachings and guidance provided herein in combination with the knowledge of (one or more) related art(s). In addition, it is not intended to attribute any term in this specification or claims to an uncommon or special meaning unless explicitly stated as such.

[0236] The various aspects disclosed herein include the present and future known equivalents of the known modules mentioned herein in an illustrative manner. In addition, although various aspects and applications have been shown and described, it will be apparent to those skilled in the art who understand the advantages of the present invention that more modifications than those mentioned above are feasible without departing from the inventive concept disclosed herein.

Claims

1. A method for detecting malicious files, the method comprising: training a mapping model for mapping files in a probability space representing one or more families of malicious files; determining a plurality of characteristics of the analyzed file based on a set of rules; generating, using the mapping model, a mapping of the analyzed file in the probability space based on the determined plurality of characteristics; searching a first database using the generated mapping of the analyzed file, wherein the first database stores mappings associated with one or more malicious file families; Determining whether the analyzed file is associated with a malicious file family; In response to determining that the analyzed file is associated with the malicious file family, identifying characteristics of files corresponding to the malicious file family, determining one or more malicious files as a source of malicious activity, and selecting one or more malware detection methods from a second database, wherein the second database stores a plurality of malware detection methods; and The associated malicious file families are detected using the selected one or more malware detection methods and the identified sources of malicious activity.

2. The method of claim 1, wherein: The probability space comprises a multi-dimensional space, and the probability space provides a probability distribution for the plurality of characteristics associated with the analyzed file.

3. The method of claim 2, wherein: The generated map includes an aggregation of a plurality of parameters describing faces associated with the analyzed file, and the faces are located in the probability space.

4. The method of claim 2, wherein: The probability space comprises a multidimensional space in which a pair of one of the multiple characteristics of the analyzed file and one of the multiple attributes associated with the behavior of the analyzed file is matched with the following probability: the behavior described by the one of the multiple attributes is unique to the analyzed file having the one of the multiple characteristics.

5. The method of claim 1, wherein: The first database includes a map of files associated with a particular malicious file family or a summary map of a plurality of files associated with a particular malicious file family.

6. The method of claim 5, wherein: Searching the first database further includes identifying an association between the analyzed file and one of the plurality of files associated with the malicious file family, wherein a similarity between a mapping of the analyzed file and a mapping of the one of the files associated with the malicious file family exceeds a predetermined threshold.

7. The method of claim 1 , further comprising retraining the mapping model to increase a corresponding similarity between a mapping of a corresponding analyzed file and a mapping of one of the multiple files associated with the malicious file family within the retrained mapping model as compared to a similarity within the mapping model before performing the retraining between a mapping of the analyzed file and a mapping of one of the multiple files associated with the malicious file family.

8. A system for detecting malicious files, the system comprising: A hardware processor and a memory storing instructions executed by the hardware processor, wherein the hardware processor and the memory are configured to: training a mapping model for mapping files in a probability space representing one or more families of malicious files; determining a plurality of characteristics of the analyzed file based on a set of rules; generating a mapping of the analyzed file in the probability space based on the determined plurality of characteristics; searching a first database using the generated mapping of the analyzed file, wherein the first database stores mappings associated with one or more malicious file families; Determining whether the analyzed file is associated with a malicious file family; In response to determining that the analyzed file is associated with the malicious file family, identifying characteristics of files corresponding to the malicious file family, determining one or more malicious files as a source of malicious activity, and selecting one or more malware detection methods from a second database, wherein the second database stores a plurality of malware detection methods; and The associated malicious file families are detected using the selected one or more malware detection methods and the identified sources of malicious activity.

9. The system of claim 8, wherein: The probability space comprises a multi-dimensional space, and the probability space provides a probability distribution for the plurality of characteristics associated with the analyzed file.

10. The system of claim 9, wherein: The generated map includes an aggregation of a plurality of parameters describing faces associated with the analyzed file, and the faces are located in the probability space.

11. The system of claim 9, wherein: The probability space comprises a multidimensional space in which a pair of one of the multiple characteristics of the analyzed file and one of the multiple attributes associated with the behavior of the analyzed file is matched with the following probability: the behavior described by the one of the multiple attributes is unique to the analyzed file having the one of the multiple characteristics.

12. The system of claim 8, wherein: The first database includes a map of files associated with a particular malicious file family or a summary map of a plurality of files associated with a particular malicious file family.

13. The system of claim 12, wherein: The hardware processor configured to search the first database is further configured to identify an association between the analyzed file and one of the plurality of files associated with the malicious file family, wherein a similarity between a mapping of the analyzed file and a mapping of the one of the files associated with the malicious file family exceeds a predetermined threshold.

14. The system of claim 8, wherein: The hardware processor is also configured to retrain the mapping model to increase a corresponding similarity within the retrained mapping model between a mapping of a corresponding analyzed file and a mapping of one of the multiple files associated with the malicious file family as compared to a similarity within the mapping model before performing the retraining between a mapping of the analyzed file and a mapping of one of the multiple files associated with the malicious file family.

15. A non-transitory computer-readable medium having stored thereon computer-executable instructions for detecting malicious files, the computer-executable instructions comprising instructions for: training a mapping model for mapping files in a probability space representing one or more families of malicious files; determining a plurality of characteristics of the analyzed file based on a set of rules; generating a mapping of the analyzed file in the probability space based on the determined plurality of characteristics; searching a first database using the generated mapping of the analyzed file, wherein the first database stores mappings associated with one or more malicious file families; Determining whether the analyzed file is associated with a malicious file family; In response to determining that the analyzed file is associated with the malicious file family, identifying characteristics of files corresponding to the malicious file family, determining one or more malicious files as a source of malicious activity, and selecting one or more malware detection methods from a second database, wherein the second database stores a plurality of malware detection methods, and The associated malicious file families are detected using the selected one or more malware detection methods and the identified sources of malicious activity.

16. The non-transitory computer readable medium of claim 15, wherein: The probability space comprises a multi-dimensional space, and the probability space provides a probability distribution for the plurality of characteristics associated with the analyzed file.

17. The non-transitory computer readable medium of claim 16, wherein: The generated map includes an aggregation of a plurality of parameters describing faces associated with the analyzed file, and the faces are located in the probability space.

18. The non-transitory computer readable medium of claim 16, wherein: The probability space comprises a multidimensional space in which a pair of one of the multiple characteristics of the analyzed file and one of the multiple attributes associated with the behavior of the analyzed file is matched with the following probability: the behavior described by the one of the multiple attributes is unique to the analyzed file having the one of the multiple characteristics.

19. The non-transitory computer readable medium of claim 15, wherein: Searching the first database further includes identifying an association between the analyzed file and one of the plurality of files associated with the malicious file family, wherein a similarity between a mapping of the analyzed file and a mapping of the one of the files associated with the malicious file family exceeds a predetermined threshold.

20. The non-transitory computer readable medium of claim 15, wherein: The computer-executable instructions also include instructions for retraining the mapping model to increase a corresponding similarity within the retrained mapping model between a mapping of a corresponding analyzed file and a mapping of one of the multiple files associated with the malicious file family as compared to a similarity within the mapping model before performing the retraining between a mapping of the analyzed file and a mapping of one of the multiple files associated with the malicious file family.

Images