Malware family truth inference method and system and medium

By extracting malware family markers based on location-first search method and improving the accuracy of malware family identification through marking situation division, the problem of extracting family markers and inferring the truth of sample families in the prior art is solved, and higher accuracy is achieved and reliability.

CN118886010AActive Publication Date: 2024-11-01ELECTRIC POWER RES INST OF STATE GRID ZHEJIANG ELECTRIC POWER COMAPNY
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202410945225.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-01
Estimated Expiration
2044-07-15

AI Technical Summary

Technical Problem

The prior art is difficult to accurately extract family markers in malware family markers, and bias is prone to inferring the truth of the sample family in inconsistent family markers.

Method used

The location-first search method is used to extract family marks from malware tags, and only the strong mark samples are inferred through the labeling situation division to improve the accuracy of family marks.

Benefits of technology

Improves the accuracy of family mark extraction and the accuracy of malware family identification, ensuring accurate inference results in case of inconsistent markup.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118886010A_ABST
    Figure CN118886010A_ABST
Patent Text Reader

Abstract

The invention relates to a malicious software family truth inference method and system and a medium, and provides a malicious software family truth inference method aiming at the problem of inaccurate marking of the existing method, and the method comprises the following steps: distributing a to-be-tested sample to a plurality of antivirus engines for scanning, and obtaining and collecting malicious software labels through the reporting of the antivirus engines; malicious software tags are decomposed into tag sequences to realize tagging of the malicious software tags; the type mark and the platform mark are regarded as positioners, and the positioners are determined through the mark sequence obtained through searching; searching and outputting family marks according to the position distribution of the locator; according to the extracted family markers, carrying out marker situation division to obtain a strong marker sample, a weak marker sample and an unmarked sample; and an n-vote voting method is adopted to output a true phase inference result of the strongly labeled sample. The position-first search algorithm has higher family mark extraction accuracy; truth inference is only carried out on the strong labeled sample, so that the accuracy of malicious software family identification is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of malware, and in particular to a method, system and medium for inferring the truth of a malware family. Background Art

[0002] Online malware analysis platforms, such as VirusTotal, aggregate well-known antivirus engines worldwide and provide a rich API to facilitate uploading samples and downloading antivirus engine scanning results. When an antivirus engine detects a sample as malware, it reports a malware label to detail the detection result. The malware label consists of tags such as family, type, and platform. The above properties are widely used by the security community to annotate and build malware benchmark data, which is called crowdsourcing malware annotation.

[0003] Crowdsourcing malware annotation includes two main tasks: category tag extraction and truth inference. Category tag extraction realizes the association between tags in malware labels and corresponding tag categories, and truth inference realizes inferring the unknown truth of samples from the extracted tags.

[0004] The first challenge in applying the above paradigm to malware family annotation is how to accurately extract family tags from the confusing malware labels. CARO proposed a virus naming convention in 1991 in an attempt to bring consistency to the naming of malware tags. Although most antivirus engines today use similar components (type, platform, and family) suggested by CARO to construct malware labels, each antivirus engine has its own unique tag vocabulary and naming pattern. The difficulty in extracting category tags is that: (1) the naming patterns of antivirus engines cannot be enumerated by regular expressions. For example, although the malware labels elf:mirai-fug and win32:adware-gen are structurally the same, adware is a type tag and mirai is a family tag; (2) the tag vocabulary of antivirus engines is always changing, and most engines do not disclose their custom tags, especially family tags, which makes it very difficult to build an effective machine learning-based tag classification model. Some methods have tried to solve the problem of extracting family tags, such as Euphony and AVClass, but the accuracy of these methods is still low.

[0005] The second challenge of applying the above annotation paradigm to malware family annotation is how to accurately infer the unknown family truth of a sample from inconsistent family markers. Antivirus engines often have different opinions on which family a sample belongs to. Existing works, such as Euphony and AVClass, usually assume that the quality of antivirus engines is equally reliable and rely heavily on n-vote voting schemes to infer the family truth of samples. This scheme is prone to biased inference results when there are family markers with similar votes. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides a method, system and medium for inferring the truth of a malware family, and innovatively proposes: a family truth labeling method based on a position-first search method and a method based on the division of labeling situations. The former effectively solves the problem of extracting family labels from chaotic malware; the latter only collects samples with no dispute over the family truth through the division of labeling situations, so as to improve the quality of family labeling.

[0007] A method for inferring the truth of a malware family includes the following steps:

[0008] S1, distribute the samples to be tested to multiple antivirus engines for scanning, and obtain and collect malware labels through antivirus engine reports;

[0009] S2, extracts family tags from the obtained malware labels using a position-first search method, including:

[0010] Malware labels are tokenized by decomposing them into tag sequences; type tags and platform tags are regarded as locators, and the locators are determined by searching the obtained tag sequences; the output family tags are searched according to the position distribution of the locators;

[0011] S3, divide the labeling situation according to the extracted family labels to obtain strong labeling samples, weak labeling samples and unlabeling samples; use the n-vote voting method to output the truth inference results of the strong labeling samples, and obtain the truth inference results of the sample family.

[0012] Furthermore, in step S2, the determined locators form an initial locator tag vocabulary.

[0013] Furthermore, the specific process of searching and outputting family tags according to the position distribution of the locator is as follows: find the locator tag in the tag sequence according to the locator tag vocabulary. If the position distribution of the locator is continuous, search at both ends of the locator position; if the position distribution of the locator is discontinuous, search in the middle of the locator position, and return the tag with the smallest position number in the search area as the family tag.

[0014] Furthermore, the specific process of step S3 is: extracting sample x i The family marker is The number of occurrences of family marker k is N k , I(.) is a symbolic function; all Arrange in descending order, denoted as According to formula (1), the marking situation is divided:

[0015]

[0016] Where: J is the total number of antivirus engines; j represents any antivirus engine; represents the kth family marker of the i-th sample; m is the first hyperparameter; K is the total number of family markers.

[0017] Furthermore, the specific process of step S4 is: inferring that a sample belongs to family k if and only if greater than or equal to n antivirus engines believe that the sample belongs to family k and k has the largest number of votes.

[0018] Furthermore, the inferred family markers are:

[0019]

[0020] in, This is an invalid inference, indicating that family annotation cannot be performed for this sample.

[0021] A malware family truth inference system, comprising:

[0022] The crowdsourcing module distributes the samples to be tested to multiple antivirus engines for scanning, and obtains and collects malware labels through antivirus engine reports;

[0023] The family tag extraction module extracts family tags from the obtained malware tags using a position-first search method, including: tokenizing malware tags by decomposing them into tag sequences; treating type tags and platform tags as locators, and determining the locators by searching the obtained tag sequences; and searching and outputting family tags according to the position distribution of the locators;

[0024] The family truth inference module divides the labeling situations according to the extracted family labels to obtain strongly-tagged samples, weakly-tagged samples and null-tagged samples; it uses the n-vote voting scheme to output the truth inference results of the strongly-tagged samples and obtain the sample family truth inference results.

[0025] A malware family truth inference system includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned malware family truth inference method.

[0026] A computer-readable storage medium stores a program, which, when executed by a processor, implements the above-mentioned malware family truth inference method.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] The present invention provides a malware family truth inference method, system and medium, which obtain the relationship between family tags and locator tags (type tags and platform tags) by analyzing a large amount of malware label data. The position priority search algorithm proposed by this method has a higher family tag extraction accuracy. At the same time, the application divides the labeling situations and only performs truth inference on strongly labeled samples, thereby further ensuring the accuracy of malware family identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a flow chart of Example 1.

[0030] Figure 2 This is a schematic diagram of family marker extraction in Example 1.

[0031] Figure 3 This is a schematic diagram of family truth inference in Example 1.

[0032] Figure 4 This is the architecture diagram of Example 2. DETAILED DESCRIPTION

[0033] A preferred embodiment of the present invention is described below in conjunction with the accompanying drawings to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0034] In the drawings, components with the same structure are indicated by the same numerical reference numerals, and components with similar structures or functions are indicated by similar numerical reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of the components is appropriately exaggerated in some places in the drawings.

[0035] Example 1

[0036] This embodiment is a method for inferring the truth of a malware family. Figures 1 to 3 As shown, the following steps are included:

[0037] The uploaded samples enter the database module for query and archiving; the data sample calculates the hash value and queries it in the database; if it does not exist, crowdsourcing, family mark extraction and family truth inference are performed to update the database; if it exists, the annotation result is directly returned.

[0038] Distribute the samples to be tested to multiple antivirus engines for scanning, and obtain and collect malware labels through antivirus engine reports;

[0039] The position-first search method is used to extract family tags from the obtained malware labels, including:

[0040] Malware label tokenization is achieved by decomposing malware labels into label sequences (for example, the malware label of trojan.ransom.gandcrab is decomposed into a label sequence of (trojan, ransom, gandcrab)); type labels and platform labels are regarded as locators, and the locators are determined by searching the obtained label sequence, and the determined locators form an initial locator label vocabulary; the family label is searched and output according to the position distribution of the locator: the locator label is searched in the label sequence according to the locator label vocabulary, if the position distribution of the locator is continuous, the search is performed at both ends of the locator position; if the position distribution of the locator is discontinuous, the search is performed in the middle of the locator position, and the label with the smallest position number in the search area is returned as the family label.

[0041] Tokenize the malware label l to get the token sequence t = (t1, t2, …, t m ); according to the locator token vocabulary v (loc) Determine the position of the locator marker in the marker sequence; search and output the family marker according to the position distribution of the locator marker. Specifically: if the position distribution of the locator is continuous, search at both ends of the locator position; if the position distribution of the locator is discontinuous, search in the middle of the locator position. Then return the marker with the smallest position number in the search area as the family marker. The description of the position priority search method is shown in Table 1.

[0042] Table 1

[0043]

[0044]

[0045] According to the extracted family markers, the labeling situation is divided to obtain strong labeling samples, weak labeling samples and unlabeled samples: Extract sample x i The family marker is The number of occurrences of family marker k is N k , I(.) is a symbolic function; all Arrange in descending order, denoted as According to formula (1), the marking situation is divided:

[0046]

[0047] Where: J is the total number of antivirus engines; j represents any antivirus engine; represents the kth family marker of the i-th sample; m is the first hyperparameter; K is the total number of family markers.

[0048] The n-vote voting scheme is used to output the truth inference results of strongly labeled samples, and the truth inference results of sample families are obtained: a sample is inferred to belong to family k if and only if n or more antivirus engines believe that the sample belongs to family k and k has the most votes. The inferred family label is:

[0049]

[0050] in, It is an invalid inference, indicating that it is impossible to perform family annotation for the sample. In this embodiment, m=2, n=2.

[0051] Example 2

[0052] like Figure 4 As shown, it is an example architecture diagram of a malware family truth inference system of a preferred embodiment of the present invention. The whole system adopts a browser / server architecture, and the core functions are implemented on the server side.

[0053] The malware family truth inference system of this embodiment includes:

[0054] A database module, configured to store crowdsourcing results of samples, family marker extraction results, and family truth inference results;

[0055] The crowdsourcing module distributes the samples to be tested to multiple antivirus engines for scanning, and obtains and collects malware labels through antivirus engine reports;

[0056] The family tag extraction module extracts family tags from the obtained malware tags using a position-first search method, including: tokenizing malware tags by decomposing them into tag sequences; treating type tags and platform tags as locators, and determining the locators by searching the obtained tag sequences; and searching and outputting family tags according to the position distribution of the locators;

[0057] The family truth inference module divides the labeling situations according to the extracted family labels to obtain strongly labeled samples, weakly labeled samples and unlabeled samples; it uses the n-vote voting scheme to output the truth inference results of the strongly labeled samples and obtain the sample family truth inference results.

[0058] The browser side of the malware family truth inference system based on location-first search and marked situation division is configured to upload data samples and present the annotated results to users. Users can upload malware and view the family truth inference results output by the annotation system by simply using any electronic device with a browser function to access the Internet.

[0059] Example 3

[0060] A malware family truth inference system, characterized in that it includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the malware family truth inference method described in Example 1.

[0061] Example 4

[0062] A computer-readable storage medium, characterized in that a program is stored thereon, and when the program is executed by a processor, the malware family truth inference method described in embodiment 1 is implemented.

[0063] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A malware family truth inference method, characterized in that: The steps include: S1, distribute the samples to be tested to multiple antivirus engines for scanning, and obtain and collect malware labels through antivirus engine reports; S2, extracts family tags from the obtained malware labels using a position-first search method, including: Malware labels are tokenized by decomposing them into tag sequences; type tags and platform tags are regarded as locators, and the locators are determined by searching the obtained tag sequences; the output family tags are searched according to the position distribution of the locators; S3, divide the labeling situation according to the extracted family labels to obtain strong labeling samples, weak labeling samples and unlabeling samples; use the n-vote voting method to output the truth inference results of the strong labeling samples, and obtain the truth inference results of the sample family.

2. The malware family truth inference method according to claim 1, characterized in that: In step S2, the determined locators form an initial locator tag vocabulary.

3. The malware family truth inference method according to claim 2, characterized in that: The specific process of searching and outputting family tags according to the position distribution of the locator is as follows: find the locator tag in the tag sequence according to the locator tag vocabulary. If the position distribution of the locator is continuous, search at both ends of the locator position; if the position distribution of the locator is discontinuous, search in the middle of the locator position; then return the tag with the smallest position number in the search area as the family tag.

4. The malware family truth inference method according to claim 1, characterized in that: The specific process of marking case division is: extract sample x i The family marker is The number of occurrences of family marker k is N k , I(.) is a symbolic function; all Arrange in descending order, denoted as According to formula (1), the marking situation is divided: Where: J is the total number of antivirus engines; j represents any antivirus engine; represents the kth family marker of the i-th sample; m is the first hyperparameter; K is the total number of family markers.

5. The malware family truth inference method according to claim 4 is characterized in that: The specific process of using the n-vote voting method to output the truth inference result of strongly labeled samples is as follows: a sample is inferred to belong to family k if and only if greater than or equal to n antivirus engines believe that the sample belongs to family k and k has the most votes.

6. The malware family truth inference method according to claim 5, characterized in that: The inferred family markers are: in, This is an invalid inference, indicating that family annotation cannot be performed for this sample.

7. A malware family truth inference system, characterized in that: include: The crowdsourcing module distributes the samples to be tested to multiple antivirus engines for scanning, and obtains and collects malware labels through antivirus engine reports; The family tag extraction module extracts family tags from the obtained malware tags using a position-first search method, including: tokenizing malware tags by decomposing them into tag sequences; treating type tags and platform tags as locators, and determining the locators by searching the obtained tag sequences; and searching and outputting family tags according to the position distribution of the locators; The family truth inference module divides the labeling situations according to the extracted family labels to obtain strongly labeled samples, weakly labeled samples and unlabeled samples; it uses the n-vote voting scheme to output the truth inference results of the strongly labeled samples and obtain the sample family truth inference results.

8. A malware family truth inference system, characterized in that: It comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the malware family truth inference method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the malware family truth inference method described in any one of claims 1-6 is implemented.

Citation Information

Patent Citations

  • Malicious software family inference method and system

    CN113468532A

  • Malware family label correction method and device based on hybrid analysis

    CN114936366A

  • Malicious software detection method based on comparative learning

    CN116541838A

  • Malicious software analysis method and device, storage medium and equipment

    CN117171738A

  • Malware families identification based upon hierarchical clustering

    US11886586B1