Android malicious software detection method based on fine-grained semantic feature extraction
Through fine-grained semantic feature extraction methods, including decompilation, API feature extraction, blacklist filtering and vocabulary processing, it solves the problems of high model aging rate and weak anti-obfuscation ability in Android malware detection technology, and achieves higher detection Accuracy and anti-aliasing capabilities.
Patent Information
- Application Number
- CN202411475632.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2025-01-24
AI Technical Summary
The existing Android malware detection technology has the problems of high aging rate and weak anti-confusion ability, which leads to low detection accuracy.
Adopt the Android malware detection method based on fine granular semantic characteristics, obtain the source code through counter -compilation, use Androguard to extract API features, perform preparation and blacklist screening, build a black -name single frequency table and vocabulary blacklist, delete the features that belong to the black list The vocabulary is embedded in the Word2VEC model, the fusion vector is used to obtain the API semantic characteristics, and the detection model is finally entered for detection.
Enhance the model's anti -confusion ability, reduce the model aging rate and aging re -training cost, and improve the accuracy of malware detection.
Smart Images
Figure CN119357961A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of Android software detection, and in particular relates to an Android malware detection method based on fine-grained semantic feature extraction. Background Art
[0002] Android is a free and open source operating system based on Linux, developed by Google and the Open Handset Alliance in 2007. It is widely used in mobile devices. The first Android smartphone was released in October 2008. Subsequently, Android gradually expanded to tablets and other fields, such as digital cameras, TVs, game consoles, smart watches, etc. According to the data provided by Tencent Mobile Security Lab's Mobile Security Report, Android has occupied 70.97% of the mobile operating system market share in 2019. At the same time, it also makes the Android system increasingly mature in the open platform.
[0003] It can be seen that in-depth research on the security issues of mobile applications is of great significance to the sound development of the smartphone ecosystem. The rapid development of mobile malware detection technology in recent years has made it quickly become a hot issue in the field of software security in academia, and a large number of excellent related research results have been published in top international conferences and journals. In the past, researchers used solutions for API semantic extraction based on the feature transformation process of NLP, from using heat encoding to proposing fun2vec based on word2vec. For the dimensionality reduction and screening process of the data set, the main idea of the previous solution is still the whitelist thinking. The dimensionality reduction process and screening process are greatly affected by many factors such as Android version and software protection. Therefore, its model aging rate is high and its anti-obfuscation ability is weak. Summary of the invention
[0004] To solve the above problems, the present invention provides an Android malware detection method based on fine-grained semantic feature extraction, which is characterized by comprising the following steps:
[0005] S1. Decompile the Android software to be detected to obtain the corresponding source code file, use the open source framework Androguard to extract API features from the source code file, and obtain an API feature set;
[0006] S2. preprocessing the API feature set to obtain a preprocessed API feature set;
[0007] S3. Using a blacklist screening method to screen the preprocessed API feature set to obtain an API list;
[0008] S4. Using a vocabulary blacklist filtering method to process the API list to obtain an embedding list;
[0009] S5. Perform word embedding on the embedding list to obtain an API word embedding matrix; perform vector fusion on the API word embedding matrix to obtain API semantic features;
[0010] S6. Input the API semantic features into the trained Android malware detection model to obtain the Android malware detection results.
[0011] Furthermore, step S3 uses a blacklist screening method to screen the preprocessed API feature set to obtain an API list, including:
[0012] S31. The number of API features in the statistical preprocessing feature set is N, and the target API scale is M;
[0013] S32. If N>M, find all API features in the preprocessing feature set that appear in the pre-built blacklist frequency table and their positions in the table, and delete the top K API features in the table from the preprocessing feature set to obtain an API list; where K=NM.
[0014] Furthermore, the process of establishing the blacklist frequency table includes:
[0015] S311. Collecting Android software sample data, including a benign Android software sample set and a malicious Android software sample set; decompiling each Android software sample to obtain the corresponding source code file, and using the open source framework Androguard to extract API features from each source code file to obtain an API feature data set;
[0016] S312. Count the benign and malicious occurrences of each API feature in the API feature data set; the benign occurrence of the API feature is the total number of times the API feature appears in the benign Android software sample set, and the malicious occurrence of the API feature is the total number of times the API feature appears in the malicious Android software sample set;
[0017] S313. Calculate the benign frequency value and malicious frequency value of each API feature in the API feature data set, expressed as:
[0018]
[0019] Among them, API C Indicates the number of API features in the API feature dataset; represents the number of benign occurrences of the ith API feature in the API feature dataset, represents the benign frequency value of the ith API feature in the API feature dataset, represents the number of malicious occurrences of the i-th API feature in the API feature dataset, Represents the malicious frequency value of the i-th API feature in the API feature dataset;
[0020] S314. Combining all API features that appear only in the benign Android software sample set into a first set, and combining all API features that appear in both the benign Android software sample set and the malicious Android software sample set into a second set;
[0021] S315. Arrange all API features in the first set in descending order according to benign frequency values to obtain a first part of the blacklist frequency table;
[0022] S316. Calculate the frequency difference of each API feature in the second set, expressed as:
[0023]
[0024] F j represents the frequency difference of the jth API feature in the second set, represents the benign frequency value of the jth API feature in the second set, represents the malicious frequency value of the jth API feature in the second set;
[0025] S317. Arrange all API features in the second set in descending order according to the frequency difference to obtain a second blacklist frequency table; add the second blacklist frequency table to the end of the first blacklist frequency table to obtain a complete blacklist frequency table.
[0026] Further, step S4 uses a vocabulary blacklist filtering method to process the API list to obtain an embedding list including:
[0027] S41. Split each API feature in the API list into multiple words, and form a vocabulary list with all the words;
[0028] S42. Delete all words in the vocabulary list that appear in the pre-built vocabulary blacklist to obtain an embedding list.
[0029] Furthermore, the process of establishing a vocabulary blacklist includes:
[0030] S411. Collecting Android software sample data, including a benign Android software sample set and a malicious Android software sample set; decompiling each Android software sample to obtain the corresponding source code file, and using the open source framework Androguard to extract API features from each source code file to obtain an API feature data set;
[0031] S412. Split each API feature in the API feature data set, and count the number of occurrences of each word in the benign Android software set and the malicious Android software set; construct a benign Android software set vocabulary occurrence number set W A ={A1,A2,…,A U}, construct the occurrence count set W of malicious Android software vocabulary B = {B1, B2, …, B U}, where A u represents the number of times the u-th word appears in the benign Android software set, B u represents the number of times the u-th word appears in the malicious Android software collection, and U represents the number of words;
[0032] S413. Calculate the benign word frequency and malicious word frequency of each word, expressed as:
[0033]
[0034] Among them, Fwb u Indicates the benign word frequency of the u-th word, Fmb u represents the malicious word frequency of the u-th word;
[0035] S414. For each word, subtract the frequency of malicious words from its own frequency of benign words to obtain the difference frequency, sort all words in descending order according to the difference frequency, and extract the first 100 words to form a vocabulary blacklist;
[0036] S415. Extract words whose benign word frequency and malicious word frequency both exceed the threshold, and select some words from them to add to the vocabulary blacklist.
[0037] Furthermore, the pre-trained Word2Vec model is used to embed each word in the embedding list to obtain the API word embedding matrix.
[0038] Beneficial effects of the present invention:
[0039] The present invention provides an Android malware detection method based on fine-grained semantic feature extraction, which uses blacklist thinking to construct a blacklist frequency table and a vocabulary blacklist, deletes the features and vocabulary belonging to the blacklist, and uses the cleaned features to train a model; the method enhances the anti-confusion ability of the model, can reduce the model aging rate and reduce the model aging retraining cost; the method improves the accuracy of malware detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0042] This paper proposes an Android malware detection method based on fine-grained semantic feature extraction, such as Figure 1 As shown, the following steps are included:
[0043] S1. Decompile the Android software to be detected to obtain the corresponding source code file, use the open source framework Androguard to extract API features from the source code file, and obtain the API feature set.
[0044] Specifically, API is an important carrier for software operation and plays an important role in identifying malware. The present invention uses the open source framework Androguard to extract API features from source code files; API features include package names, class names, function names, and sub-references.
[0045] S2. Preprocess the API feature set to obtain a preprocessed API feature set.
[0046] S3. Use the blacklist screening method to screen the preprocessed API feature set to obtain the API list.
[0047] Specifically, step S3 uses a blacklist screening method to screen the preprocessed API feature set to obtain an API list, including:
[0048] S31. The number of API features in the statistical preprocessing feature set is N, and the target API scale is M;
[0049] S32. If N>M, find all API features in the preprocessing feature set that appear in the pre-built blacklist frequency table and their positions in the table, and delete the top K API features in the table from the preprocessing feature set to obtain an API list; where K=NM.
[0050] Specifically, the process of establishing the blacklist frequency table includes:
[0051] S311. Collect Android software sample data, which includes a benign Android software sample set and a malicious Android software sample set; decompile each Android software sample to obtain the corresponding source code file, use the open source framework Androguard to extract API features from each source code file, and obtain an API feature data set; the API feature data set includes API features extracted from all source code files. The same API feature may be extracted from different source code files, so the API feature data set contains repeated API features, and each API feature may contain multiple.
[0052] Preferably, the Android software sample data includes 1,700 benign Android software samples and 1,700 malicious Android software samples; the Android software sample data can be extracted from the Android malware dataset (CICMalDroid 2020).
[0053] Specifically, each extracted APK feature is saved as a csv file; the APK feature data set also needs to be cleaned up, because among the originally extracted APK features, some data sets contain software obfuscation reinforcement solutions, so these APKs need to be cleaned up. For example, if a large number of API names are only one letter, they will be regarded as obfuscated APKs and removed.
[0054] S312. Count the benign and malicious occurrences of each API feature in the API feature data set; the benign occurrence count of the API feature is the total number of times the API feature appears in the benign Android software sample set, and the malicious occurrence count of the API feature is the total number of times the API feature appears in the malicious Android software sample set;
[0055] S313. Calculate the benign frequency value and malicious frequency value of each API feature in the API feature data set, expressed as:
[0056]
[0057]
[0058] Among them, API C Indicates the number of API features in the API feature dataset; represents the number of benign occurrences of the ith API feature in the API feature dataset, represents the benign frequency value of the ith API feature in the API feature dataset, represents the number of malicious occurrences of the i-th API feature in the API feature dataset, Represents the malicious frequency value of the i-th API feature in the API feature dataset;
[0059] S314. Combining all API features that appear only in the benign Android software sample set into a first set, and combining all API features that appear in both the benign Android software sample set and the malicious Android software sample set into a second set;
[0060] S315. Arrange all API features in the first set in descending order according to benign frequency values to obtain a first part of the blacklist frequency table;
[0061] S316. Calculate the frequency difference of each API feature in the second set, expressed as:
[0062]
[0063] F j represents the frequency difference of the jth API feature in the second set, represents the benign frequency value of the jth API feature in the second set, represents the malicious frequency value of the jth API feature in the second set;
[0064] S317. Arrange all API features in the second set in descending order according to the frequency difference to obtain a second blacklist frequency table; add the second blacklist frequency table to the end of the first blacklist frequency table to obtain a complete blacklist frequency table.
[0065] S4. Use the vocabulary blacklist filtering method to process the API list to obtain the embedding list.
[0066] Specifically, step S4 uses a vocabulary blacklist filtering method to process the API list to obtain an embedding list including:
[0067] S41. Split each API feature in the API list into multiple words, and form a vocabulary list with all the words;
[0068] Specifically, the following rules are used for word segmentation:
[0069] Replace special characters in API features with spaces, such as ' / ', '->', etc.;
[0070] Replace the return value methods in API features with the full English names of the actual data types, such as 'v' for 'void' and 'Z' for 'boolean'.
[0071] S42. Delete all words in the vocabulary list that appear in the pre-built vocabulary blacklist to obtain an embedding list.
[0072] Specifically, the process of establishing a vocabulary blacklist includes:
[0073] S411. Collect Android software sample data, which includes a benign Android software sample set and a malicious Android software sample set; decompile each Android software sample to obtain the corresponding source code file, and use the open source framework Androguard to extract API features from each source code file to obtain an API feature data set; the data set used here is the same as step S311.
[0074] S412. Split each API feature in the API feature data set, and count the number of occurrences of each word in the benign Android software set and the malicious Android software set; construct a benign Android software set vocabulary occurrence number set W A ={A1,A2,…,A U}, construct the occurrence count set W of malicious Android software vocabulary B = {B1, B2, …, B U}, where A u represents the number of times the u-th word appears in the benign Android software set, B u It represents the number of times the u-th word appears in the malicious Android software collection, and U represents the number of words; the number of words here does not include repeated words.
[0075] S413. Calculate the benign word frequency and malicious word frequency of each word, expressed as:
[0076]
[0077] Among them, Fwb u Indicates the benign word frequency of the u-th word, Fmb u represents the malicious word frequency of the u-th word;
[0078] S414. For each word, subtract the frequency of malicious words from its own frequency of benign words to obtain the difference frequency, sort all words in descending order according to the difference frequency, and extract the first 100 words to form a vocabulary blacklist;
[0079] S415. Extract words whose benign word frequency and malicious word frequency both exceed the threshold, and consider whether to add them to the vocabulary blacklist based on experience.
[0080] Specifically, the pre-trained Word2Vec model is used to embed each word in the embedding list to obtain the API word embedding matrix. The size of the API word embedding matrix is the number of words × embedding dimension.
[0081] S5. Perform word embedding on the embedding list to obtain the API word embedding matrix; perform vector fusion on the API word embedding matrix to obtain API semantic features.
[0082] Specifically, the API semantic features obtained by vector fusion of the API word embedding matrix include:
[0083] Each word embedding result in the API word embedding matrix is converted into a one-dimensional vector of size 1×d, and all one-dimensional vectors are superimposed in the X-axis direction to obtain an API semantic feature of size a×d. d represents the number of dimensions, and a represents the number of word embedding results in the API word embedding matrix.
[0084] S6. Input the API semantic features into the trained Android malware detection model to obtain the Android malware detection results.
[0085] In the present invention, unless otherwise clearly stipulated and limited, the terms such as "installation", "setting", "connection", "fixation" and "rotation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, ordinary technicians in this field can understand the specific meanings of the above terms in the present invention according to the specific circumstances.
[0086] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. An Android malware detection method based on fine-grained semantic feature extraction, characterized in that: The following steps are involved: S1. Decompile the Android software to be detected to obtain the corresponding source code file, use the open source framework Androguard to extract API features from the source code file, and obtain an API feature set; S2. preprocessing the API feature set to obtain a preprocessed API feature set; S3. Using a blacklist screening method to screen the preprocessed API feature set to obtain an API list; S4. Using a vocabulary blacklist filtering method to process the API list to obtain an embedding list; S5. Perform word embedding on the embedding list to obtain an API word embedding matrix; perform vector fusion on the API word embedding matrix to obtain API semantic features; S6. Input the API semantic features into the trained Android malware detection model to obtain the Android malware detection results.
2. The method for detecting Android malware based on fine-grained semantic feature extraction according to claim 1, characterized in that: Step S3 uses a blacklist screening method to screen the pre-processed API feature set to obtain an API list, including: S31. The number of API features in the statistical preprocessing feature set is N, and the target API scale is M; S32. If N>M, find all API features in the preprocessing feature set that appear in the pre-built blacklist frequency table and their positions in the table, and delete the top K API features in the table from the preprocessing feature set to obtain an API list; where K=NM.
3. The method for detecting Android malware based on fine-grained semantic feature extraction according to claim 2, characterized in that: The process of establishing the blacklist frequency table includes: S311. Collecting Android software sample data, including a benign Android software sample set and a malicious Android software sample set; decompiling each Android software sample to obtain the corresponding source code file, and using the open source framework Androguard to extract API features from each source code file to obtain an API feature data set; S312. Count the benign and malicious occurrences of each API feature in the API feature data set; the benign occurrence count of the API feature is the total number of times the API feature appears in the benign Android software sample set, and the malicious occurrence count of the API feature is the total number of times the API feature appears in the malicious Android software sample set; S313. Calculate the benign frequency value and malicious frequency value of each API feature in the API feature data set, expressed as: Among them, API C Indicates the number of API features in the API feature dataset; represents the number of benign occurrences of the ith API feature in the API feature dataset, represents the benign frequency value of the ith API feature in the API feature dataset, represents the number of malicious occurrences of the i-th API feature in the API feature dataset, Represents the malicious frequency value of the i-th API feature in the API feature dataset; S314. Combining all API features that appear only in the benign Android software sample set into a first set, and combining all API features that appear in both the benign Android software sample set and the malicious Android software sample set into a second set; S315. Arrange all API features in the first set in descending order according to benign frequency values to obtain a first part of the blacklist frequency table; S316. Calculate the frequency difference of each API feature in the second set, expressed as: F j represents the frequency difference of the jth API feature in the second set, represents the benign frequency value of the jth API feature in the second set, represents the malicious frequency value of the jth API feature in the second set; S317. Arrange all API features in the second set in descending order according to the frequency difference to obtain a second blacklist frequency table; add the second blacklist frequency table to the end of the first blacklist frequency table to obtain a complete blacklist frequency table.
4. The method for detecting Android malware based on fine-grained semantic feature extraction according to claim 1, characterized in that: Step S4 uses a vocabulary blacklist filtering method to process the API list to obtain an embedding list including: S41. Split each API feature in the API list into multiple words, and form a vocabulary list with all the words; S42. Delete all words in the vocabulary list that appear in the pre-built vocabulary blacklist to obtain an embedding list.
5. The method for detecting Android malware based on fine-grained semantic feature extraction according to claim 4, characterized in that: Step S41 uses the following rules to perform word segmentation: Replace special characters in API signatures with spaces; Replace the return value methods in the API features with the full English name of the actual data type.
6. The method for detecting Android malware based on fine-grained semantic feature extraction according to claim 4, characterized in that: The process of building a vocabulary blacklist includes: S411. Collecting Android software sample data, including a benign Android software sample set and a malicious Android software sample set; decompiling each Android software sample to obtain the corresponding source code file, and using the open source framework Androguard to extract API features from each source code file to obtain an API feature data set; S412. Split each API feature in the API feature data set, and count the number of occurrences of each word in the benign Android software set and the malicious Android software set; construct a benign Android software set vocabulary occurrence number set W A ={A1,A2,…,A U }, construct the occurrence count set W of malicious Android software vocabulary B = {B1, B2, …, B U }, where A u represents the number of times the u-th word appears in the benign Android software set, B u represents the number of times the u-th word appears in the malicious Android software collection, and U represents the number of words; S413. Calculate the benign word frequency and malicious word frequency of each word, expressed as: Among them, Fwb u Indicates the benign word frequency of the u-th word, Fmb u represents the malicious word frequency of the u-th word; S414. For each word, subtract the frequency of malicious words from its own frequency of benign words to obtain the difference frequency, sort all words in descending order according to the difference frequency, and extract the first 100 words to form a vocabulary blacklist; S415. Extract words whose benign word frequency and malicious word frequency both exceed the threshold, and select some of the words to add to the vocabulary blacklist.
7. The Android malware detection method based on fine-grained semantic feature extraction according to claim 1 is characterized in that: Use the pre-trained Word2Vec model to embed each word in the embedding list and get the API word embedding matrix.