A method and system for detecting Android counterfeit applications based on user reviews

Abstract

The invention belongs to the field of software security of information security, and specifically relates to a method and system for detecting Android counterfeit applications based on user comments. The method comprises: obtaining an apk sample, application metadata and user comment data of an application to be tested; inputting the user comment data into a comment classification model to obtain a comment classification result; using a multi-keyword matching method to perform feature extraction processing on comment data related to counterfeiting behavior, and constructing an original application name candidate set; screening applications in the original application name candidate set according to apk sample files and application metadata; calculating the similarity between a sample of the application to be tested and a sample file of the screened application, and obtaining a detection result according to the similarity; the invention uses natural language processing technology to accurately search for user comment data of an application, obtains comment data related to describing counterfeiting behavior, and detects Android counterfeit applications according to the comment data, thereby improving the detection precision.

Description

Technical Field

[0001] The present invention belongs to the field of software security of information security, and in particular relates to a method and system for detecting Android counterfeit applications based on user comments. Background Art

[0002] Due to the openness and popularity of the Android system, counterfeit applications have appeared and become more widespread on the Android platform. Counterfeit applications not only steal users and revenue from original applications, bringing many negative impacts to the brand, but are also a major way for malicious applications to spread. As one of the main download channels for Android applications, third-party app stores need to prevent the influx of counterfeit applications and maintain a safe environment for the application market. However, the number of existing Android applications has reached millions and is still in a stage of rapid growth, which puts higher requirements on the accuracy and efficiency of counterfeit application detection.

[0003] Most app stores will review apps before they are put on the shelves, using a fast comparison method to determine whether an app is a counterfeit app. However, such methods are easily circumvented by attackers, resulting in a large number of counterfeit apps still entering app stores. Therefore, app stores will further rely on user feedback information and continue to monitor counterfeit apps through manual review. However, the number of user comments is huge, and the comments contain a lot of irrelevant information such as noise. It is impossible to find relevant information in time for analysis and judgment by manual methods alone.

[0004] At present, the main counterfeit application detection methods can be divided into: counterfeit application detection methods based on code features and counterfeit application detection methods based on interface features. Counterfeit application detection methods based on code features include extracting static features of functional codes from dimensions such as Dalvik opcode sequence and API call sequence, using hash technology to generate a fingerprint signature for the application, and detecting counterfeit applications by comparing the edit distance of the fingerprint. A small number of studies have discovered counterfeit applications by comparing the similarity of other files in the application, such as pictures, layout files, multimedia files, etc. Some other studies have improved the detection accuracy by extracting higher-level code semantic features, such as extracting the application's program dependency graph (PDG), control flow graph (CFG), etc., and then using the subgraph isomorphism method to detect counterfeit applications. Counterfeit application detection methods based on interface features include extracting dynamic features of applications from the application interface to detect counterfeit applications. There are two main types of interface dynamic features. One is the interface content / interface structure features based on control attributes. Researchers extract and analyze interface codes to obtain relevant attribute information of controls defined in each interface of the application, including text, background color, coordinate area, etc., and build feature vectors of the interface based on relevant attribute information for comparison. The other is the dynamic fingerprint feature based on application interface screenshots. Researchers use automated testing technology to start and traverse all interfaces of the application in sequence, extract interface screenshots during the application running process, and use hashing technology to generate dynamic fingerprint features of the application for comparison.

[0005] Although the counterfeit application detection method based on code static features has a fast comparison speed, it is easy to be evaded by attackers; the counterfeit application detection method based on interface features can resist common code-level attack methods such as application code obfuscation and instruction replacement. However, the slow traversal speed of existing automated testing technology also makes it inefficient to support millions of applications. In addition, due to the low interface coverage of existing automated testing, it is impossible to fully extract the dynamic features of all application interfaces, resulting in the difficulty in defining the similarity threshold for counterfeit application determination, and the accuracy cannot be guaranteed. Summary of the invention

[0006] In order to solve the problems existing in the above prior art, the present invention proposes a method for detecting Android counterfeit applications based on user reviews, which comprises:

[0007] S1: Obtain the apk sample, application metadata, and user review data of the application to be tested;

[0008] S2: Preprocess user comment data;

[0009] S3: input the preprocessed user comment data into the comment classification model to obtain comment classification results; wherein the comment classification results include comment data related to counterfeiting and comment data not related to counterfeiting;

[0010] S4: Using keyword matching method to extract features from the review data related to counterfeiting behavior, obtain text information describing the original application, and construct a candidate set of the original application name based on the text information;

[0011] S5: Obtaining an apk sample file and corresponding application metadata for each application in the original application name candidate set;

[0012] S6: Screening applications in the original application name candidate set according to the apk sample file and application metadata;

[0013] S7: Calculate the similarity between the apk sample of the application to be tested and the apk sample files of the screened application, and obtain the detection result according to the similarity.

[0014] Preferably, preprocessing the user comment data includes: removing stop words and special characters from the user comment data, and performing word segmentation processing on the user comment data.

[0015] Preferably, the process of building a review classification model includes:

[0016] Step 1: Obtain comment data, and mark the comment data as comment data related to counterfeiting and comment data not related to counterfeiting;

[0017] Step 2: Preprocess the labeled review data;

[0018] Step 3: Extract keywords from the preprocessed marked comment data and sort the keywords using the TF-IDF algorithm; filter the sorted keywords to obtain a rule set;

[0019] Step 4: Construct constraints for the rule set;

[0020] Step 5: Build a semantic rule set based on the constraints and rule set, and build a classifier based on the semantic rule set.

[0021] Furthermore, the process of screening the sorted keywords includes: traversing each keyword from top to bottom in turn, if the frequency sets corresponding to two keywords belong to an inclusion relationship, then the two keywords are associated with a rule set, otherwise the latter keyword is put into a new rule set; when the union of the comment sets corresponding to all traversed keywords contains all the comments, stop traversal and output the rule set.

[0022] Preferably, the constraints for constructing a rule set include: constructing initial constraints based on the rule set, the initial constraints including the context of keywords and the allowed word spacing; enumerating constraints to calculate the accuracy and recall of the rule set under different orders and allowed word spacings, and selecting the constraint with the highest F1-Score as the constraint of the current rule set.

[0023] Preferably, the process of using the keyword matching method to perform feature extraction processing on the counterfeit behavior related comment data includes: obtaining the leading and trailing delimiting keywords of the counterfeit application name; matching the obtained leading and trailing delimiting keywords with the counterfeit behavior related comment data to obtain the text information of the counterfeit application; segmenting the text information to generate an n-dimensional word vector w; concatenating adjacent words in the word vector; and using the n-dimensional word vector w and the concatenated vector as a candidate set for the counterfeit application name.

[0024] Preferably, screening the applications in the original application name candidate set includes: obtaining the size of the applications and the category of the applications in the original application name candidate set, and calculating the correlation between the applications and the application to be tested based on the size and category of the applications; setting a correlation threshold, and deleting applications with a correlation less than the set correlation threshold from the original application name candidate set, until all applications are compared, and then outputting the final original application name candidate set.

[0025] An Android counterfeit application detection system based on user reviews, the system comprising: an input module, an information extraction module, an original application candidate set construction module and a result output module;

[0026] The input module is used to obtain application samples, application metadata and user comment information corresponding to the application;

[0027] The information extraction module includes a comment information classification module and an original application name extraction module;

[0028] The comment information classification module uses semantic rule matching technology to mark the input user comments into behavior categories, and the marked categories include counterfeit behavior related comment data and counterfeit behavior unrelated comment data;

[0029] The original application name extraction module extracts all possible original application names from the counterfeit behavior related comment data through keyword matching technology;

[0030] The original application candidate set construction module includes an application search module and a relevance analysis module;

[0031] The application search module obtains the apk sample file and the corresponding application metadata of each original application according to the application name, and inputs the obtained data into the correlation analysis module;

[0032] The correlation analysis module is provided with a correlation threshold, and the correlation between each application's apk sample file and the corresponding application metadata is calculated with respect to the application to be tested, and the calculated correlation is compared with the set correlation threshold. If the correlation is greater than the set correlation threshold, it is retained, otherwise it is discarded; all applications greater than the correlation threshold are collected to obtain a candidate set;

[0033] The result output module includes an application similarity comparison module and a result report output module;

[0034] The application similarity comparison module is used to compare the similarity of the application to be detected with the applications in the candidate set one by one to obtain a comparison result;

[0035] The result report output module is used to output the comparison result of the application similarity comparison module.

[0036] Beneficial effects of the present invention:

[0037] The present invention uses natural language processing technology to accurately search for user comment data of applications, obtains comment data related to description of counterfeiting behavior, detects Android counterfeit applications based on the comment data, and improves the precision and accuracy of detection; the present invention automatically extracts keywords based on manually annotated training set data, traverses keywords in order to associate, and constructs a semantic rule set. The semantic rules of multiple keywords automatically calculate the keyword order and word spacing constraints to obtain the final semantic rule classifier, and realize the automatic classification of user comments; for the classified user comments, the present invention automatically locates the front and back positions describing the original application information based on multiple keywords, and performs word segmentation on the extracted information, combines two or more adjacent words, constructs a set of original application name candidates, and improves the accuracy of original application name extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flowchart of a method for detecting counterfeit Android applications based on user comments is provided by the present invention;

[0039] Figure 2 Constructing a flow chart for the review classification model of the present invention;

[0040] Figure 3 Extraction flow chart of the original application name of the present invention;

[0041] Figure 4 This is a framework diagram of the Android counterfeit application detection system based on user reviews of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The main purpose of the present invention is to provide an automatic discovery method for Android counterfeit applications based on user comments. On the one hand, this method can help application store managers reduce the human and material costs required for review and approval. On the other hand, by constructing a set of original application candidates, it is not necessary to compare them with all applications in the application store, thereby improving the efficiency of counterfeit application detection and helping to discover and deal with counterfeit applications more timely. This method can also be used for other types of application behavior audits (for example, advertising behavior), as well as other mobile smart terminals such as iOS platforms.

[0044] A specific implementation method of a method for detecting counterfeit Android applications based on user reviews, such as Figure 1 As shown, the method includes:

[0045] S1: Obtain the apk sample, application metadata, and user review data of the application to be tested;

[0046] S2: Preprocess user comment data;

[0047] S3: input the preprocessed user comment data into the comment classification model to obtain comment classification results; wherein the comment classification results include comment data related to counterfeiting and comment data not related to counterfeiting;

[0048] S4: Use keyword matching method to extract features from the review data related to counterfeiting behavior, obtain text information describing the original application, and construct a candidate set of the original application name based on the text;

[0049] S5: Obtaining an apk sample file and corresponding application metadata for each application in the original application name candidate set;

[0050] S6: Screening applications in the original application name candidate set according to the apk sample file and application metadata;

[0051] S7: Calculate the similarity between the apk sample of the application to be tested and the apk sample files of the screened application, and obtain the detection result according to the similarity.

[0052] Preprocessing the user comment data includes: removing stop words and special characters from the user comment data, and performing word segmentation on the user comment data.

[0053] like Figure 2 As shown in Figure 1, the process of building a review classification model includes:

[0054] Step 1: Obtain comment data, and mark the comment data as comment data related to counterfeiting and comment data not related to counterfeiting;

[0055] Step 2: Preprocess the labeled review data;

[0056] Step 3: Extract keywords from the preprocessed marked comment data and sort the keywords using the TF-IDF algorithm; filter the sorted keywords to obtain a rule set;

[0057] The process of screening the sorted keywords includes: traversing each keyword from top to bottom in turn. If the frequency sets corresponding to two keywords belong to an inclusion relationship, the two keywords are associated with a rule set, otherwise the latter keyword is put into a new rule set; when the union of the comment sets corresponding to all traversed keywords contains all the comments, stop traversal and output the rule set.

[0058] Multi-keyword extraction: Based on the preprocessed data of the categories related to counterfeiting, the keywords are sorted using the TF-IDF algorithm, and each keyword is traversed from top to bottom. If the comment sets corresponding to two keywords are in a containment relationship, the two keywords are associated into a rule set, otherwise the latter keyword is put into a new rule set. When the union of the comment sets corresponding to all previously traversed keywords contains all the comments, the traversal stops. The output of this step is a rule set containing one or more keywords.

[0059] Step 4: Construct constraints for the rule set;

[0060] The constraints for constructing a rule set include: constructing initial constraints based on the rule set, the initial constraints include the context of keywords and the allowed word spacing; enumerating constraints to calculate the accuracy and recall of the rule set under different orders and allowed word spacings, and selecting the constraint with the highest F1-Score as the constraint of the current rule set.

[0061] Step 5: Build a semantic rule set based on the constraints and rule set, and build a classifier based on the semantic rule set.

[0062] The process of extracting features from counterfeit behavior related comment data using the keyword matching method includes: obtaining the leading and trailing delimiting keywords that describe the original application name; matching the obtained leading and trailing delimiting keywords with the counterfeit behavior related comment data to obtain text information describing the original application; segmenting the text information to generate an n-dimensional word vector w; concatenating adjacent words in the word vector; and using the n-dimensional word vector w and the concatenated vector as a candidate set for the original application name.

[0063] like Figure 3 As shown, the process of extracting the application name using the original application name extractor includes:

[0064] Step 1. Keyword extraction: Extract the keywords that describe the original application name, such as "imitate" and the end symbol (such as a period or the end of a sentence) to be ....

[0065] Step 2. Keyword matching: Based on the separated keywords extracted in step 1, the classified comments output by the comment classification model are matched to extract the intermediate text information describing the original application name.

[0066] Step 3. Text segmentation: Segment the information output in step 2 to generate an n-dimensional word vector w. This step cannot remove stop words because some common stop words (for example, ) are part of the application name.

[0067] Step 4. Text splicing: Splice adjacent words in the word vector. For example, two-by-two splicing will generate an n-1 dimensional vector, and three-by-three splicing will generate an n-2 dimensional vector. The specific splicing method can be set in the device according to actual conditions.

[0068] Step 5. Extract the original application name: Use the word vector w and the multiple concatenated vectors output by S44 as a candidate set for the original application name.

[0069] Screening the counterfeit applications in the original application name candidate set includes: obtaining the size and category of the applications in the original application name candidate set, and calculating the correlation between the applications and the application to be tested based on the size and category of the applications; setting a correlation threshold, and deleting applications with a correlation less than the set correlation threshold from the original application name candidate set, until all applications are compared, and then outputting the final original application name candidate set.

[0070] The range of the set correlation threshold is an arbitrary constant between 0 and 1, where the size of the correlation threshold is related to the inspection accuracy and efficiency of the model, that is, the correlation threshold can be set according to the user's inspection requirements. For example, the higher the accuracy of the model detection, the smaller the threshold is set. If the detection efficiency of the model is improved, the larger the correlation threshold is set.

[0071] The process of calculating the similarity between the apk sample of the application to be tested and the apk sample file of the screened application includes: setting a similarity threshold, extracting text features in the apk sample of the application to be tested and the apk sample of the application, wherein the extracted text features include data related to the number of software codes, language, operation mode, etc.; comparing the two extracted text features to determine a similarity score, and comparing the similarity score with the set similarity threshold. If the score is less than the similarity threshold, the software is not counterfeit software, otherwise the software to be tested is counterfeit software.

[0072] An Android counterfeit application detection system based on user reviews, such as Figure 4 As shown, the system includes: an input module, an information extraction module, an original application candidate set construction module and a result output module;

[0073] The input module is used to obtain application samples, application metadata and user comment information corresponding to the application;

[0074] The information extraction module includes a comment information classification module and an original application name extraction module;

[0075] The comment information classification module uses semantic rule matching technology to mark the input user comments into behavior categories, and the marked categories include counterfeit behavior related comment data and counterfeit behavior unrelated comment data;

[0076] The original application name extraction module extracts all possible original application names from the counterfeit behavior related comment data through keyword matching technology;

[0077] The original application candidate set construction module includes an application search module and a relevance analysis module;

[0078] The application search module obtains the apk sample file and corresponding application metadata of each application according to the original application name, and inputs the obtained data into the correlation analysis module;

[0079] The correlation analysis module is provided with a correlation threshold, and the correlation is calculated with the application to be tested according to the apk sample file and the corresponding application metadata of each original application, and the calculated correlation is compared with the set correlation threshold. If it is greater than the set correlation threshold, it is retained, otherwise it is discarded; all counterfeit applications with a correlation greater than the correlation threshold are collected to obtain a candidate set;

[0080] The result output module includes an application similarity comparison module and a result report output module;

[0081] The application similarity comparison module is used to compare the similarity between the application to be detected and the counterfeit applications in the candidate set to obtain a comparison result;

[0082] The result report output module is used to output the comparison result of the application similarity comparison module.

[0083] The specific implementation of the system of the present invention is the same as the specific implementation of the method.

[0084] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting counterfeit Android applications based on user reviews, characterized in that: include: S1: Obtain the apk sample, application metadata, and user review data of the application to be tested; S2: Preprocess user comment data; S3: Input the preprocessed user comment data into the comment classification model to obtain the comment classification results; The review classification results include review data related to counterfeiting and review data not related to counterfeiting; S4: extracting features from the counterfeit behavior related comment data using a keyword matching method to obtain text information describing the original application, and constructing a candidate set of the original application name based on the text information; specifically including: obtaining the leading and trailing delimiting keywords of the original application name; matching the obtained leading and trailing delimiting keywords with the counterfeit behavior related comment data to obtain text information describing the original application; segmenting the text information to generate an n-dimensional word vector w; concatenating adjacent words in the word vector; and using the n-dimensional word vector w and the concatenated vector as a candidate set of counterfeit application names; S5: Obtaining an apk sample file and corresponding application metadata for each application in the original application name candidate set; S6: Screening the applications in the original application name candidate set according to the apk sample file and the application metadata; specifically including: obtaining the size and category of the applications in the original application name candidate set, and calculating the relevance with the application to be tested according to the size and category of the applications; setting a relevance threshold, deleting applications with a relevance less than the set relevance threshold from the original application name candidate set, until all applications are compared, and outputting the final original application name candidate set; S7: Calculate the similarity between the apk sample of the application to be tested and the apk sample file of the screened original application in sequence, and obtain the detection result according to the similarity.

2. According to claim 1, a method for detecting Android counterfeit applications based on user reviews is characterized in that: Preprocessing the user comment data includes: removing stop words and special characters from the user comment data, and performing word segmentation on the user comment data.

3. The method for detecting Android counterfeit applications based on user reviews according to claim 1, characterized in that: The process of building a review classification model includes: Step 1: Obtain comment data, and mark the comment data as comment data related to counterfeiting and comment data not related to counterfeiting; Step 2: Preprocess the labeled review data; Step 3: Extract keywords from the preprocessed marked comment data and sort the keywords using the TF-IDF algorithm; filter the sorted keywords to obtain a rule set; Step 4: Construct constraints for the rule set; Step 5: Build a semantic rule set based on the constraints and rule set, and build a classifier based on the semantic rule set.

4. The method for detecting Android counterfeit applications based on user reviews according to claim 3, characterized in that: The process of screening the sorted keywords includes: traversing each keyword from top to bottom in turn. If the frequency sets corresponding to two keywords belong to an inclusion relationship, the two keywords belonging to the inclusion relationship are associated into a rule set, otherwise the second keyword is put into a new rule set; when the union of the comment sets corresponding to all traversed keywords contains all the comments, stop traversing and output the rule set.

5. The method for detecting Android counterfeit applications based on user reviews according to claim 3, characterized in that: The constraints for constructing a rule set include: constructing initial constraints based on the rule set, the initial constraints include the context of keywords and the allowed word spacing; enumerating constraints to calculate the accuracy and recall of the rule set under different orders and allowed word spacings, and selecting the constraint with the highest F1-Score as the constraint of the current rule set.

6. A system for detecting counterfeit Android applications based on user reviews, the system being used to execute the method for detecting counterfeit Android applications based on user reviews as claimed in any one of claims 1 to 5, characterized in that: The system includes: an input module, an information extraction module, an original application candidate set construction module and a result output module; The input module is used to obtain application samples, application metadata and user comment information corresponding to the application; The information extraction module includes a comment information classification module and an original application name extraction module; The comment information classification module uses semantic rule matching technology to mark the input user comments into behavior categories, and the marked categories include counterfeit behavior related comment data and counterfeit behavior unrelated comment data; The original application name extraction module extracts all possible original application names from the counterfeit behavior related comment data through keyword matching technology; The original application candidate set construction module includes an application search module and a relevance analysis module; The application search module obtains the apk sample file and corresponding application metadata of each application according to the original application name, and inputs the obtained data into the correlation analysis module; The correlation analysis module is provided with a correlation threshold, and the correlation is calculated with the application to be tested according to the apk sample file and the corresponding application metadata of each original application, and the calculated correlation is compared with the set correlation threshold. If it is greater than the set correlation threshold, it is retained, otherwise it is discarded; all applications greater than the correlation threshold are collected to obtain the original application candidate set; The result output module includes an application similarity comparison module and a result report output module; The application similarity comparison module is used to compare the similarity of the application to be detected with the applications in the candidate set one by one to obtain a comparison result; The result report output module is used to output the comparison result of the application similarity comparison module.

Images