Software similarity detection method and system
Through the multi-dimensional software similarity detection method, the APK reverse tool and deep learning algorithm are used to compare software from multiple dimensions, and the problems of low manual comparison efficiency and lack of intelligence in the existing technology are solved, and the software similarity is effectively judged by efficiently judging software similarity and homologousness improves the effectiveness of analysis and judgment and reduces the rate of misjudgment.
Patent Information
- Application Number
- CN202410203372.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-23
- Publication Date
- 2024-12-06
AI Technical Summary
The existing software comparison technology has problems such as low manual comparison efficiency, lack of intelligent methods for homology and similarity comparison, and it is difficult to efficiently judge the similarity and homology between software.
Using multi-dimensional software similarity detection method, unpacking information is obtained through reverse engineering of APK reverse tool, and from digital signatures, application permissions, application frameworks, code and sensitive information, application interface screenshots, mixed portrait labeling system, development team and initial development team The seven dimensions of the installation user are compared, and the similarity indicators between software are obtained through weighted calculations.
It realizes more efficient judgment of similarity and homology between software, fills the shortcomings of traditional digital signatures or Hash values "one-size-fits-all" and can better discover similar and counterfeit software, improves manual analysis and judgment efficiency, and reduces errors. The judgment rate is applicable to public safety and software version classification.
Smart Images

Figure CN119088450A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a technology in the field of information security, specifically a multi-level and multi-dimensional software similarity detection method and system. Background Art
[0002] When analyzing the same type of Android software installation packages, researchers usually establish a signature or hash value library of the software installation packages for retrieval and classification. When the retrieval is ineffective or new software is released, they often judge the similarity by manually checking the name and icon of the APP, or use a simulator to install it and actually perceive the similarities of the software functions and interfaces, and give a rough similarity range. Summary of the invention
[0003] The present invention aims at the limitations of database query in existing software comparison technology, solves the problems of low efficiency of manual comparison, lack of intelligent methods for software homology comparison and similarity comparison, and proposes a software similarity detection method and system, which deeply compares the similarity of the frameworks and the similarity of the contents of two or more software installation packages from multiple dimensions, more efficiently judges the similarity and homology between software, and accurately locates the connection between software.
[0004] The present invention is achieved through the following technical solutions:
[0005] The present invention relates to a software similarity detection method. First, an APK reverse tool is used to reverse engineer an APP to obtain unpacking information. For two APP software to be compared, seven dimensions are compared, including digital signature, application permission, application framework, code and sensitive information, application interface screenshot, mixed portrait label system, and development team and first-time user. Finally, a similarity index is obtained by weighting.
[0006] The unpacking information includes but is not limited to source code, application images, and application permissions.
[0007] The present invention relates to a software similarity detection system for implementing the above method, comprising: an installation package reverse parsing unit, a digital signature unit, an application permission unit, an application framework unit, a code and sensitive information unit, an activity interface unit, a mixed portrait unit and an initial installation group unit, wherein: the installation package reverse unit performs reverse parsing according to the software installation APK to obtain all reverse files; the digital signature unit obtains a hash value and digital signature content according to the basic features of the APK and performs comparison; the application permission unit calculates a score according to the permission importance and similarity by parsing the permission file obtained by the reverse unit; the application framework unit calculates a score according to the ranking of the reverse files The code and sensitive information unit extracts key codes and sensitive information based on the reversed files, and designs a model to obtain a similarity score. The activity interface unit calculates the interface similarity between applications according to functional modules based on the screenshots of the actual installation interface in the simulator. The mixed portrait unit collects historical user groups and application market partitions of the software, and comprehensively calculates the similarity scores between software. The initial installation group unit calculates the similarity score based on the initial installation user information and Wi-Fi information obtained by the mobile phone SDK, and the final similarity between software is obtained by weighted calculation of all similarity scores.
[0008] The reverse engineering files include but are not limited to: codes, images, and components. Technical Effects
[0009] The present invention can fill the gap in comparing the similarities of APP software, and transforms the traditional "one-size-fits-all" approach of digital signatures or Hash values into a multi-dimensional, in-depth and multi-faceted comparison, so that similar, counterfeit and homologous software APPs can be better discovered. For example, in the field of public security, handling similar fraud-related cases and cracking down on the manufacturing dens of the same fraud-related APP have positive social benefits. Of course, for the changes in software versions, APPs produced by the same company can also be better classified and distinguished, so as to achieve the purpose of distinguishing between benign and malicious software, and facilitate developers, researchers, and managers to work better. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 It is a schematic diagram of the principle of the present invention;
[0011] Figure 2 Flowchart for digital signature similarity calculation;
[0012] Figure 3 Convert software APK into a framework structure flowchart;
[0013] Figure 4 Flowchart for comparing software code and sensitive information;
[0014] Figure 5This is a screenshot comparison of the software application interface;
[0015] Figure 6 It is a diagram of the software hybrid portrait label system;
[0016] Figure 7 Compare flow charts for software development teams and first-time users. DETAILED DESCRIPTION
[0017] like Figure 1 As shown, this embodiment involves a software similarity detection method, which first uses an APK reverse tool to reverse engineer the APP to obtain source code, application images, and application permissions; for two APP software to be compared, seven dimensions are compared, including digital signatures, application permissions, application frameworks, codes and sensitive information, application interface screenshots, hybrid portrait label system, and development team and first-time users, and finally weighted to obtain a similarity index.
[0018] The reverse tools include androguard, APKTool, DARE, and DEX2JAR tools.
[0019] The seven dimensions specifically include:
[0020] 1) Digital signature: A method that combines the digital signature of a file with the signature of the author. When the two are identical, they are judged to be identical. When any one of them is identical, they are judged to have a strong similarity. When the two are different, they are judged to be different.
[0021] 2) Application permissions, including:
[0022] 2.1) According to the application permission characteristics, the application permissions are classified into three categories of high, medium and low importance. The present invention only exemplifies the three categories of high, medium and low, including but not limited to the above three categories, and a more detailed division can be performed according to actual conditions.
[0023] 2.2) Extract the application permissions of the two apps respectively to obtain sets P1 and P2, where the number of application permissions is LP1 and LP2 respectively; divide the application permission comparison level library into high, medium and low levels, corresponding to P 1high ,P 1mid ,P 1low and P 2high ,P 2mid ,P 2low The number of high, medium and low level application permissions are LP 1high ,LP 1mid ,LP 1low and LP 2high ,LP 2mid ,LP 2low, compare the divided application permissions at the same level, and calculate the similarity scores of high, medium, and low application permissions, recorded as sim high ,sim mid ,sim low , specifically: Where: The formula len(X) is used to calculate the number of elements in the set X. The similarity scores of high, medium and low levels are weighted and accumulated respectively, and the corresponding weights are set to wp high ,wp mid ,wp low , which is generally set to wp high ≥wp mid ≥wp low , indicating that high-level application permissions are the most important, followed by medium-level application permissions, and finally low-level application permissions.
[0024] 2.3) Calculate the similarity score of the two APP application permissions sim2=wp high ×sim high +wp mid ×sim mid +wp low ×sim low .
[0025] 3) Application framework: Determine whether the language of the APK is consistent and whether the file directory structure after the software APK is unpacked is similar. Use the file directory structure after the APK is unpacked to convert it into a directed graph of the application framework (the maximum depth of the graph is g). Train the graph neural network to compare the framework types to obtain similarity, including:
[0026] 3.1) Figure 3 As shown, according to the file hierarchy structure, folders or files are used as nodes, the node name is the file (folder) name concatenated with the file (folder) attributes, the folder is defined as folder, the file is defined as jpg, png, c, py, java, xml according to the suffix name, the relationship between the folder and the subordinate file (folder) is the edge, and the edges are extended to the file at the end (that is, the starting point is the folder and the end is the file), thus constructing a directed graph.
[0027] 3.2) According to the maximum depth g of the graph, if the file directory parsed by the installation package reverse parsing unit does not have a unique root node, the name of the APK is used as the root node folder; if the depth of the graph exceeds g layers, the g layers are calculated starting from the root node of the directed graph, and the remaining excess parts are ignored;
[0028] 3.3) By training GNN or GCN to compare the framework types, the vector groups of the two APP frameworks are obtained through the model, which are G1=[s 11 ,s 12 ,...,s 1g ] and G2=[s 21 ,s 22 ,...,s 2g ], where the length of the vector is g, and the s in the vector are all less than 1. The specific calculation of similarity is sim3 = G1·G2 = s 11 s 21 +s 12 s 22 +...+s 1g s 2g .
[0029] 4) Code and sensitive information, i.e. name, phone number, email address, bank card number, ID number, IP address, domain name address, SDK and other information of companies with certain persons, locations, IP locations and service providers, which are contained in the code, are extracted using key information extraction technology, including:
[0030] 4.1) After extracting the source code of the two apps through reverse engineering tools, the source code contains most of the functions and call information. All the call information is removed, and the functions in the source code are summarized using deep learning algorithms. The functions of the functions generated by natural language are output, and the functions of all functions are spliced together;
[0031] 4.2) Summarize again using the deep learning model, train a neural network-based similarity scoring model, and score the similarity of the summaries of the two apps;
[0032] 4.3) When analyzing the source code, use key information extraction technology to extract and compare key information, use one-hot encoding or custom digital encoding to train the neural network model, and calculate the corresponding similarity. The similarity calculation is specifically ||M1-M2||, where M1 and M2 correspond to two Embedding codes respectively.
[0033] The neural network model is the CodeBERT algorithm, specifically: when analyzing the source code, there are usually two parts: code comments and source code fields. The comments are composed of natural language (for example: Chinese, English), and the source code includes codes of various programming languages (for example: Python, Java, C++). Directly comparing different natural languages or programming languages will increase the burden of the model and the effect will decline. The present invention inputs the natural language and source code of the source file into the CodeBERT pre-training model, uniformly converts them into English code summaries, and outputs English Embedding codes, corresponding to M1 and M2, and the similarity calculation is specifically ||M1-M2||.
[0034] The key information extraction technology includes but is not limited to regular expressions, template extraction, and named entity recognition technology. Specifically, regular expressions are used to extract key sensitive information that meets certain rules, and information templates are used to detect whether each paragraph is sensitive. The fuzzy sensitive content of the entire paragraph is extracted, and then 70% of the fuzzy content is randomly selected. The accurate sensitive fields are manually labeled, and the named entity recognition UIE model is trained, and the trained model is used to automatically identify the key information fields.
[0035] 4.4) After weighting the scores of the similarity after code summary and the similarity of key information, the code and sensitive information scores between the two software are obtained.
[0036] 5) Screenshots of the application interface, which are obtained by:
[0037] 5.1) For the main running interfaces of the application, install and run the APK in the simulator respectively, and capture the main running screenshots. Among them, the startup page, boot page, login page, registration page, main interface, and personal center are the most important pages. If they exist in the APP, they must be captured. The number of other interfaces to be captured depends on the total number of interfaces of the application, but at least one page must be captured.
[0038] 5.2) Take screenshots of the seven major operating pages of the two APPs, including the startup page, boot page, login page, registration page, main interface, function navigation page, and personal center, and save them separately. Use the deep learning CLIP algorithm to calculate the similarity scores of the corresponding pages and obtain the corresponding scores.
[0039] The deep learning CLIP algorithm is specifically as follows: using the pre-trained CLIP model, the corresponding pages of two APPs to be compared are respectively input (startup page, boot page, login page, registration page, main interface, function navigation page, personal center one), and the Image Encoder encoding of the two pages is output, which is recorded as E1 and E2, that is, [e 1-1 ,e 1-2 ,...,e 1-512 ]and [e 2-1 ,e 2-2 ,...,e 2-512 ], the vector encoding length of each page is 512, and the cosine similarity of the two vectors is calculated to obtain the similarity score, which is
[0040] 5.3) When the above seven pages are missing in the APP or the relevant interface is not designed in the original APP, the similarity of the page is calculated as 0; if the similarity score of the corresponding page exceeds a certain threshold, the similarity of the page can be calculated as 1; if both APPs do not contain the page, the weight of the page is ignored, and the weights of other pages can be increased.
[0041] The above-mentioned increase in the weight of other pages is specifically: if and only if both apps lack the same interface, ignore the interface and increase the weight score ratio of the remaining pages, specifically:
[0042] 5.4) Accumulate the scores of all pages in a weighted manner.
[0043] 6) Hybrid portrait label system: Incorporate software user portrait labels, software function labels, software application market labels, and software common device labels into the hybrid portrait label system, such as Figure 6 As shown in the figure, starting from the software functions, user groups, market share and common models, the similarities between software are explored, including:
[0044] 6.1) All APP data are labeled after big data analysis, among which: each subcategory in the software user portrait label must give a measurement indicator, and in the non-quantitative category, it is converted into a quantitative label, for example: the age (segment) is 35-40 years old, whether there is a car is 75.5% with a car portrait label, such as occupation, workplace quantitative information exceeds a certain number (or threshold), it is automatically converted to "unlimited"; multiple software function labels can be selected, but generally not more than 5 labels, 1-3 labels are most appropriate; multiple software application market labels can also be selected, for a wide range of software APP listings; each subcategory in the software device label, a measurement indicator must be given, for example: the mobile phone language is 96.3% Chinese, the mobile phone operating system is 74.6% ios, and when it exceeds a certain number (or threshold), it is automatically converted to "unlimited".
[0045] 6.2) For all non-quantitative labels, for example, age (segment) is 35-40 years old, whether there is a car is 75.5% have a car, further quantified into numerical values or symbols, through not limited to Xgboost, GBDT, LightGBM, Catboost tree model or DeepGBM combined with deep neural network for training, to obtain the confidence of similarity, which more comprehensively reflects the similarity of portrait labels between software.
[0046] The software user portrait labels include age (segment), gender, occupation, place of work (province or city), place of residence (province or city), whether there is a car, whether there is a house, marital status, education (qualification) level, consumption level, frequency of use, and duration of use.
[0047] The software function tags include: games, chatting, social networking, financial management, reading, communication, shopping, sports, news, education, party building, music, video, sports and health, cars, travel, tools, illegal, and other categories.
[0048] The software application market labels include: android, ios, Huawei (application market), Xiaomi (application market), vivo (application market), OPPO (application market), Google Play, Samsung (application market), and other categories.
[0049] The software device tags include: mobile phone language, mobile phone model, mobile phone operating system, and mobile phone operating system version number category.
[0050] All tags are encoded in one-hot format, and the similarity is calculated using the Euclidean distance. The final calculation result is: the same result is 1, and the different result is 0.
[0051] All category portrait tags are collected by collecting SDK running data from all current application markets and strictly confidential personal information. They do not involve sensitive information such as people’s names, ID numbers, and mobile phone numbers, and are only used for software similarity calculations.
[0052] 7) Development team and first-time users: Analyze the personal information and Wi-Fi network sharing of the first-time users of the APP software to confirm the similarity between the development team and the initial testers and installers to determine whether there are similarities between the software, including:
[0053] 7.1) Set to collect 50 bits of user information of the first installation of the software APP (the threshold can be modified according to actual conditions), including IP address information, Wi-Fi address information, and installation time information after the first installation;
[0054] 7.2) Train the spatiotemporal graph similarity network to combine the time dimension and location information dimension of first-time users, and output the similarity score between the development team (developers and testers) and the first-time users. In some complex scenarios (for example, the comparison between different products of the same company, genuine and counterfeit APPs), it can better reflect the similarity between the development and testing cycles of software or between personnel.
[0055] The training spatiotemporal graph similarity network described above determines the similarity between the development team (developers and testers) and the initial installation users, specifically: ① Collect the installation time, IP address and Wi-Fi address information of the first 50 users who installed the software APP, and obtain the following: Figure 7 The installation time-space directed graph of the software APP shown sets the software APP installed on the same day as the same type and the same color in the time-space directed graph. The time dimension can be set dynamically according to the situation changes (for example: within two hours, within 2 days or within 1 week). The directed graph is composed of chronological order and geographical location characteristics. The last node in the time period of the previous day connects all the nodes of the next day, and so on. If there is no new installation on a certain day in the middle, that day will be skipped and the nodes of the next day will be connected. If there are multiple installation records on the same day, they are marked with the same color and are connected to each other in both directions.
[0056] ② After forming a directed graph, different nodes are encoded as one-hot vectors according to the time type, and the convolutional features of each node are obtained through the graph convolutional neural network. The average method is used to obtain the representation of the entire graph information, which is recorded as the global graph context feature vector The result of the inner product of the whole graph information and the node information is used as the attention coefficient Calculate the final graph embedding feature (graphembedding), that is, use feature h to represent the entire directed graph Where: u n is a node, f1 is the activation function, N(n) is the first-order domain of the node n and itself, d n Add one to the node degree, W1 (l) Represents the weight matrix associated with the lth layer of convolution of the graph convolutional neural network, represents weight offset, W2 is the weight matrix, and f2 is the activation function.
[0057] When comparing the image embedding features h1 and h2 of two images, the similarity is calculated using a neural tensor network. Among them: f3 is the activation function, is a weight tensor, K represents a hyperparameter, V is a weight vector, and b3 represents a weight offset. Finally, g1(h1,h2) can calculate a K-dimensional similarity vector, where K is 50 in the present invention.
[0058] ③ In order to further consider the temporal and spatial feature relationship between nodes, the directed graph adjacency feature matrix AFM1 is constructed, and each element in the adjacency feature matrix is multiplied by the influence factor T n Perform time constraints, and the adjacency feature matrix after the constraints is AFM final ; Generate AFM for the two pre-compared images final1 and AFM final2 , are all 50*50 square matrices, calculate the similarity of their corresponding points g2(h1,h2)=AFM final1 ×AFM final2 , and a 50*50 square matrix is calculated.
[0059] The one-way distance information of two nodes with directionality in the directed graph is defined as follows: 1 for the same street, 2 for different streets in the same district, 3 for different districts in the same city, 4 for different cities in the same province, and 5 for inter-province nodes. The node adjacency feature matrix constructed based on the distance information is as follows: Figure 7 As shown in the Adjacency feature matrix (AFM1).
[0060] The influence factor mentioned above refers to: taking into account the time factor, as the number of people in the communication channel increases, the subsequent influence gradually decreases, and the time influence factor Where: t N The installation date of the 50th installation user, t n is the installation date of the current node, t all The installation period (total number of days) for the first 50 users.
[0061] ④ In response to the situation where two APP software have overlapping location information among the first 50 installers, and the number of appearances on the same street is greater than 2 times, set the offset b4. When the same overlapping location appears multiple times, the offset b4 needs to be multiplied by the number of occurrences.
[0062] ⑤ Input g1(h1,h2) and g2(h1,h2) into a multi-layer fully connected neural network (number of layers>3), add the output of the neural network to the offset b4, and then apply the sigmoid function to ensure that the similarity score is in the range of (0,1).
[0063] 8) Output the similarity scores of the above 7 dimensions of the two APP software. Suppose the similarity scores of digital signature, application permission, application framework, code and sensitive information, application interface screenshot, mixed portrait label system, and development team and first-time users are sim1, sim2, sim3, sim4, sim5, sim6, and sim7 respectively.
[0064] The similarity index is obtained by calculating the weighted similarity scores, specifically: in: That is, the sum of the weights of all dimensions is 1.
[0065] After specific practical experiments, tests were conducted on 827 suspicious apps. In the specific environment of a server with a graphics card installed, it was found that there were actually 167 pairs of similar (homologous) apps. The number of pairs with similarity scores exceeding 0.8 output by the present invention was 145 pairs, and the number of pairs with similarity scores exceeding 0.9 was 89 pairs. The model comparison result output time was 45.81 seconds per pair, which greatly shortened the time for manual analysis and judgment.
[0066] Compared with the prior art, this method uses information from more dimensions, enriches APP similarity comparison items, compares APP similarity more comprehensively, improves manual research and judgment efficiency, and reduces the misjudgment rate; adds an image similarity comparison module, improves the sensitivity of the software user interface, and comprehensively compares the similarity of operating and using APP; proposes a new spatiotemporal directed graph similarity comparison calculation method, fully utilizes the location information and installation time of the early installation users, and mines the similarity of the habits of the development and testing personnel team, as well as the similarity of the communication channels. In addition, the present invention conforms to practical scenarios, can flexibly deal with the problem of missing data in a variety of complex scenarios, and the score information between different dimensions complements each other, all of which have reliable practical significance.
[0067] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.
Claims
1. A software similarity detection method, characterized in that: First, use the APK reverse tool to reverse engineer the APP to obtain the unpacking information; for the two APP software to be compared, compare them from seven dimensions: digital signature, application permissions, application framework, code and sensitive information, application interface screenshots, mixed portrait label system, and development team and first-time users, and finally weight them to obtain the similarity index; The digital signature is a combination of the file's digital signature and the author's signature. When the two are identical, they are considered to be identical. When any one of them is identical, they are considered to have strong similarity. When the two are different, they are considered to be different. The similarity of the application framework is obtained in the following manner: determining in turn whether the language of the APK is consistent and whether the file directory structure after the software APK is unpacked is similar, using the file directory structure after the APK is unpacked to convert it into a directed graph of the application framework, and training a graph neural network to compare the framework types to obtain the similarity; The code and sensitive information, namely the name, phone number, email address, bank card number, ID number, IP address, domain name address, SDK and other information of the company that can identify the person, location, IP location and provide services, are extracted using key information extraction technology; The similarity of the application interface screenshots is obtained specifically in the following manner: 5.1) For the main running interfaces of the application, install and run the APK in the simulator and capture the main running screenshots. The startup page, boot page, login page, registration page, main interface, and personal center are the most important pages. If they exist in the APP, they must be captured. The number of other interfaces to be captured depends on the total number of interfaces of the application, but at least one page must be captured; 5.2) Take screenshots of the startup page, guide page, login page, registration page, main interface, function navigation page, and personal center of the two apps and save them respectively. Use the deep learning CLIP algorithm to calculate the similarity scores of the corresponding pages and obtain the corresponding scores; 5.3) When the above seven pages are missing in the APP or the relevant interface is not designed in the original APP, the similarity of the page is calculated as 0; if the similarity score of the corresponding page exceeds a certain threshold, the similarity of the page can be calculated as 1; if both APPs do not contain the page, the weight of the page is ignored, and the weight of other pages can be increased; 5.4) Accumulate the scores of all pages in a weighted manner; The hybrid portrait label system specifically obtains its similarity in the following manner: software user portrait labels, software function labels, software application market labels, and software common device labels are incorporated into the hybrid portrait label system, and the similarity between software is discovered based on the software's functions, user groups, market share, and common models; The similarity between the development team and the initial installation user is obtained in the following manner: by analyzing the personal information and Wi-Fi network sharing of the initial installation user of the APP software, confirming the similarity between the development team and the initial tester and installer to determine whether there is similarity between the software, specifically including: 7.1) Set to collect 50 bits of user information of the first installation of the software APP (the threshold can be modified according to actual conditions), including IP address information, Wi-Fi address information, and installation time information after the first installation; 7.2) Train the spatiotemporal graph similarity network to combine the time dimension and location information dimension of first-time users, and output the similarity score between the development team (developers and testers) and the first-time users. In some complex scenarios (for example, the comparison between different products of the same company, genuine and counterfeit APPs), it can better reflect the similarity between the development and testing cycles of software or between personnel.
2. The software similarity detection method according to claim 1, characterized in that: The similarity of the application permissions is obtained in the following manner: 2.1) According to the application permission characteristics, the application permissions are classified into three categories of importance: high, medium and low. The present invention only exemplifies the three categories of high, medium and low, including but not limited to the above three categories. A more detailed classification can be performed according to the actual situation; 2.2) Extract the application permissions of the two apps respectively to obtain sets P1 and P2, where the number of application permissions is LP1 and LP2 respectively; divide the application permission comparison level library into high, medium and low levels, corresponding to P 1high ,P 1mid ,P 1low and P 2high ,P 2mid ,P 2low The number of high, medium and low level application permissions are LP 1high ,LP 1mid ,LP 1low and LP 2high ,LP 2mid ,LP 2low , compare the divided application permissions at the same level, and calculate the similarity scores of high, medium, and low application permissions, recorded as sim high ,sim mid ,sim low , specifically: Where: The formula len(X) is used to calculate the number of elements in the set X. The similarity scores of high, medium and low levels are weighted and accumulated respectively, and the corresponding weights are set to wp high ,wp mid ,wp low , which is generally set to wp high ≥wp mid ≥wp low , indicating that high-level application permissions are the most important, followed by medium-level application permissions, and finally low-level application permissions; 2.3) Calculate the similarity score of the two APP application permissions sim2=wp high ×sim high +wp mid ×sim mid +wp low ×sim low 。 3. The software similarity detection method according to claim 1, characterized in that: The similarity of the application framework is obtained specifically in the following manner: 3.1) According to the file hierarchy structure, folders or files are taken as nodes. The node name is the file (folder) name concatenated with the file (folder) attribute. Folders are defined as folders. Files are defined as jpg, png, c, py, java, and xml according to the suffix name. The relationship between folders and subordinate files (folders) is an edge, and the edges are extended to the files at the end (that is, the starting point is the folder and the end is the file), thus constructing a directed graph. 3.2) According to the maximum depth g of the graph, if the file directory parsed by the installation package reverse parsing unit does not have a unique root node, the name of the APK is used as the root node folder; if the depth of the graph exceeds g layers, the g layers are calculated starting from the root node of the directed graph, and the remaining excess parts are ignored; 3.3) By training GNN or GCN to compare the framework types, the vector groups of the two APP frameworks are obtained through the model, which are G1=[s 11 ,s 12 ,...,s 1g ] and G2=[s 21 ,s 22 ,...,s 2g ], where the length of the vector is g, and the s in the vector are all less than 1. The specific calculation of similarity is sim3 = G1·G2 = s 11 s 21 +s 12 s 22 +...+s 1g s 2g .
4. The software similarity detection method according to claim 1, characterized in that: The similarity between the code and the sensitive information is obtained in the following way: 4.1) After extracting the source code of the two apps through reverse engineering tools, the source code contains most of the functions and call information. All the call information is removed, and the functions in the source code are summarized using deep learning algorithms. The functions of the functions generated by natural language are output, and the functions of all functions are spliced together; 4.2) Summarize again using the deep learning model, train a neural network-based similarity scoring model, and score the similarity of the summaries of the two apps; 4.3) When analyzing the source code, use key information extraction technology to extract and compare key information, use one-hot encoding or custom digital encoding to train the neural network model, and calculate the corresponding similarity. The similarity calculation is specifically ||M1-M2||, where M1 and M2 correspond to two Embedding codes respectively; 4.4) After weighting the scores of the similarity after code summary and the similarity of key information, the code and sensitive information scores between the two software are obtained.
5. The software similarity detection method according to claim 4 is characterized in that: The neural network model is the CodeBERT algorithm, specifically: when analyzing the source code, there are usually two parts: code comments and source code fields. The comments are composed of natural language (for example: Chinese, English), and the source code includes codes of various programming languages (for example: Python, Java, C++). Directly comparing different natural languages or programming languages will increase the burden of the model and reduce the effect. The natural language and source code of the source file are input into the CodeBERT pre-trained model, uniformly converted into English code summaries, and output English Embedding encodings, corresponding to M1 and M2, and the similarity calculation is specifically ||M1-M2||.
6. The software similarity detection method according to claim 1, characterized in that: The deep learning CLIP algorithm is specifically as follows: using the pre-trained CLIP model, the corresponding pages of two APPs to be compared are respectively input (startup page, boot page, login page, registration page, main interface, function navigation page, personal center one), and the Image Encoder encoding of the two pages is output, which is recorded as E1 and E2, that is, [e 1-1 ,e 1-2 ,...,e 1-512 ] and [e 2-1 ,e 2-2 ,...,e 2-512 ], the vector encoding length of each page is 512, and the cosine similarity of the two vectors is calculated to obtain the similarity score, which is The above-mentioned increase in the weight of other pages is specifically: if and only if both apps lack the same interface, ignore the interface and increase the weight score ratio of the remaining pages, specifically:
7. The software similarity detection method according to claim 1, characterized in that: The hybrid image label system obtains its similarity in the following way: 6.1) All APP data are analyzed through big data and the software is labeled. Among them: each subcategory in the software user portrait label must give a measurement indicator. In the non-quantitative category, it is converted into a quantitative label. For example, if the age (segment) is 35-40 years old, whether there is a car is 75.5%. If the quantitative information of occupation and work place exceeds a certain number (or threshold), it will be automatically converted to "unlimited"; multiple software function labels can be selected, but generally no more than 5 labels, 1-3 labels are most appropriate; multiple software application market labels can also be selected to ensure that the software APP has a wide range of listings; each subcategory in the software device label must give a measurement indicator, for example: the mobile phone language is 96.3% Chinese, and the mobile phone operating system is 74.6% ios. When it exceeds a certain number (or threshold), it is also automatically converted to "unlimited"; 6.2) For all non-quantitative labels, Xgboost, GBDT, LightGBM, Catboost tree models or DeepGBM combined with deep neural networks are used for training to obtain the confidence level of similarity, which more comprehensively reflects the similarity of portrait labels between software.
8. The software similarity detection method according to claim 7, characterized in that: The software user portrait tags include age (segment), gender, occupation, place of work (province or city), place of residence (province or city), whether there is a car, whether there is a house, marital status, education (education level), consumption level, frequency of use, and duration of use; The software function tags include: games, chatting, social networking, financial management, reading, communication, shopping, sports, news, education, party building, music, video, sports and health, cars, travel, tools, illegal, and other categories; The software application market labels include: android, ios, Huawei (application market), Xiaomi (application market), vivo (application market), OPPO (application market), Google Play, Samsung (application market), and other categories; The software device tags include: mobile phone language, mobile phone model, mobile phone operating system, and mobile phone operating system version number category; All tags are encoded in one-hot format, and the similarity is calculated using the Euclidean distance. The final calculation result is: the same result is 1, and the different result is 0. All category portrait tags are collected by collecting SDK running data from all current application markets and strictly confidential personal information. They do not involve sensitive information such as people’s names, ID numbers, and mobile phone numbers, and are only used for software similarity calculations.
9. The software similarity detection method according to claim 1, characterized in that: The training spatiotemporal graph similarity network determines the similarity between the development team (developers and testers) and the first-time users, specifically as follows: ① Collect the installation time, IP address and Wi-Fi address information of the first 50 users who installed the software APP, and obtain the spatiotemporal directed graph of the installation of the software APP as shown in Figure 7. Set the software APPs installed on the same day to be of the same type and the same color in the spatiotemporal directed graph. The time dimension can be dynamically set according to the situation change (for example: within two hours, within 2 days or within 1 week). The directed graph is composed of chronological order and geographical location features. The last node in the time period of the previous day is connected to all nodes of the next day, and so on. If there is no new installation on a certain day in the middle, skip that day and replace it with the node of the next day for connection. If there are multiple installation records on the same day, they are marked with the same color and are bidirectionally connected to each other. ② After forming a directed graph, different nodes are encoded as one-hot vectors according to the time type, and the convolutional features of each node are obtained through the graph convolutional neural network. The average method is used to obtain the representation of the entire graph information, which is recorded as the global graph context feature vector The result of the inner product of the whole graph information and the node information is used as the attention coefficient Calculate the final graph embedding feature, that is, use feature h to represent the entire directed graph Where: u n is a node, f1 is the activation function, N(n) is the first-order domain of the node n and itself, d n Add one to the node degree, W1 (l) Represents the weight matrix associated with the lth layer of convolution of the graph convolutional neural network, represents weight offset, W2 is the weight matrix, and f2 is the activation function; ③ In order to further consider the temporal and spatial feature relationship between nodes, the directed graph adjacency feature matrix AFM1 is constructed, and each element in the adjacency feature matrix is multiplied by the influence factor T n Perform time constraints, and the adjacency feature matrix after the constraints is AFM final ; Generate AFM for the two pre-compared images final1 and AFM final2 , are all 50*50 square matrices, calculate the similarity of their corresponding points g2(h1,h2)=AFM final1 ×AFM final2 , calculate and get a 50*50 square matrix; ④ In response to the overlapping location information of two APP software in the first 50 installs, if the number of times the same street appears is greater than 2, set the offset b4. When the same overlapping location appears multiple times, the offset b4 needs to be multiplied by the number of occurrences; ⑤ Input g1(h1,h2) and g2(h1,h2) into a multi-layer fully connected neural network (number of layers>3), add the output of the neural network to the offset b4, and then apply the sigmoid function to ensure that the similarity score is in the range of (0,1); Similarity is calculated using a neural tensor network Among them: f3 is the activation function, is a weight tensor, K represents a hyperparameter, V is a weight vector, b3 represents a weight offset, and finally, g1(h1,h2) calculates a K-dimensional similarity vector; The influence factor mentioned above refers to: taking into account the time factor, as the number of people in the communication channel increases, the subsequent influence gradually decreases, and the time influence factor Where: t N The installation date of the 50th installation user, t n is the installation date of the current node, t all The installation period (total number of days) for the first 50 users.
10. A software similarity detection system for implementing the method described in any one of claims 1 to 9, characterized in that: include: The installation package reverse parsing unit, the digital signature unit, the application permission unit, the application framework unit, the code and sensitive information unit, the activity interface unit, the mixed portrait unit and the initial installation group unit, wherein: the installation package reverse unit performs reverse parsing according to the software installation APK to obtain all reverse files; the digital signature unit obtains the hash value and the digital signature content according to the basic characteristics of the APK and compares them; the application permission unit calculates the score according to the importance and similarity of the permission by parsing the permission file obtained by the reverse unit; the application framework unit designs a model to calculate the similarity score according to the arrangement and file type of the reverse file; the code and sensitive information unit extracts key code and sensitive information according to the reversed file, and designs a model to obtain the similarity score; the activity interface unit calculates the interface similarity between applications according to the functional modules based on the screenshots of the actual installation interface in the simulator; the mixed portrait unit collects the historical user groups and application market partitions of the software, and comprehensively calculates the similarity scores between the software; the initial installation group unit comprehensively calculates the similarity score based on the initial installation user information and Wi-Fi information obtained by the mobile phone SDK, and obtains the final similarity between the software by weighted calculation of all similarity scores.
