File similarity determination method and device and electronic equipment

Abstract

The invention discloses a file similarity determination method and device and electronic equipment. The method comprises the steps that bytecode text files corresponding to an original file and a target file are obtained, the target file comprises a file obtained by rewriting and repackaging the original file, and the bytecode text files comprise files obtained by decompiling bytecode files of the original file and the target file respectively; an original feature vector corresponding to the original file and a target feature vector corresponding to the target file are determined according to an instruction set in the bytecode text file, and the instruction set is used for determining operation logic of codes in the bytecode text file; and determining the similarity between the original file and the target file according to the similarity between the original feature vector and the target feature vector. The technical problem of low similarity evaluation precision caused by performing similarity analysis on application package files by using a function call graph of a code in related technologies is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a file similarity determination method and device and electronic equipment. BACKGROUND

[0002] With the rapid popularization of the Internet, various convenient mobile application products have become an indispensable part of people's daily life, but most of the mobile application products come from third-party platforms or non-official application stores. Due to the lack of strict application release management mechanism, many secondary developers will decompile the obtained application package file, and then rewrite the code therein to achieve the purpose of implanting malicious code into the application, and finally repack the various resource files obtained by decompiling to obtain a new application package file and republish it to the third-party application platform.

[0003] However, for application package file similarity analysis, the related technology extracts the function call graph in the code as a feature for similarity calculation, but the secondary developer will rewrite the function call in the code when repacking the application package file, so that the function call graph changes, resulting in inaccurate similarity evaluation.

[0004] In view of the above problems, no effective solution has been proposed so far. SUMMARY

[0005] The embodiments of the present application provide a file similarity determination method and device and electronic equipment, to at least solve the technical problem of low similarity evaluation accuracy caused by using the function call graph of the code for application package file similarity analysis in the related technology.

[0006] According to an aspect of an embodiment of the present application, a file similarity determination method is provided, comprising: obtaining bytecode text files corresponding to an original file and a target file, wherein the target file includes a file obtained by rewriting and repacking the original file, and the bytecode text files include files obtained by respectively decompiling bytecode files of the original file and the target file; determining an original feature vector corresponding to the original file and a target feature vector corresponding to the target file according to instruction sets in the bytecode text files, wherein the instruction sets are used to determine the running logic of the code in the bytecode text files; and determining the similarity between the original file and the target file according to the similarity between the original feature vector and the target feature vector.

[0007] In some embodiments of the present application, the original feature vector corresponding to the original file is determined according to the instruction set in the bytecode text file, comprising: extracting a first instruction set from the code of the first bytecode text file corresponding to the original file, and classifying all instructions in the first instruction set according to instruction types; for each first instruction subset corresponding to an instruction type, using an inter-class autoencoder corresponding to the instruction type to encode the first instruction subset to obtain a first feature vector, wherein the inter-class autoencoder is used to learn the relevance between the first instructions in the first instruction subset; and performing feature aggregation on the first feature vectors corresponding to all instruction types respectively to obtain the original feature vector.

[0008] In some embodiments of the present application, the target feature vector corresponding to the target file is determined according to the instruction set in the bytecode text file, comprising: extracting a second instruction set from the code of the second bytecode text file corresponding to the target file, and classifying all instructions in the second instruction set according to instruction types; for each second instruction subset corresponding to an instruction type, using an inter-class autoencoder corresponding to the instruction type to encode the second instruction subset to obtain a second feature vector, wherein the inter-class autoencoder is used to learn the relevance between the second instructions in the second instruction subset; and performing feature aggregation on the second feature vectors corresponding to all instruction types respectively to obtain the target feature vector.

[0009] In some embodiments of the present application, the inter-class autoencoder is trained in the following manner: obtaining a first training feature vector and a second training feature vector, wherein the first training feature vector comprises an instruction feature vector extracted from the original file training set, and the second training feature vector comprises an instruction feature vector with the same instruction type as the first training feature vector extracted from the target file training set; determining a training sample pair corresponding to the first training feature vector and the second training feature vector, and determining a label for each training sample pair, wherein the label is used to quantitatively represent the degree of association between the first training feature vector and the second training feature vector in the training sample pair; using an initial inter-class autoencoder to encode the first training feature vector in the training sample to obtain an encoding result; determining a predicted label according to the encoding result and the second training feature vector, wherein the predicted label is used to reflect the predicted degree of association between the first training feature vector and the second training feature vector; determining a loss according to the encoding result and the predicted label, and iteratively updating the parameters of the initial inter-class autoencoder according to the loss, and stopping iteration under the condition that a stop condition is met to obtain the inter-class autoencoder.

[0010] In some embodiments of the present application, the initial inter-class autoencoder is used to encode the first training feature vector in the training sample to obtain an encoding result, including: using the encoder of the initial inter-class autoencoder to encode the first training feature vector to obtain a first encoding result, wherein the first encoding result includes an intermediate representation after dimensionality reduction of the first training feature vector; using the decoder of the initial inter-class autoencoder to decode the first encoding result to obtain a second encoding result, wherein the second encoding result is used to reconstruct the first training feature vector based on the first encoding result; and determining the first encoding result and the second encoding result as the encoding result.

[0011] In some embodiments of the present application, the loss is determined according to the encoding result and the predicted label, including: determining a first loss corresponding to the second encoding result and the first training feature vector, and determining a second loss corresponding to the predicted label and the label, to obtain a loss corresponding to the first loss and the second loss.

[0012] In some embodiments of the present application, the original file and / or the target file correspond to a plurality of bytecode text files respectively, and each bytecode text file corresponds to a class; the method further includes: determining a sub-type of the target instruction contained in each bytecode text file, and assigning a value to each sub-type according to a preset attribute; determining a keyword of the bytecode text file in the storage structure according to the attribute values corresponding to all sub-types of each bytecode text file; and storing the feature vector corresponding to each bytecode text file according to the keyword.

[0013] In some embodiments of the present application, the similarity between the original feature vector and the target feature vector is determined by: traversing a first keyword corresponding to the target file, and sequentially comparing the first keyword with all second keywords corresponding to the original file; in a case where the comparison result indicates that there is a target keyword identical to the first keyword in the second keyword, performing similarity calculation on the feature vector corresponding to the target keyword and the feature vector corresponding to the first keyword to obtain a similarity value; and determining the similarity between the original feature vector and the target feature vector according to the similarity value.

[0014] In some embodiments of the present application, it further includes: in a case where the feature vector corresponding to the target keyword and the feature vector corresponding to the first keyword are similar according to the similarity value, marking the feature vector corresponding to the target keyword, wherein the marking is used to indicate that the feature vector corresponding to the target keyword is no longer subjected to similarity calculation.

[0015] In some embodiments of the present application, the similarity between the original feature vector and the target feature vector is determined by: determining the feature distribution similarity of the instruction sets respectively corresponding to the original feature vector and the target feature vector in the high-dimensional feature space, to obtain a first similarity; determining the coincidence degree between the instruction sets respectively corresponding to the original feature vector and the target feature vector, to obtain a second similarity; and determining the similarity according to the first similarity and the second similarity.

[0016] In some embodiments of the present application, the plurality of bytecode text files corresponding to the original file are sorted in descending order of file size, and files in the bytecode text file sequence are deleted in the order from front to back at a preset proportion, to obtain the target bytecode text file.

[0017] According to still another aspect of the embodiments of the present application, a file similarity determination apparatus is further provided, comprising: an acquisition module configured to acquire bytecode text files corresponding to an original file and a target file, wherein the target file comprises a file obtained by modifying and repacking the original file, and the bytecode text files comprise files obtained by respectively decompiling bytecode files of the original file and the target file; a determination module configured to determine an original feature vector corresponding to the original file and a target feature vector corresponding to the target file according to instruction sets in the bytecode text files, wherein the instruction sets are used to determine the running logic of the codes in the bytecode text files; and a comparison module configured to determine the similarity between the original file and the target file according to the similarity between the original feature vector and the target feature vector.

[0018] According to still another aspect of the embodiments of the present application, an electronic device is further provided, comprising: a memory and a processor, the memory is configured to store program instructions; the processor is connected with the memory and is configured to execute the above-mentioned file similarity determination method.

[0019] According to still another aspect of the embodiments of the present application, a non-volatile storage medium is further provided, comprising a stored computer program, wherein a device where the non-volatile storage medium is located executes the above-mentioned file similarity determination method by running the computer program.

[0020] According to still another aspect of the embodiments of the present application, a computer program product is further provided, comprising computer instructions, which are executed by a processor to implement the above-mentioned file similarity determination method.

[0021] In the embodiment of the present application, the similarity of the instruction set in the bytecode text file is analyzed, the similarity of the code running logic in the original application package file and the target application package file is compared, the purpose of accurately identifying the repackaged file is achieved, the technical effect of accurately detecting similar applications in the case of complex code is realized, and the technical problem that the related art uses the function call graph of the code to analyze the similarity of the application package file, resulting in low similarity evaluation accuracy is solved. BRIEF DESCRIPTION OF DRAWINGS

[0022] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and serve to explain the present application. In the drawings:

[0023] Figure 1 Fig. 1 is a hardware structure block diagram of a computer terminal according to the file similarity determination method of an embodiment of the present application;

[0024] Figure 2 Fig. 2 is a flowchart of the file similarity determination method according to the embodiment of the present application;

[0025] Figure 3 Fig. 3 is a schematic diagram of the overall flow of the file similarity determination method according to the embodiment of the present application;

[0026] Figure 4 Fig. 4 is a schematic diagram of the application package file according to the file similarity determination method of the embodiment of the present application;

[0027] Figure 5 Fig. 5 is a schematic diagram of the multi-class feature aggregation according to the file similarity determination method of the embodiment of the present application;

[0028] Figure 6 Fig. 6 is a flowchart of the inter-class autoencoder training according to the file similarity determination method of the embodiment of the present application;

[0029] Figure 7 Fig. 7 is a schematic diagram of the similarity calculation principle according to the file similarity determination method of the embodiment of the present application;

[0030] Figure 8 Fig. 8 is a schematic diagram of the similarity calculation flow according to the file similarity determination method of the embodiment of the present application;

[0031] Figure 9 Fig. 9 is a structural schematic diagram of the file similarity determination device according to the embodiment of the present application. DETAILED DESCRIPTION

[0032] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to only those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product or device.

[0034] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0035] Application Package File (APK): Application Package File is a packaging format specially designed for distributing and installing software applications on the Android operating system, which contains all the components required for application running. In the embodiments of the present application, APK file is taken as an analysis object, providing code and resource files from which Smali code is extracted for similarity analysis.

[0036] Smali language: a human-readable text format based on Dalvik virtual machine bytecode, i.e. an interpretation language of Davlik bytecode, mainly used for reverse engineering and debugging Android applications. In the embodiments of the present application, Smali code is a key analysis object extracted from the decompiled APK file, and the features of Smali code are analyzed to detect whether the application is repackaged.

[0037] Inter-Class Autoencoder (ICA): ICA is a machine learning model for encoding and decoding different types of features, especially suitable for multi-modal feature fusion. In the embodiments of the present application, ICA is used to process and fuse different types of Smali instruction features, reduce feature redundancy, and increase the distinguishability and relevance between features.

[0038] In the current Internet technology and security field, especially for the application distribution and security detection of the Android platform, the related technology has encountered significant challenges in identifying and detecting repackaged APK files (i.e. illegally developed application package files). With the rapid expansion of the mobile application market, third-party platforms and unofficial application stores have become an important channel for users to obtain applications. However, the lack of strict application release management mechanism enables repackaged APK files to be widely distributed. These repackaged APK files threaten user privacy and property safety by rewriting Smali code in the original APK file and implanting malicious code, and also infringe the rights and interests of legitimate developers.

[0039] The similarity analysis method of the APK file adopted by the related technology mainly relies on extracting the function call graph in the code file as a feature for comparison, but its effectiveness is limited when facing complex or obfuscated code. The function call graph only considers the calling relationship of functions in the code, ignoring the details of the code, and illegal developers will change the structure of the original APK file when repackaging the application, including modifying class names, function names, or adjusting function call sequences, which makes it difficult to directly compare the Smali code of the APK file before and after repackaging based on the similarity analysis method of the function call graph.

[0040] To solve the above technical problems, the embodiments of the present application provide corresponding solutions, which are described in detail below.

[0041] The file similarity determination method embodiments provided by the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 A hardware structure block diagram of a computer terminal for implementing the file similarity determination method is shown. As shown in Figure 1 The computer terminal 10 can include one or more processors (processors can include but are not limited to processing devices such as microprocessors MCU or programmable logic devices FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected through wired and / or wireless networks. In addition, it can also include a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a BUS bus. Those skilled in the art can understand, Figure 1 The structure shown is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 can also include more or fewer components than those shown in Figure 1 the above-mentioned electronic device, or have different structures from those shown in the above-mentioned electronic device.Figure 1 different configurations.

[0042] It should be noted that the one or more processors and / or other data processing circuitry described above can be generally referred to herein as "data processing circuitry". The data processing circuitry can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any of the other elements of the computer terminal 10. As referred to in the embodiments of the present application, the data processing circuitry functions as a processor to control, for example, the selection of the variable resistance terminal path in connection with the interface.

[0043] The memory 104 can be used to store software programs of application software and modules, such as program instructions / data storage means corresponding to the file similarity determination method in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, i.e. implements the file similarity determination method described above. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory disposed remotely with respect to the processor, which can be connected to the computer terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0044] The transmission module 106 is configured to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network interface controller (NIC) that can be connected to other network devices through a base station so as to be able to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0045] The display can be, for example, a touch screen type liquid crystal display (LCD) that enables a user to interact with the user interface of the computer terminal 10.

[0046] It should be noted that in some alternative embodiments, the above Figure 1 The computer terminal shown can include hardware elements (including circuitry), software elements (including computer code stored on a computer readable medium), or a combination of both hardware and software elements. It should be noted that in some embodiments, the functions of the computer terminal described above can be combined in a single module or implemented in a distributed manner over multiple modules. Figure 1This is merely one instance of a specific, concrete example and is intended to illustrate the types of components that can be present in the computer terminal described above.

[0047] Under the above operating environment, the embodiment of the present application provides a file similarity determination method embodiment. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0048] Figure 2 is a flowchart of a file similarity determination method according to an embodiment of the present application, as shown in Figure 2 The method comprises the following steps:

[0049] Step S202, obtaining byte code text files corresponding to the original file and the target file, wherein the target file comprises a file obtained by rewriting and repacking the original file, and the byte code text file comprises a file obtained by respectively decompiling the byte code files of the original file and the target file.

[0050] In the above step S202, the original file refers to the APK file version of the application program or software that has not been modified and is initially published by the developer, and the target file refers to the APK file obtained by rewriting (such as inserting malicious code, modifying logic, etc.) and repacking the original file.

[0051] The byte code text file is a text representation extracted from the byte code files of the original file and the target file based on the decompilation operation, such as a Smali file presented in Smali language format. In some embodiments of the present application, the original file and / or the target file correspond to a plurality of byte code text files respectively, and each byte code text file corresponds to a class.

[0052] In some embodiments of the present application, the original APK file and the target APK file can be decompiled using a decompilation tool (such as baksmali, dex2jar, etc.) to convert the Dex byte code into a Smali language text format. In addition, the generated Smali code can be preprocessed, including but not limited to decommenting, formatting the code structure, and possibly removing irrelevant code segments (such as known non-critical system library code) to more accurately extract the instruction set features reflecting the core functions and logic of the application program.

[0053] In the case of repackaging, since the application may be obfuscated and modified by illegal developers, direct comparison of source code or binary form of bytecode files often cannot effectively detect these changes, and by decompiling the bytecode file into Smali code, the underlying logic of the code can be accessed, and even in the case of obfuscated class names and function names, potential modifications can be identified through instruction set characteristics.

[0054] Figure 4 is an application package file (APK file) according to an embodiment of the file similarity determination method, as shown in Figure 4 , including:

[0055] (1) assets: an optional directory for storing raw data files, such as multimedia files, databases, or JSON configuration files, which are accessed in the application in the form of a stream.

[0056] (2) lib: for storing native library files required by the application, such as so (shared object) files written in C or C++.

[0057] (3) META-INF: contains metadata information of the application, such as digital signature and license file.

[0058] (4) res: stores resource files of the application, including layout files, images, animations, sounds, and fonts, etc.

[0059] (5) AndroidManifest.xml: the manifest file of the application, which lists the application name, permission requirements, Activity, Service, BroadcastReceiver, and other component information.

[0060] (6) classes.dex: the main execution part of the APK file, containing the Dalvik bytecode of the application. The Dex file is the compiled result of Smali code, converted from Java source code, and is a code file that can be directly executed by the Android runtime environment (Dalvik virtual machine or ART).

[0061] (7) resources.arsc: binary storage format of resource files, containing references to various resources used by the application, such as strings, colors, and dimensions, etc.

[0062] Specifically, the Android reverse analysis technology is to decompile the APK file, use the code generated by decompilation and related resources to analyze the application, and use apktool to decompile to get the reverse file, including Manifest, resources, original, etc. and Smali file. Smali file is decompiled from classes.dex, which contains the code content after decompilation. Smali file is generated after APK file is decompiled by apktool tool, and the architecture of the file generated by decompilation is consistent with the original application code file. The developer's custom class, third-party library class in the application or class in the application framework all correspond to a separate ".Smali" file. The code in the Smali file contains various functionally different operation codes to correspond to the Java code.

[0063] In order to solve the efficiency problem when processing a large number of Smali code files, the following steps can also be performed: sorting the multiple bytecode text files corresponding to the original files in descending order of file size, and deleting a preset proportion of files in the bytecode text file sequence in the order from front to back, to obtain the target bytecode text file.

[0064] Specifically, first, the size of all Smali files obtained by decompiling the original APK file is obtained, then sorted in descending order of size, and the first 5% (only as an example) of the sorted bytecode text file sequence is deleted. This operation greatly speeds up the file processing time and has little effect on the final similarity calculation result.

[0065] During the sorting and deleting process, there is a risk that key features may be missed, especially when the core logic of the application is distributed in the 5% largest files that are deleted, which may negatively affect the accuracy of subsequent similarity analysis. In order to solve this problem, the following steps can also be performed:

[0066] (1) Define a set of key instructions: this set focuses on instructions that have a significant impact on application logic and functionality, such as data manipulation instructions, data operation instructions, comparison instructions, method call instructions, etc. These instructions are more likely to be modified by repackers to implant malicious code or change application behavior;

[0067] (2) Calculate instruction density: for each Smali file obtained by decompilation, calculate the density of key instructions appearing therein, thereby indirectly assessing the degree of contribution of the file to the application functionality;

[0068] (3) Set a density threshold: according to the characteristics of the analysis target and application type, set a threshold for the density of key instructions to distinguish which files contain a large number of key instructions;

[0069] (4) Reordering and filtering: Based on the calculated instruction density, all Smali files are reordered. In this case, the order is no longer based on file size, but on instruction density. That is, files with high density and more critical to application functions are prioritized. Files are then filtered based on a density threshold, and only those with density above the threshold are retained, even if they are in the top 5% by size.

[0070] Step S204, determining the original feature vector corresponding to the original file and the target feature vector corresponding to the target file respectively according to the instruction set in the bytecode text file, wherein the instruction set is used to determine the running logic of the code in the bytecode text file.

[0071] In step S204, the instruction set refers to a set of opcodes that appear in the Smali file and have a decisive effect on the code execution logic. These instructions cover functions such as data manipulation, data calculation, comparison, and method call.

[0072] The original feature vector is a mathematical representation built based on the instruction set of the Smali code in the original file, which contains quantitative information related to the code function, structure and logic. The target feature vector is similar to the original feature vector, but it comes from the target file, that is, the APK file that may be repackaged.

[0073] In some embodiments of the present application, when converting the instruction set extracted from the bytecode text file into a feature vector, a one-hot encoding technique can be used, that is, a specific position is assigned to each key instruction. If a certain instruction appears in a Smali file, the corresponding position is marked as 1, otherwise it is marked as 0. It should be noted that, taking into account the different importance of different instructions to the code logic, each one-hot encoded feature vector can also be weighted, and the weight value is calculated based on the global frequency of the instruction, which means that instructions that appear more frequently in the original APK file will receive a higher weight.

[0074] Related techniques for APK file similarity analysis use a method that extracts the function call graph from the code as a feature for similarity calculation. However, this method does not fully consider the details of the code. This is because secondary developers rewrite function calls in the code when repackaging APK files, causing the function call graph to change. Therefore, this method is difficult to accurately assess the similarity of APK file pairs that have similar functions but slight differences in code implementation details.

[0075] The Smali instruction set as the underlying bytecode not only reflects various details of the Smali code, but also determines the running logic of the Smali code. Even if the secondary developer repackages the APK file, the instruction set will not change basically. The Smali instruction set is various, and its types include but are not limited to data operation instructions, data definition instructions, data operation instructions, comparison instructions, field operation instructions, method call instructions, and the like. Since the secondary developer mainly involves data operation instructions, data operation instructions, comparison instructions, and method call instructions in the process of repackaging the APK file, in some embodiments of the present application, feature extraction can be performed on these instructions, and one-hot coding can be performed on each type of instruction (for repeated instructions, multiple times of extraction). For example, by traversing each line of code in the Smali file, the instructions in the Smali file are extracted and stored in the one-hot coding manner.

[0076] The original feature vector can be determined in the following manner: extracting a first instruction set from the code in the first bytecode text file corresponding to the original file, and classifying all instructions in the first instruction set according to instruction types; for each first instruction subset corresponding to an instruction type, using an inter-class autoencoder corresponding to the instruction type to encode the first instruction subset to obtain a first feature vector, wherein the inter-class autoencoder is used to learn the relevance between the first instructions in the first instruction subset; and performing feature aggregation on the first feature vectors respectively corresponding to all instruction types to obtain the original feature vector.

[0077] The first bytecode text file refers to the Smali bytecode file obtained by decompiling the original APK file, the first instruction set includes a set of all key instructions extracted from the first bytecode text file, and the first instruction subset is a small set divided from the first instruction set according to different instruction types, such as all data operation instructions forming a subset and all method call instructions forming another subset.

[0078] The first feature vector refers to a mathematical representation obtained by encoding the first instruction subset by the inter-class autoencoder for each instruction type, which is used to quantify the features of the instruction type in the code. For example, by using feature aggregation techniques such as weighted average, splicing or more complex neural network structures (such as attention mechanism), the feature information scattered in different instruction types is integrated together to form a comprehensive description of the features of the entire APK file code.

[0079] Similarly, the target feature vector can be determined in the following manner: extracting a second instruction set from the code of a second bytecode text file corresponding to the target file, and classifying all instructions in the second instruction set according to instruction type; for a second instruction subset corresponding to each instruction type, encoding the second instruction subset using an inter-class autoencoder corresponding to the instruction type to obtain a second feature vector, wherein the inter-class autoencoder is used to learn the correlation between the second instructions in the second instruction subset; and performing feature aggregation on the second feature vectors corresponding to all instruction types to obtain a target feature vector.

[0080] The determination of the target feature vector can refer to the determination of the original feature vector described above, which will not be repeated here.

[0081] Figure 5 is a schematic diagram of multi-category feature aggregation of a method for determining file similarity according to an embodiment of the present application, such as Figure 5 As shown, different categories of instruction features are first extracted from the Smali code file, such as data operation instructions, data calculation instructions, method call instructions, etc. Each category of instruction features represents a specific aspect of code operation. Next, these extracted instruction features are independently encoded using inter-class autoencoders that match each category of instructions. These autoencoders are specially trained to learn and understand the internal correlation of the category of instructions and their importance in the code, thereby mapping the original instruction features to a more compact and information-rich intermediate layer representation. After encoding, the multiple intermediate layer feature representations are aggregated (i.e., integrated). This integration process uses an efficient algorithm, such as weighted averaging or feature splicing, to ensure that the final feature vector contains comprehensive information of all categories of instructions. It also takes into account the actual weight of each category of instructions in the code, and finally obtains a feature vector that can fully reflect the logic and structure of the Smali code of the original APK file.

[0082] In some embodiments of the present application, the inter-class autoencoder is trained by: obtaining a first training feature vector and a second training feature vector, wherein the first training feature vector comprises an instruction feature vector extracted from an original file training set, and the second training feature vector comprises an instruction feature vector of the same instruction type as the first training feature vector extracted from a target file training set; determining a training sample pair corresponding to the first training feature vector and the second training feature vector, and determining a label of each training sample pair, wherein the label is used to quantitatively represent the degree of association between the first training feature vector and the second training feature vector in the training sample pair; using an initial inter-class autoencoder to encode the first training feature vector in the training sample to obtain an encoding result; determining a predicted label according to the encoding result and the second training feature vector, wherein the predicted label is used to reflect the predicted degree of association between the first training feature vector and the second training feature vector; determining a loss according to the encoding result and the predicted label, and iteratively updating parameters of the initial inter-class autoencoder according to the loss, and stopping iteration under the condition that a stop condition is met, to obtain the inter-class autoencoder.

[0083] Because there are multiple categories of instruction sets in Smali code, such as invoke instructions, move instructions, and const instructions, etc., in some embodiments of the present application, an inter-class autoencoder can be trained for each category of instruction set to increase the distinction between different categories of instructions, and after encoding each category of instruction feature, the intermediate layer output obtained is aggregated to obtain a final feature vector for calculation. It should be noted that by introducing the training method of sample pairs, the output of the inter-class autoencoder can be guided to be more consistent with the expected task, and the use of the inter-class autoencoder can reduce the dimension of the extracted category feature vector, and through training, the inter-class autoencoder can also learn the relationship between instructions of the same category.

[0084] Specifically, the first training feature vector is a key instruction feature vector extracted from the Smali code of the original APK file, which is used to train the inter-class autoencoder to understand the features and structures of normal code. The second training feature vector corresponding to the first training feature vector is an instruction feature vector of the same instruction type extracted from the target APK file (which can be legal or repackaged), and the second training feature vector is used to compare with the first training feature vector in the training sample pair to help the inter-class autoencoder learn how to distinguish between normal and repackaged code.

[0085] The initial inter-class autoencoder is used to encode the first training feature vector in the training sample to obtain an encoding result: the encoder of the initial inter-class autoencoder is used to encode the first training feature vector to obtain a first encoding result, wherein the first encoding result includes an intermediate representation after dimensionality reduction of the first training feature vector; the decoder of the initial inter-class autoencoder is used to decode the first encoding result to obtain a second encoding result, wherein the second encoding result is used to reconstruct the first training feature vector on the basis of the first encoding result; and the first encoding result and the second encoding result are determined as the encoding result.

[0086] The loss is determined according to the encoding result and the predicted label: a first loss corresponding to the second encoding result and the first training feature vector is determined, and a second loss corresponding to the predicted label and the label is determined, to obtain a loss corresponding to the first loss and the second loss.

[0087] Specifically, in the training process of the inter-class autoencoder, a sample pair (f, g) is introduced and labeled as s, where f is the fused key feature (i.e., the first training feature vector), g is the contrast feature (i.e., the second training feature vector), and s reflects the correlation degree of f and g. The higher the correlation degree, the larger s. It should be noted that some positive sample pairs with larger s and negative sample pairs with smaller s can be introduced to train the model, so that the model can better understand the relationship between the features. The loss function of the inter-class autoencoder consists of two parts: the mean square error between the decoding output z (i.e., the second encoding result) and the fused feature f, and the mean square error between the label s and the neural network output S (i.e., the predicted label).

[0088] For feature fusion of key instruction codes, a set of extracted instruction features is as follows:

[0089] F={f1,f2,...,f n} (Formula 1)

[0090] Where F is the initial fused feature (initial first training feature vector), f i represents various types of instruction features extracted.

[0091] Considering the difference in importance of various instructions in each instruction class, for example, the number of uses of the invoke-virtual instruction is significantly more than that of the invoke-else instruction, a weight coefficient can be assigned to each instruction. After preliminary fusion, a set of instruction features is as follows:

[0092] f=α1*f1+α2*f2+...+α n *f n (Formula 2)

[0093] Among them, f is the weighted fusion of various features (the first training feature vector), α i Represents the weights of various instruction features, α i The value of can be obtained, for example, by counting the occurrence frequencies of various instructions in the original APK file.

[0094] Then, the fused instruction feature f is input into the inter-class autoencoder to obtain the intermediate layer output (i.e., the first encoding result). Finally, the intermediate layer output F obtained by all categories of instruction features is obtained. i Aggregation is performed to obtain the feature vector for calculation.

[0095] Figure 6 This is a flow chart of inter-class autoencoder training according to a method for determining file similarity according to an embodiment of the present application, such as Figure 6 As shown in the figure, the rectangle on the left represents the original multi-class instruction feature input (i.e., fusion feature f). These features are instruction sets that have been preliminarily processed and are classified into different instruction types, such as invoke, move, const, etc. Each type of instruction feature will enter a corresponding rectangle (referring to the encoding stage of the inter-class autoencoder). The autoencoder will learn the internal connections of the instruction set and convert it into a low-dimensional intermediate representation (i.e., the intermediate layer output F). The intermediate low-dimensional feature vector is then sent to the decoding part of the autoencoder, attempting to reconstruct the input instruction features (i.e., the decoding output z). The reconstructed features are compared with the original input instruction features, and the difference between the two is calculated, which is the loss function (not directly shown in the figure). The loss function not only comes from the error of decoding and reconstruction, but also involves the error of sample label prediction, that is, Figure 6 As shown, the contrast feature g is input into the neural network for prediction to obtain the association degree S (ie, the predicted label).

[0096] In some embodiments of the present application, the following steps can also be performed: determining the subtype of the target instruction contained in each bytecode text file, and assigning a value to each subtype according to preset attributes; determining the keyword of the bytecode text file in the storage structure based on the attribute values ​​corresponding to all subtypes of each bytecode text file; and storing the feature vector corresponding to each bytecode text file based on the keyword.

[0097] Because repackaging changes the names of various classes and functions in the APK file's Smali code, the one-to-one correspondence between the original and repacked Smali files is lost, making direct similarity comparison difficult. For example, a Smali file named "Send Email" in the original APK file might be renamed "Send Message" after repackaging, making direct similarity comparison impossible. Therefore, hash storage technology is used to store the resulting feature vectors.

[0098] It should be noted that the target instruction can include a call instruction, wherein the subtypes of the call instruction include at least one of the following: static method call instruction, virtual method call instruction, direct construction method call instruction, superclass method call instruction, interface method call instruction, polymorphic method call instruction, and custom method call instruction.

[0099] Since there are almost always invoke instructions (i.e., call instructions) in each Smali file, the invoke instruction type can be used as the key value in the hash storage technique to store the final feature vector. Specifically, there are six types of invoke instructions, namely invoke-static, invoke-virtual, invoke-direct, invoke-super, invoke-interface, and invoke-else, and each type can be assigned an "A", "B", "C", "D", "E", "F" attribute. The corresponding relationship between the assigned instructions and attributes is shown in Table 1:

[0100] Table 1: Corresponding attributes of various invoke instructions.

[0101] invoke instruction category attribute

[0102]

[0103] The code snippets in the Smali file are traversed to query the occurrence frequency of the invoke instructions, and then the key is generated according to the attributes of each type. The attributes are then de-duplicated and sorted by ASCII code value. The purpose of de-duplicating the attributes to generate the key value is to consider the possibility that the number of certain invoke instructions may change after repackaging with obfuscation techniques. For example, there are two invoke-static instructions in the original APK file in the Smali code segment, and after repackaging, there is only one invoke-static instruction in the Smali code segment. Therefore, the de-duplication operation on the attributes has a certain degree of anti-obfuscation.

[0104] Step S206, according to the similarity between the original feature vector and the target feature vector, determine the similarity between the original file and the target file.

[0105] In the above step S206, the original feature vector and the target feature vector can be input into the self-learning weight composite similarity calculation formula for comparison. Since the feature vector has been optimized by the inter-class autoencoder, the comparison here can more accurately reflect the logical structure of the code rather than superficial changes in the code name.

[0106] For one target APK file, there can be multiple Smali files, so for each Smali file, the similarity between the generated feature vector and the feature vector of all Smali files in the original APK file needs to be calculated, and then the similarity values are aggregated (such as taking the average or maximum value) to obtain the overall similarity between the target APK and the original APK.

[0107] In some embodiments of the present application, the similarity between the original feature vector and the target feature vector can be determined by: determining the feature distribution similarity of the respective instruction sets in the high-dimensional feature space between the original feature vector and the target feature vector to obtain a first similarity; determining the degree of overlap between the respective instruction sets in the original feature vector and the target feature vector to obtain a second similarity; and determining the similarity according to the first similarity and the second similarity.

[0108] After storing the feature vectors (including the original feature vector and the target feature vector) with the key value, the next step is to calculate the similarity between the feature vectors. In order to avoid the limitations of a single calculation method and the situation that the information of the sample cannot be accurately and comprehensively reflected, a self-learning weight compound similarity calculation formula can be used, that is, two vector similarity calculation formulas are used and different weights are assigned to them, and the weights are obtained by gradient descent method. It should be noted that in the training process of the weight, some sample pairs can be introduced, and the sample pairs are the feature vectors extracted from the original Smali file and the feature vectors extracted from the corresponding repackaged Smali file, and the label is the similarity degree.

[0109] Specifically, cosine similarity and Jaccard similarity can be used to calculate the similarity between two feature vectors. Cosine similarity reflects the similarity between two vectors by calculating the angle between the two vectors. The larger the cosine similarity, the smaller the angle between the two vectors, and the higher the similarity. Jaccard similarity measures the similarity between two vectors by calculating the percentage of the number of identical elements in the two vectors to the number of elements in the union of the two vectors. The larger the Jaccard similarity, the closer the elements of the two vectors, and the higher the similarity. The final vector similarity is obtained by weighting and fusing the two similarities (the specific process is shown in Figure 7 ).

[0110] For two vectors X (original feature vector, i.e. the original Smali feature in Figure 7 ) and Y (target feature vector, i.e. the repackaged Smali feature in Figure 7 ), the cosine similarity (i.e. the first similarity) is defined as follows:

[0111]

[0112] The Jaccard similarity (i.e., the second similarity) is defined as follows:

[0113]

[0114] The composite similarity sim of two feature vectors is defined as follows:

[0115] sim(X, Y) = a * cos(X, Y) + β * Jaccard(X, Y) (Equation 5)

[0116] where a and β represent the weight parameters of the two similarities. Since the sizes of the two weight parameters are not easy to set, they can be determined by a self-learning method, as follows:

[0117] 1) First, initialize a and β to 0.5.

[0118] 2) Introduce some sample pairs, which are the feature vectors extracted from the original Smali files and the feature vectors extracted from the corresponding repackaged Smali files, and the labels are the similarity degrees.

[0119] 3) Calculate the cosine similarity and Jaccard similarity of these sample pairs, and obtain the similarity by weighted summation of a and β.

[0120] 4) Calculate the loss using the mean square error function.

[0121] 5) Update a and β using the gradient descent method.

[0122] In some embodiments of the present application, the similarity between the original feature vector and the target feature vector can be determined by the following method: traversing the first keywords corresponding to the target file, and comparing the first keywords with all the second keywords corresponding to the original file in turn; in the case where the comparison result indicates that there is a target keyword in the second keyword which is the same as the first keyword, calculating the similarity between the feature vector corresponding to the target keyword and the feature vector corresponding to the first keyword to obtain a similarity value; determining the similarity between the original feature vector and the target feature vector according to the similarity value.

[0123] The first keyword represents the type of Smali code invoke instruction in the target file, such as A (invoke-static), B (invoke-virtual), AB, etc., and is used to locate a specific type of feature vector in the hash storage. The second keyword represents the type of Smali code invoke instruction in the original file. For example, the first keyword in the target file is traversed, and the hash storage of the original file is searched for a matching second keyword. When a matching second keyword is found, the feature vector corresponding to the first keyword is extracted from the original file and compared with the feature vector in the target file using a composite similarity calculation formula to obtain a similarity value.

[0124] Specifically, combined Figure 8 The similarity calculation process diagram can traverse the key values ​​of the repackaged APK file (such as key1, key2, key3...), find the corresponding key value in the original APK file, and then sequentially extract the feature vectors in the corresponding key value of the repackaged APK file and all the feature vectors in the corresponding key value of the original APK file (such as Figure 8 The similarity is calculated for f1, f2, f3, f4, etc. pointed to by key1 shown in , and multiple similarities are obtained, such as sim1, sim2, sim3, etc., from which the maximum similarity value is taken. If the maximum value is greater than a preset threshold (such as 0.7), the two feature vectors are considered similar, that is, the Smali files represented by the two feature vectors are basically similar.

[0125] It should be noted that the following steps can also be performed: when it is determined based on the similarity value that the feature vector corresponding to the target keyword is similar to the feature vector corresponding to the first keyword, the feature vector corresponding to the target keyword is marked, wherein the mark is used to indicate that the feature vector corresponding to the target keyword is no longer subject to similarity calculation.

[0126] During the calculation process, dynamic optimization can also be introduced. That is, when a feature vector in the original APK file is calculated to be similar to a feature vector in the repackaged APK file, it is marked. Each subsequent similarity calculation will no longer involve the marked feature vector. This operation reduces the amount of calculation and greatly improves the program running speed.

[0127] Through steps S202 to S206, the similarity of the instruction sets in the bytecode text file is analyzed, the similarity of the code running logic in the original application package file and the target application package file is compared, the purpose of accurately identifying the repackaged file is achieved, the technical effect of accurately detecting similar applications in the case of complex code is achieved, and the technical problem of low similarity evaluation accuracy caused by using the function call graph of the code to analyze the similarity of the application package file in the related art is solved.

[0128] Figure 3 is a whole flowchart of a file similarity determination method according to an embodiment of the present application, as shown in some embodiments of the present application, the following steps can be included: Figure 3

[0129] Step 1: decompile the APK file to obtain Smali code.

[0130] Use tools such as apktool to decompile the original APK file and the target APK file to obtain a file set containing Smali code. During the decompilation process, since part of the file may be too large, affecting the processing efficiency, a preprocessing measure can be taken to sort the files by size and discard the first 5% of the files to optimize the calculation speed.

[0131] Step 2: feature extraction of the instruction set of Smali code.

[0132] Iterate through each Smali file, extract the key instruction set such as invoke, move, const, etc., and one-hot encode each instruction to retain the detailed features of the instruction. Considering the importance difference of different instructions, the frequency of the instruction in the code can be counted to assign a weight to it.

[0133] Step 3: multi-instruction set feature fusion based on inter-class autoencoder.

[0134] For each type of instruction feature extracted, an inter-class autoencoder is trained, which can learn the internal relationship between the instruction features and perform dimension reduction and fusion on the features. The training of the autoencoder introduces positive and negative sample pairs, and the mean square error between the decoding output and the fused features and the mean square error between the sample pair labels and the neural network output are calculated to iteratively update the model parameters.

[0135] Step 4: composite similarity calculation.

[0136] ​The invoke instruction type in the Smali code is used to generate a key value, and the fused feature vector is hashed and stored. This allows the feature vector to be matched based on the invoke instruction type, even if the code is obfuscated. Subsequently, a composite formula of cosine similarity and Jaccard similarity is used to calculate the similarity of the feature vectors using self-learning weights (α and β). The weight parameters are iteratively updated using gradient descent, ultimately resulting in a similarity value that accurately reflects the sample information.

[0137] Figure 9 is a structural diagram of a device for determining file similarity according to an embodiment of the present application, such as Figure 9 As shown, the device includes:

[0138] An acquisition module 902 is configured to acquire bytecode text files corresponding to an original file and a target file, wherein the target file comprises a file obtained by rewriting and repackaging the original file, and the bytecode text file comprises a file obtained by decompiling the bytecode files of the original file and the target file, respectively;

[0139] A determination module 904 is configured to determine an original feature vector corresponding to the original file and a target feature vector corresponding to the target file based on an instruction set in the bytecode text file, wherein the instruction set is used to determine the execution logic of the code in the bytecode text file;

[0140] The comparison module 906 is configured to determine the similarity between the original file and the target file based on the similarity between the original feature vector and the target feature vector.

[0141] It should be noted that Figure 9 The device for determining the file similarity shown is used to perform Figure 2 The method for determining the similarity of files shown is, therefore Figure 2 The explanations in the method for determining the similarity of files in also apply to Figure 9 The device for determining the file similarity shown will not be described in detail here.

[0142] An embodiment of the present application further provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the steps of the method for determining file similarity in each embodiment of the present application.

[0143] An embodiment of the present application further provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the steps of the method for determining file similarity in each embodiment of the present application by running the computer program.

[0144] The embodiment of the present application further provides a computer program product comprising computer instructions which, when executed by a processor, implement the steps of the file similarity determination method in the various embodiments of the present application.

[0145] The embodiment of the present application further provides a computer program which, when executed by a processor, implements the steps of the file similarity determination method in the various embodiments of the present application.

[0146] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0147] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0148] In the several embodiments provided by the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit described as the division is only a logical division, and there can be other division manners in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0149] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0150] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0151] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0152] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method for determining file similarity, characterized in that: include: Obtaining bytecode text files corresponding to an original file and a target file, wherein the target file comprises a file obtained by rewriting and repackaging the original file, and the bytecode text file comprises a file obtained by decompiling the bytecode files of the original file and the target file, respectively; Determining an original feature vector corresponding to the original file and a target feature vector corresponding to the target file based on an instruction set in the bytecode text file, wherein the instruction set is used to determine the running logic of the code in the bytecode text file; The similarity between the original file and the target file is determined based on the similarity between the original feature vector and the target feature vector.

2. The method according to claim 1, characterized in that Determining an original feature vector corresponding to the original file according to an instruction set in the bytecode text file includes: Extracting a first instruction set from the code of a first bytecode text file corresponding to the original file, and classifying all instructions in the first instruction set according to instruction type; For a first instruction subset corresponding to each instruction type, encoding the first instruction subset using an inter-class autoencoder corresponding to the instruction type to obtain a first feature vector, wherein the inter-class autoencoder is used to learn the correlation between the first instructions in the first instruction subset; The first feature vectors corresponding to all instruction types are subjected to feature aggregation to obtain the original feature vector.

3. The method according to claim 1, characterized in that Determining a target feature vector corresponding to the target file according to an instruction set in the bytecode text file includes: Extracting a second instruction set from the code of a second bytecode text file corresponding to the target file, and classifying all instructions in the second instruction set according to instruction type; For each second instruction subset corresponding to the instruction type, encode the second instruction subset using an inter-class autoencoder corresponding to the instruction type to obtain a second feature vector, wherein the inter-class autoencoder is used to learn the correlation between the second instructions in the second instruction subset; The second feature vectors corresponding to all instruction types are subjected to feature aggregation to obtain the target feature vector.

4. The method according to claim 2 or 3, characterized in that The inter-class autoencoder is trained in the following way: Obtaining a first training feature vector and a second training feature vector, wherein the first training feature vector includes an instruction feature vector extracted from an original file training set, and the second training feature vector includes an instruction feature vector extracted from a target file training set and having the same instruction type as the first training feature vector; Determining training sample pairs corresponding to the first training feature vector and the second training feature vector, and determining a label for each of the training sample pairs, wherein the label is used to quantitatively represent the degree of association between the first training feature vector and the second training feature vector in the training sample pair; Encoding the first training feature vector in the training sample using an initial inter-class autoencoder to obtain an encoding result; Determining a prediction label based on the encoding result and the second training feature vector, wherein the prediction label is used to reflect the degree of prediction correlation between the first training feature vector and the second training feature vector; A loss is determined according to the encoding result and the predicted label, and parameters of the initial inter-class autoencoder are iteratively updated according to the loss. When a stopping condition is met, the iteration is stopped to obtain the inter-class autoencoder.

5. The method according to claim 4, characterized in that Encoding the first training feature vector in the training sample using an initial inter-class autoencoder to obtain an encoding result, including: Encoding the first training feature vector using the encoder of the initial inter-class autoencoder to obtain a first encoding result, wherein the first encoding result includes an intermediate representation of the first training feature vector after dimensionality reduction; Decoding the first encoding result using a decoder of the initial inter-class autoencoder to obtain a second encoding result, wherein the second encoding result is used to reconstruct the first training feature vector based on the first encoding result; The first encoding result and the second encoding result are determined as the encoding result.

6. The method according to claim 5, characterized in that Determining a loss based on the encoding result and the predicted label includes: determining a first loss corresponding to the second encoding result and the first training feature vector, and determining a second loss corresponding to the predicted label and the label, to obtain the loss corresponding to the first loss and the second loss.

7. The method according to claim 1, characterized in that The original file and / or the target file respectively correspond to a plurality of bytecode text files, each of the bytecode text files corresponds to a class; the method further comprises: Determining the subtype of the target instruction contained in each of the bytecode text files, and assigning a value to each of the subtypes according to a preset attribute; Determining a keyword of the bytecode text file in a storage structure according to attribute values ​​corresponding to all subtypes of each bytecode text file; The feature vector corresponding to each of the bytecode text files is stored according to the keyword.

8. The method according to claim 7, characterized in that The similarity between the original feature vector and the target feature vector is determined by: Traversing the first keyword corresponding to the target file, and sequentially comparing the first keyword with all the second keywords corresponding to the original file; If the comparison result indicates that the second keyword contains a target keyword identical to the first keyword, performing similarity calculation between a feature vector corresponding to the target keyword and a feature vector corresponding to the first keyword to obtain a similarity value; The similarity between the original feature vector and the target feature vector is determined according to the similarity value.

9. The method according to claim 8, characterized in that The method further includes: when it is determined based on the similarity value that the feature vector corresponding to the target keyword is similar to the feature vector corresponding to the first keyword, marking the feature vector corresponding to the target keyword, wherein the mark is used to indicate that the feature vector corresponding to the target keyword is no longer subject to similarity calculation.

10. The method according to claim 1, characterized in that The similarity between the original feature vector and the target feature vector is determined by: Determining feature distribution similarities of instruction sets corresponding to the original feature vector and the target feature vector in a high-dimensional feature space to obtain a first similarity; Determining the degree of overlap between the instruction sets respectively corresponding to the original feature vector and the target feature vector to obtain a second similarity; The similarity is determined according to the first similarity and the second similarity.

11. The method according to claim 1, characterized in that The method also includes: sorting the multiple bytecode text files corresponding to the original file in descending order of file size, and deleting a preset proportion of files in the bytecode text file sequence in order from front to back to obtain a target bytecode text file.

12. A device for determining file similarity, characterized in that: include: an acquisition module, configured to acquire bytecode text files corresponding to an original file and a target file, wherein the target file comprises a file obtained by rewriting and repackaging the original file, and the bytecode text file comprises a file obtained by decompiling the bytecode files of the original file and the target file, respectively; a determination module, configured to determine, based on an instruction set in the bytecode text file, an original feature vector corresponding to the original file and a target feature vector corresponding to the target file, wherein the instruction set is used to determine the operation logic of the code in the bytecode text file; The comparison module is configured to determine the similarity between the original file and the target file based on the similarity between the original feature vector and the target feature vector.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the method for determining file similarity according to any one of claims 1 to 11.

14. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the method for determining file similarity according to any one of claims 1 to 11 by running the computer program.

15. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method for determining the file similarity according to any one of claims 1 to 11 is implemented.

Images