A program similarity analysis method, device and electronic equipment

Abstract

Embodiments of the present application provide a program similarity analysis method, device and electronic equipment, the method comprising: performing structural analysis and semantic normalization processing on instructions of a first program sample and a second program sample, constructing a first instruction semantic feature sequence and a second semantic feature sequence reflecting local behavior characteristics of the program; based on the first instruction semantic feature sequence and the second semantic feature sequence, generating a first multiple set and a second multiple set of instruction semantic segment features at a class level granularity; determining the number of similar instruction semantic segments of the first multiple set and the second multiple set; taking the first program sample and the second program sample corresponding to the first type of effective instructions and the second type of effective instructions as a benchmark, determining a first class level similarity index and a second class level similarity index based on the number of similar instruction semantic segments; and determining the similarity of the first program sample and the second program sample at the behavior structure level based on the first class level similarity index and the second class level similarity index.

Description

Technical Field

[0001] This application relates to the field of program analysis technology, specifically to a program similarity analysis method, apparatus, and electronic device. Background Technology

[0002] In related technologies, program similarity analysis is based on text content segmentation, semantic matching, or document similarity, or employs dynamic programming algorithms based on global alignment of instruction sequences. The similarity analysis process requires constructing a matching matrix or backtracking path for instruction alignment, which introduces uncertainties and errors during decompilation due to heuristic processing such as control flow reconstruction, variable recovery, and abnormal structure inference. Therefore, improving the accuracy of program similarity analysis, reducing its computational complexity, and minimizing uncertainties have become urgent technical challenges. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, and electronic device for program similarity analysis, and the specific technical solution adopted is as follows: In a first aspect, embodiments of this application provide a program similarity analysis method, the method comprising: The instructions of the first program sample and the second program sample are subjected to structured parsing and semantic normalization to construct a first instruction semantic feature sequence and a second semantic feature sequence that can reflect the local behavioral characteristics of the program. Based on the first instruction semantic feature sequence and the second semantic feature sequence, a first multiset and a second multiset, including multiple instruction semantic fragment features, are generated at the class-level granularity. Determine the number of similar instruction semantic fragments between the first multiset and the second multiset; Based on the first type of valid instructions and the second type of valid instructions corresponding to the first program sample and the second program sample, respectively, a first-class similarity index and a second-class similarity index are determined based on the number of similar instruction semantic segments. The first type of valid instructions and the second type of valid instructions are determined based on the function of the instructions. The similarity between the first program sample and the second program sample at the behavioral structure level is determined based on the first class-level similarity index and the second class-level similarity index.

[0004] Secondly, a program similarity analysis apparatus is provided, the apparatus comprising: The parsing and normalization module is used to perform structured parsing and semantic normalization on the instructions of the first program sample and the second program sample, and to construct a first instruction semantic feature sequence and a second semantic feature sequence that can reflect the local behavioral characteristics of the program. A multiset generation module is used to generate a first multiset and a second multiset, which include multiple instruction semantic fragment features, at the class-level granularity based on the first instruction semantic feature sequence and the second semantic feature sequence, respectively. The first determining module is used to determine the number of similar instruction semantic fragments between the first multiset and the second multiset; The second determining module is used to determine a first-class similarity index and a second-class similarity index based on the number of similar instruction semantic segments, using the first-class and second-class valid instructions corresponding to the first program sample and the second program sample as benchmarks, respectively. The first-class and second-class valid instructions are determined based on the function of the instructions. The third determining module is used to determine the similarity between the first program sample and the second program sample at the behavioral structure level based on the first class-level similarity index and the second class-level similarity index.

[0005] Thirdly, an electronic device is provided, including one or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to perform the methods described in any of the possible implementations above.

[0006] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the method in any of the possible implementations described above.

[0007] This application offers the following advantages: It uses bytecode instructions or DEX instructions in the compiled program as the direct analysis object. Through structured parsing and semantic normalization of the instructions, it constructs a sequence of instruction semantic features that reflects the local behavioral characteristics of the program. Furthermore, it generates multiple sets of instruction semantic fragment features at the class-level granularity, thereby achieving quantitative analysis of program behavioral structural similarity. This realizes the scientific quantification of program behavioral structural similarity. Its core lies in the accuracy of semantic normalization, the statistical expressiveness of multiple sets, and the scale independence of normalization processing. While reducing the computational complexity of program similarity analysis, it also reduces uncertainty errors and effectively improves the accuracy of program analysis. Attached Figure Description

[0008] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 A flowchart illustrating a program similarity analysis method provided in an embodiment of this application; Figure 2 A schematic diagram of a portion of the source code provided for an embodiment of this application; Figure 3 This application provides a partial content after instruction-level parsing; Figure 4 This is a schematic diagram illustrating the content of instruction normalization provided in an embodiment of this application; Figure 5 A schematic diagram illustrating a comparison result provided for an embodiment of this application; Figure 6 A schematic diagram illustrating the content of the instruction features provided in this application after being mapped by a hash function; Figure 7 A schematic diagram of a program similarity analysis device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a computer block device provided in an embodiment of this application. Detailed Implementation

[0010] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a program similarity analysis method proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined from any suitable form.

[0011] In the description of the embodiments of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more.

[0012] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0014] This application provides a method for program similarity analysis, such as... Figure 1 As shown, this can be achieved through the following steps: Step S110: Perform structured parsing and semantic normalization on the instructions of the first program sample and the second program sample to construct a first instruction semantic feature sequence and a second semantic feature sequence that can reflect the local behavioral characteristics of the program. Here, the first program sample and the second program sample are program samples to be compared for similarity, and may include any of the following types of program samples: APK file, DEX file, JAR file, CLASS file or a directory containing the above files.

[0015] Instruction structured parsing can convert low-level binary or assembly instructions into understandable, operable, and analyzable high-level structured representations, providing a foundation for subsequent tasks such as semantic normalization and behavioral feature extraction.

[0016] During implementation, the first and second program samples can be parsed at the instruction level, directly parsing bytecode instructions or DEX instructions to extract the instruction sequences corresponding to various methods. DEX instructions are the bytecode instruction set executed by the virtual machine (or Android runtime ART) on the Android platform.

[0017] The statistics are compiled for the first and second program samples, including the total number of instructions, the number of valid instructions (arithmetic and logical instructions, conditional branches, field read / write, constant loading, etc.), the number of reference instructions (method calls, field accesses, etc.), and the number of invalid instructions (NOP, MOVE, FRAME, etc.). Here, "class" refers to the class definition in the program, which is the code module containing data (attributes) and behavior (methods).

[0018] Semantic normalization can unify the syntactic differences between different instruction sets to high-level behavioral semantics through semantic mapping rules. For example, Java's `iadd` and DEX's `add-int` are both mapped to "arithmetic addition" semantics; conditional jump instructions (such as `if_eq`) are mapped to "conditional branch" semantics when combined with context.

[0019] During implementation, a first instruction semantic feature sequence and a second instruction semantic feature sequence reflecting the local behavioral characteristics of the program are generated by performing structured parsing and semantic normalization on the instructions.

[0020] Figure 2 This is a schematic diagram of a portion of the source code provided for an embodiment of this application. Figure 3 This application provides a partial content of instruction-level parsing as an embodiment. During implementation, it can be based on... Figure 2 The partial source code shown was parsed and obtained Figure 3 The image shows a portion of the parsed instructions.

[0021] Figure 4 This is a schematic diagram illustrating instruction normalization as provided in an embodiment of this application. During implementation, it can be... Figure 3 The parsed portion of the instruction shown is normalized to obtain the following result: Figure 4 The instructions shown are normalized content.

[0022] Step S120: Based on the first instruction semantic feature sequence and the second semantic feature sequence, generate a first multiset and a second multiset, which include multiple instruction semantic fragment features, at the class-level granularity. At the class granularity level, a sliding window traversal is performed on the feature sequence generated in step S110 (the window length k can be set to 5), and k consecutive features are combined into instruction semantic fragment features, and their occurrence frequency is counted to form a multiset.

[0023] Here, the first multiset may include K instruction semantic fragment features; the second multiset may include K instruction semantic fragment features.

[0024] Step S130: Determine the number of similar instruction semantic fragments between the first multiset and the second multiset; During implementation, an intersection operation is performed on the first type of multiset and the second type of multiset, and the sum of the minimum number of occurrences of the common instruction semantic fragment features in the two sets is taken as the number of similar fragments.

[0025] For example, the first type of set is {[A,B,C]:3, [B,C,D]:2}, and the second type of set is {[A,B,C]:2,[C,D,E]:1}; the contribution of the common segment [A,B,C] is min(3,2)=2; the non-common segments have no contribution; the total number of similar segments is 2.

[0026] This calculation is equivalent to finding the minimum intersection of multisets, which can accurately quantify the degree of overlap between multisets of the first type and multisets of the second type.

[0027] Step S140: Based on the first type of valid instructions and the second type of valid instructions corresponding to the first program sample and the second program sample, respectively, determine the first type of similarity index and the second type of similarity index based on the number of similar instruction semantic segments, wherein the first type of valid instructions and the second type of valid instructions are determined based on the function of the instructions; Here, valid instructions include arithmetic and logical instructions, conditional branches, field read / write, constant loading, and other instructions that can perform operations or read / write / load functions.

[0028] During implementation, the number of similar segments is normalized based on the number of effective instructions in the first / second category (such as the total number of segments or the total number of instructions), resulting in the following formula (1) for the bidirectional similarity index: (1); Among them, Sim A→B This represents the first-level similarity index; Sim B→A This represents the second-level similarity index.

[0029] Example: If the total number of segments in the first category is 10, the number in the second category is 5, and the number of similar segments is 2, then: Sim A→B =2 / 10=0.2, Sim B→A =2 / 5=0.4.

[0030] Two-way benchmarks can reflect the similarity from the perspective of different classes and are suitable for asymmetric scenarios (such as distinguishing between source and clone classes when detecting code clones).

[0031] Step S150: Determine the similarity between the first program sample and the second program sample at the behavioral structure level based on the first class-level similarity index and the second class-level similarity index.

[0032] During implementation, the overall similarity between program samples can be calculated based on the class-level similarity index of all class pairs in the first and second program samples, combined with the total number of effective instructions (such as the total number of segments) of the first or second program samples.

[0033] An overall similarity of nearly 100% indicates that the program behavior is highly consistent (such as a variant of the same program), while a similarity of nearly 0% indicates that the behavior is significantly different (such as different functional modules).

[0034] Figure 5 A schematic diagram of a comparison result provided for an embodiment of this application, such as... Figure 5 As shown, this statistic includes: A: Total instructions = 662, Valid instructions = 662, Reference instructions = 222, Invalid instructions = 0 B: Total instructions = 656, Valid instructions = 656, Reference instructions = 199, Invalid instructions = 0 Overall similarity (based on valid instructions) Overall number of similar valid instructions (standard match only) = 0 Overall similarity (standard match only) based on A = 0 / 662 * 100% = 0.00% Overall similarity (standard match only) based on B = 0 / 656 * 100% = 0.00% Overall number of similar valid instructions (standard matching + fuzzy matching) = 469 Based on A, the overall similarity (standard + fuzzy) = 469 / 662 * 100% = 70.85% Based on B, the overall similarity (standard + fuzzy) = 469 / 656 * 100% = 71.49% 1. Number of standard matches: 0 2. Number of fuzzy matches: 3 3. Unmatched class: A=1 B=1 like Figure 5 The display provides clear and concise comparison information, making it easy to read and understand. It mainly showcases the statistical and similarity analysis results of the two samples at the instruction level.

[0035] In this embodiment, bytecode instructions or DEX instructions in the compiled program are used as the direct analysis object. By performing structured parsing and semantic normalization on the instructions, a sequence of instruction semantic features reflecting the local behavioral characteristics of the program is constructed. Furthermore, multiple sets of instruction semantic fragment features are generated at the class-level granularity, thereby achieving quantitative analysis of program behavioral structural similarity. This realizes the scientific quantification of program behavioral structural similarity. Its core lies in the accuracy of semantic normalization, the statistical expressiveness of multiple sets, and the scale independence of normalization processing. While reducing the computational complexity of program similarity analysis, it also reduces uncertainty errors and effectively improves the accuracy of program analysis.

[0036] In some embodiments, the above step S110, "performing structured parsing and semantic normalization of the instructions of the first program sample and the second program sample to construct a first instruction semantic feature sequence and a second semantic feature sequence that can reflect the local behavioral characteristics of the program," can be achieved through the following steps: Step 111: Identify the file format types of the first program sample and the second program sample; During implementation, you can initially determine the possible file type by checking the file or directory extension. The correspondence between common file extensions and file types is as follows: .apk: APK file (Android application package) .dex: DEX file (Dalvik executable file) .jar: JAR file (Java archive file) .class: CLASS file (Java bytecode file) For files initially identified as belonging to a specific type, further verification can be performed by reading the file header (usually the first few bytes).

[0037] For example, an APK file is a ZIP archive with a PK header (a marker for the ZIP format).

[0038] DEX files: The file header is dex\n035 (where \n is a newline character and 035 is the version number, which may vary slightly depending on the version).

[0039] JAR file: It is also a ZIP archive, with the file header being PK.

[0040] CLASS files: The file header is CAFEBABE (the magic number that marks Java bytecode).

[0041] Step 112: Based on the identified file format type, disassemble or parse the first program sample and the second program sample respectively to obtain the first instruction sequence of the first program sample and the second instruction sequence of the second program sample; Here, disassembly is the process of converting a computer program from machine language (binary code) or bytecode into assembly language (low-level code that is readable by humans). It is a key technology in reverse engineering, program analysis, security research, and debugging, and aims to reveal the underlying logic and instruction structure of a program.

[0042] During implementation, the first and second program samples are disassembled or parsed at the instruction level based on the identified file format type. The bytecode instructions or DEX instructions are directly parsed to extract the instruction sequences corresponding to the methods in each category. The total number of instructions, the number of valid instructions (arithmetic and logical instructions, conditional branches, field read and write, constant loading, etc.), the number of reference instructions (method calls, field accesses, etc.) and the number of invalid instructions (NOP, MOVE, FRAME, etc.) in each category are counted.

[0043] Step 113: Normalize the first valid instruction in the first instruction set and the second valid instruction in the second instruction set to obtain the first instruction semantic features and the second instruction semantic features. The normalization process is used to map valid instructions of different forms but with the same behavioral semantics into unified instruction semantic features. During implementation, the valid instructions obtained in step S112 are subjected to instruction semantic normalization processing, which maps instructions of different forms but with the same behavioral semantics into unified instruction semantic features.

[0044] In some embodiments, the normalization process includes at least one of the following: Unify the mapping of conditional branch instructions to branch semantic identifiers; Map all return-type instructions to return semantic identifiers; Based on the invocation type of the invocation instruction and the category to which the invocation target belongs, the invocation instructions are semantically classified; String constants and numeric constants are bucketed according to preset ranges.

[0045] Step 114: Based on the order in which the first valid instruction appears in the first program sample, construct the semantic features of the first instruction into a one-dimensional ordered sequence of first instruction semantic features; at the same time, based on the order in which the second valid instruction appears in the second program sample, construct the semantic features of the second instruction into a one-dimensional ordered sequence of second instruction semantic features.

[0046] During implementation, the normalized instruction semantic features are constructed into a one-dimensional ordered sequence of instruction semantic features according to the order in which the instructions appear in the program.

[0047] In this embodiment, a normalization rule for instruction semantics oriented towards program behavior is introduced. While weakening the influence of non-critical factors such as variable names, specific values ​​of constants and access modifiers, the core instruction features that can reflect the program execution logic and behavior pattern are retained according to the order in which the instructions appear in the program.

[0048] In some embodiments, the processing of the instruction semantic feature sequence further includes the following steps: Step 115: Convert the one-dimensional ordered first instruction semantic feature sequence into a numerical instruction feature first instruction semantic feature sequence using a hash mapping method; Step 116: Convert the one-dimensional ordered second instruction semantic feature sequence into a second instruction semantic feature sequence with numerical instruction features using a hash mapping method.

[0049] During implementation, the normalized instruction semantic features are constructed into a one-dimensional ordered sequence of instruction semantic features according to the order in which the instructions appear in the program, and each instruction semantic feature is converted into a numerical instruction feature through hash mapping.

[0050] Figure 6 This is a schematic diagram illustrating the content of the instruction features provided in this application after being mapped using a hash function. During implementation, the parsed instructions can be hash-mapped to obtain, as shown below. Figure 6 The numerical instruction features shown.

[0051] In this embodiment, numerical instruction features can be directly used for mathematical operations, avoiding the need for line-by-line parsing and comparison of complex semantic features, thus significantly improving computation speed. Numerical features occupy less space and can be quickly retrieved through structures such as hash tables and database indexes, making them suitable for rapid querying and comparison of large-scale code repositories.

[0052] In some embodiments, the above step S120, "generating a first multiset and a second multiset of instruction semantic fragment features at the class-level granularity based on the first instruction semantic feature sequence and the second semantic feature sequence respectively," can be achieved through the following steps: Step 121: At the class level, the first instruction semantic feature sequence and the second instruction semantic feature sequence are traversed using a sliding window of length K, and K consecutive instruction semantic features within the sliding window are combined into a first instruction semantic fragment feature and a second instruction semantic fragment feature. Step 122: Construct a first multiset based on the first instruction semantic fragment features, and construct a second multiset based on the second instruction semantic fragment features.

[0053] At the class level, the instruction semantic feature sequence is traversed using a sliding window of length k. K consecutive instruction semantic features within the window are combined into instruction semantic fragment features to characterize the local behavioral structure of the program. The generated instruction semantic fragment features are then constructed into a multiset of instruction semantic fragment features. The number of times the same instruction semantic fragment feature appears in the multiset is recorded as the corresponding count value (the value of k can be modified according to the strictness, and k=5 in general).

[0054] Here, the choice of k value can be dynamically adjusted according to the needs of the scenario, balancing behavior capture capability and noise suppression.

[0055] In this embodiment, efficient structured representation of local program behavior is achieved through class-level granular sliding window traversal and multi-set construction. It can be widely applied in fields such as program similarity analysis, malware detection, and code clone detection. It improves computational efficiency through numerical features and hash mapping, supports cross-platform analysis, and preserves key behavioral semantics.

[0056] In some embodiments, the first multiset includes multiple first code classes, and the second multiset includes multiple second code classes. The above step S130, "determining the number of similar instruction semantic fragments between the first multiset and the second multiset," can be achieved through the following steps: Step 131: Determine the semantic features of the target instruction fragments that coexist in both the first target code class and the second target code class; Here, code class can refer to class definition in a program, that is, code module containing data (attributes) and behavior (methods).

[0057] A class, the core abstract concept in object-oriented programming (OOP), is a blueprint or template used to describe a collection of objects that share common attributes, behaviors, and relationships. It encapsulates data (attributes / fields) and methods for manipulating that data (functions / behaviors), and is a key unit for achieving code modularity, reusability, and structured design.

[0058] The target first code class can be any first code class determined from the first multiset; the target second code class can be any second code class determined from the second multiset.

[0059] During implementation, a target first code class is determined from the first code class set, and a target second code class is determined from the second code class set.

[0060] Next, determine the semantic features of all target instruction fragments that exist simultaneously in the first target code class and the second target code class.

[0061] Step 132: For each target instruction semantic fragment feature, obtain the first occurrence count of the target instruction semantic fragment feature in the first target code class and the second occurrence count in the second target code class; During implementation, the first occurrence count of the target instruction semantic fragment feature in the target first code class can be obtained. For example, the first occurrence count of the target instruction semantic fragment feature A in the target first code class can be obtained as 5 times.

[0062] The second occurrence number of the target instruction semantic fragment feature in the target second code class can be obtained. For example, the second occurrence number of the target instruction semantic fragment feature A in the target second code class can be obtained as 8 times.

[0063] Step 133: Take the smaller value between the first occurrence count and the second occurrence count as the contribution value of the target instruction semantic fragment feature to the inter-class similarity between the target first code class and the target second code class; During implementation, the smaller of the first occurrence count and the second occurrence count can be used as the contribution value for inter-class similarity. For example, if the first occurrence count is 5 and the second occurrence count is 8, then 5 can be used as the contribution value for similarity.

[0064] Step 133: Accumulate the contribution values ​​of the inter-class similarity of all the first code classes and the second code classes to obtain the number of similar instruction semantic fragments between the first multiset and the second multiset.

[0065] During implementation, the contribution values ​​between all classes in the first and second code classes can be summed to obtain the number of similar instruction semantic fragments between the first and second multisets. For example, if the first code class set includes 3 first code classes and the second code class set includes 2 second code classes, then the contribution value of the similarity between each pair of classes in the 3 first code classes and 2 second code classes can be determined first, and then the contribution values ​​between all classes can be summed to obtain the number of similar instruction semantic fragments between the first and second multisets.

[0066] In this embodiment of the application, the use of multiple sets and minimum number rules more accurately reflects the actual degree of semantic overlap between the two types of instructions.

[0067] In some embodiments, the above step S140, "determining the first-class similarity index and the second-class similarity index based on the number of similar instruction semantic segments, using the first type of valid instructions and the second type of valid instructions corresponding to the first program sample and the second program sample respectively as benchmarks," can be achieved through the following steps: Step 141: Divide the number of similar instruction semantic segments by the total number of valid instructions in the first class to obtain a first-class similarity index based on the first class. Step 142: Divide the number of similar instruction semantic fragments by the total number of valid instructions in the second category to obtain a second-class similarity index based on the second category.

[0068] During implementation, the number of similar instruction semantic segments is normalized to the number of effective instructions in the first and second categories, respectively, to obtain class-level similarity indices based on the first category and class-level similarity indices based on the second category.

[0069] Here, the execution order of steps 141 and 142 is not restricted.

[0070] In this embodiment, the number of similar instruction semantic segments is normalized to the number of effective instructions in the first and second classes, respectively, to obtain class-level similarity indices for the two benchmarks. By eliminating the influence of scale differences and providing a two-way comparison perspective, the accuracy and practicality of similarity assessment are significantly improved.

[0071] In some embodiments, the above step S150, "determining the similarity between the first program sample and the second program sample at the behavioral structure level based on the first class-level similarity index and the second class-level similarity index," can be achieved through the following steps: Step 151: Determine the code classes with the same classpath in the first program sample and the second program sample; During implementation, the fully qualified name of the class (package name + class name) can be extracted by parsing the bytecode file of the program sample (such as the .class file of Java) and the code class with completely identical paths in the two samples can be compared.

[0072] Store the classpath information of two samples as a set (such as Python's set), and quickly locate the code class with the same path through the intersection operation, as shown in the following example: Python Set set1 = {"com.example.ClassA", "com.example.ClassB"} Set set2 = {"com.example.ClassA", "com.example.ClassC"} The intersection operation `common_classes = set1&set2` results in: `{"com.example.ClassA"}` It can be determined that the code class with the same path in set1 and set2 is "com.example.ClassA".

[0073] Step 152: For code classes with the same classpath, perform similarity calculation to determine the first matching result; Here, for code classes with the same path, features can be extracted from three dimensions: structure, behavior, and dependency to quantify similarity.

[0074] During implementation, feature dimensions can be extracted first, including structural features, behavioral features, and dependency features. Structural features include the signatures of class members (fields, methods), inheritance relationships, and interface implementations. Behavioral features include the instruction sequence of method calls (such as bytecode for `invokevirtual`, `invokestatic`, etc.). Dependency features include referenced third-party libraries and strings / class names in the constant pool.

[0075] Use at least one of the following similarity calculation methods to calculate similarity: Cosine similarity: suitable for vectorized features (such as a list of method signatures), measuring directional consistency. Jaccard similarity: suitable for set features (such as a set of fields, a set of dependencies), measuring the percentage of intersection. Edit distance: suitable for string features (such as class comments, method body code), measuring textual differences.

[0076] The above method can be used to systematically complete the path comparison and similarity calculation of code classes.

[0077] Step 153: For code classes with different class paths, if the class-level similarity index of either the first program sample or the second program sample exceeds a preset threshold, the corresponding code class is determined as a candidate matching code class. When different code classes are identified by different paths, it can be determined that the class-level similarity index of either the first program sample or the second program sample exceeds a preset threshold. The preset threshold can be set based on actual needs.

[0078] Code classes that exceed a preset threshold are identified as candidate matching code classes.

[0079] Step 154: Select the code class with the highest similarity from the candidate matching code classes, and generate the second matching result using a one-to-one greedy matching strategy; Here, selecting the code class with the highest similarity among the candidate matching code classes can identify the two most similar code classes in the first program sample and the second program sample.

[0080] During implementation, fuzzy matching (second matching) is performed on classes that do not participate in standard matching. When the class-level similarity index on either side exceeds the preset threshold, the corresponding class pair is determined as a candidate match, and a one-to-one greedy matching strategy is used to generate fuzzy matching results (second matching results). Finally, the class with the highest similarity is selected for matching (the threshold can be set according to the application scenario).

[0081] Step 155: Based on the first matching result and the second matching result, summarize the number of similar instruction semantic fragments for each matched code class; Step 156: Based on the number of similar instruction semantic fragments of each matched code class and the number of valid instructions of the first program sample and the second program sample, determine the overall similarity between the program samples.

[0082] During implementation, based on the standard matching results and fuzzy matching results, the number of similar instruction semantic fragments corresponding to each matching class is summarized, and combined with the number of effective instructions of the first program sample and the second program sample, the overall similarity between the program samples is calculated, which is used to characterize the degree of similarity between the two programs at the overall program behavior level. Figure 5 A schematic diagram of a comparison result provided for an embodiment of this application, such as... Figure 5 As shown, a summary diagram of the standard matching results and fuzzy matching results can be seen.

[0083] In this embodiment, two complementary matching mechanisms, precise matching and fuzzy matching, are introduced simultaneously in the class matching strategy. Precise matching (first matching) directly calculates the similarity of classes with the same classpath to ensure the accuracy of comparison between classes with the same name or corresponding structure. Fuzzy matching (second matching), on the other hand, filters candidate matches based on a class-level similarity threshold for classes that fail precise matching, employing a one-to-one matching strategy to effectively avoid missed detections of similar classes caused by operations such as class renaming, package path adjustments, or code splitting. Through the coordinated use of precise matching and fuzzy matching, this embodiment significantly improves the coverage of potentially similar content while ensuring comparison accuracy.

[0084] This application provides a method for determining program similarity. This method uses bytecode instructions or DEX instructions in the compiled program as the direct analysis object. By performing structured parsing and semantic normalization on the instructions, it constructs a sequence of instruction semantic features that can reflect the local behavioral characteristics of the program, and generates multiple sets of instruction semantic fragment features at the class level, thereby realizing quantitative analysis of program behavioral structural similarity.

[0085] Unlike existing technologies, this application does not use source code text, decompiled pseudocode, or abstract syntax trees as comparison objects, nor does it perform global alignment calculations on instruction sequences, construct instruction matching matrices or backtracking paths, nor does it aim to find the longest common subsequence or the optimal alignment path. Instead, it assesses the similarity between programs by statistically analyzing the degree of overlap of local instruction behavior fragments in different classes.

[0086] In the similarity calculation process, this embodiment of the application normalizes the results by considering the number of intersections between multiple sets of instruction semantic fragment features and the number of valid instructions in the corresponding class, thereby obtaining similarity indices based on different program samples. These indices characterize the degree of similarity between programs at the behavioral structure level. Through this method, this embodiment of the application can improve the accuracy, interpretability, and recalculation of similarity analysis results while ensuring computational efficiency, making it suitable for applications such as large-scale program comparison, software homology analysis, and program forensics.

[0087] The program similarity analysis method provided in this application can be implemented through the following steps: Step S1: Receive the first program sample and the second program sample to be compared, and perform type identification on the program sample to determine whether the program sample is an APK file, DEX file, JAR file, CLASS file or a directory containing the above files.

[0088] Step S2: Perform instruction-level parsing on the first and second program samples respectively, directly parse bytecode instructions or DEX instructions, extract the instruction sequences corresponding to methods in each category, and without generating pseudocode through decompilation, count the total number of instructions, the number of valid instructions (arithmetic and logical instructions, conditional branches, field read and write, constant loading, etc.), the number of reference instructions (method calls, field accesses, etc.) and the number of invalid instructions (NOP, MOVE, FRAME, etc.) in each category.

[0089] Figure 2 This is a schematic diagram of a portion of the source code provided for an embodiment of this application. Figure 3 This application provides a partial content of instruction-level parsing as an embodiment. During implementation, it can be based on... Figure 2 The partial source code shown was parsed and obtained Figure 3 The image shows a portion of the parsed instructions.

[0090] Step S3: Perform instruction semantic normalization processing on the valid instructions obtained in step S2, mapping instructions of different forms but with the same behavioral semantics to a unified instruction semantic feature, wherein the instruction semantic normalization processing includes at least one of the following or a combination thereof: Unify the mapping of conditional branch instructions to branch semantic identifiers; Map all return-type instructions to return semantic identifiers; Based on the invocation type of the invocation instruction and the category to which the invocation target belongs, the invocation instructions are semantically classified; String constants and numeric constants are bucketed according to preset ranges.

[0091] Figure 4 This is a schematic diagram illustrating instruction normalization as provided in an embodiment of this application. During implementation, it can be... Figure 3 The parsed portion of the instruction shown is normalized to obtain the following result: Figure 4 The instructions shown are normalized content.

[0092] Step S4: According to the order in which the instructions appear in the program, construct the normalized instruction semantic features into a one-dimensional ordered sequence of instruction semantic features, and convert each instruction semantic feature into a numerical instruction feature through hash mapping.

[0093] Figure 6This is a schematic diagram illustrating the content of the instruction features provided in this application after being mapped using a hash function. During implementation, the parsed instructions can be hash-mapped to obtain, as shown below. Figure 6 The numerical instruction features shown.

[0094] Step S5: At the class level, the instruction semantic feature sequence is traversed using a sliding window of length k. K consecutive instruction semantic features within the window are combined into instruction semantic fragment features to characterize the local behavioral structure of the program. The generated instruction semantic fragment features are then constructed into a multiset of instruction semantic fragment features. The number of times the same instruction semantic fragment feature appears in the multiset is recorded as the corresponding count value (the value of k can be modified according to the strictness, and k=5 in general).

[0095] Step S6: Based on the multiset of instruction semantic fragment features, quantify the similarity between different classes, specifically including: For instruction semantic fragment features that exist simultaneously in the multiset of instruction semantic fragment features corresponding to the first and second categories, the smaller value of the number of times it appears in the two multisets is taken as the contribution value of the instruction semantic fragment feature to the inter-class similarity, and all contribution values ​​are accumulated to obtain the number of similar instruction semantic fragments between the first and second categories. The number of similar instruction semantic fragments is normalized to the number of effective instructions in the first and second categories, respectively, to obtain the class-level similarity index based on the first category and the class-level similarity index based on the second category.

[0096] Step S7: Perform multi-strategy class matching processing based on the class-level similarity index, including: Perform standard matching on classes with the same classpath to obtain the standard matching results; For classes that do not participate in standard matching, perform fuzzy matching. When the class-level similarity index on either side exceeds the preset threshold, the corresponding class pair is identified as a candidate match, and a one-to-one greedy matching strategy is used to generate fuzzy matching results. Finally, the class with the highest similarity is selected for matching (the threshold can be set according to the application scenario). Based on standard matching results and fuzzy matching results, the number of similar instruction semantic fragments corresponding to each matching class is summarized. Combined with the number of valid instructions in the first and second program samples, the overall similarity between the program samples is calculated to characterize the degree of similarity between the two programs at the overall program behavior level. The method does not perform global alignment calculations on instruction sequences, does not construct instruction matching matrices or backtracking paths, and does not use finding the longest common subsequence or optimal alignment relationship as the basis for similarity determination.

[0097] Figure 5A schematic diagram of a comparison result provided for an embodiment of this application, such as... Figure 5 As shown, this statistic includes: A: Total instructions = 662, Valid instructions = 662, Reference instructions = 222, Invalid instructions = 0 B: Total instructions = 656, Valid instructions = 656, Reference instructions = 199, Invalid instructions = 0 Overall similarity (based on valid instructions) Overall number of similar valid instructions (standard match only) = 0 Overall similarity (standard match only) based on A = 0 / 662 * 100% = 0.00% Overall similarity (standard match only) based on B = 0 / 656 * 100% = 0.00% Overall number of similar valid instructions (standard matching + fuzzy matching) = 469 Based on A, the overall similarity (standard + fuzzy) = 469 / 662 * 100% = 70.85% Based on B, the overall similarity (standard + fuzzy) = 469 / 656 * 100% = 71.49% 1. Number of standard matches: 0 2. Number of fuzzy matches: 3 3. Unmatched class: A=1 B=1 like Figure 5 The display provides clear and concise comparison information, making it easy to read and understand. It mainly showcases the statistical and similarity analysis results of the two samples at the instruction level.

[0098] This application directly uses compiled bytecode or DEX instructions as the analysis object, and constructs program features by performing format parsing and semantic normalization on the instructions, thus avoiding dependence on source code text, decompiled pseudocode, or natural language code from the source.

[0099] The embodiments of this application do not involve word segmentation, semantic matching or document similarity determination technology based on text content, nor do they adopt dynamic programming algorithms based on global alignment of instruction sequences, nor do they construct matching matrices or backtracking paths for instruction alignment, thereby avoiding the uncertainty errors introduced by heuristic processing such as control flow reconstruction, variable recovery and abnormal structure inference during the decompilation process.

[0100] We introduce instruction semantic normalization rules oriented towards program behavior. While weakening the influence of non-critical factors such as variable names, specific values ​​of constants and access modifiers, we retain the core instruction features that can reflect the program execution logic and behavior patterns, and construct multiple sets of instruction fragment features at the class level.

[0101] By statistically analyzing the number of intersections of multiple sets of instruction fragment features and normalizing them in conjunction with the number of valid instructions, a quantitative determination of the similarity of program behavior structure can be achieved.

[0102] Regarding the class matching strategy, this application's embodiments simultaneously introduce two complementary matching mechanisms: precise matching and fuzzy matching. Among them, Exact matching is used to directly calculate the similarity between classes with the same classpath to ensure the accuracy of comparison between classes with the same name or structure. Fuzzy matching, on the other hand, filters candidate matches based on class-level similarity thresholds for classes that fail to pass the exact match and adopts a one-to-one matching strategy, thereby effectively avoiding the problem of missing similar classes caused by operations such as class renaming, package path adjustment or code splitting.

[0103] By using precise matching and fuzzy matching in combination, the embodiments of this application significantly improve the coverage of potentially similar content while ensuring comparison accuracy.

[0104] In the similarity calculation process, this application embodiment mainly adopts a set operation method between linear scanning and instruction fragment feature sets to complete the similarity measurement, avoiding global alignment calculation of instruction sequences or code structures. Compared with technical solutions based on sequence alignment algorithms such as Longest Common Subsequence (LCS), whose time complexity is usually O(N×M), the computational cost is extremely high when dealing with software programs containing a large number of classes and methods. However, this application embodiment significantly reduces the computational complexity of the similarity calculation process by transforming instruction sequences into instruction fragment feature sets. The time complexity is linear or near-linear with the number of behavioral fragments, making it more suitable for batch comparison and large-scale program analysis scenarios containing a large number of class files or APKs.

[0105] In summary, the embodiments of this application differ from existing technologies in terms of object selection, feature construction method, similarity calculation model and matching strategy. They can effectively balance comparison accuracy and computational efficiency while ensuring the stability, interpretability and recalculation of results, and have good engineering practical value and broad application prospects.

[0106] This application provides a program similarity analysis device. Please refer to [link to relevant documentation]. Figure 7 The system 700 includes: The parsing and normalization module 710 is used to perform structured parsing and semantic normalization processing on the instructions of the first program sample and the second program sample, and to construct a first instruction semantic feature sequence and a second semantic feature sequence that can reflect the local behavioral characteristics of the program. The multiset generation module 720 is used to generate a first multiset and a second multiset, which include multiple instruction semantic fragment features, at the class-level granularity based on the first instruction semantic feature sequence and the second semantic feature sequence, respectively. The first determining module 730 is used to determine the number of similar instruction semantic fragments between the first multiset and the second multiset; The second determining module 740 is used to determine a first-class similarity index and a second-class similarity index based on the number of similar instruction semantic segments, respectively, using the first-class and second-class valid instructions corresponding to the first program sample and the second program sample as benchmarks. The first-class and second-class valid instructions are determined based on the function of the instructions. The third determining module 750 is used to determine the similarity between the first program sample and the second program sample at the behavioral structure level based on the first class-level similarity index and the second class-level similarity index.

[0107] Figure 8 This is a schematic diagram of the structure of a computer block device provided in an embodiment of this application. For example, as shown... Figure 8 As shown, the computer block device 800 includes: a memory 801, a processor 802, and a computer program 803 stored in the memory 801 and running on the processor 802, wherein when the processor 802 executes the computer program 803, the computer block device can execute any of the program similarity analysis methods described above.

[0108] Furthermore, this application also protects a control block device, which may include a memory and a processor. The memory stores executable program code, and the processor is used to call and execute the executable program code to perform a program similarity analysis method provided in this application. This application can divide the control block device into functional modules based on the above method example. For example, it can correspond to individual functional modules, or two or more functions can be integrated into a processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this application is illustrative and only represents a logical functional division; other division methods may exist in actual implementation. It should also be noted that all relevant content of each step involved in the above method embodiment can be referenced to the functional description of the corresponding functional module, and will not be repeated here. It should be understood that the control block device provided in this application is used to execute the above-mentioned program similarity analysis method, and therefore can achieve the same effect as the above-mentioned implementation method. When using integrated units, the control block device may include a processing module and a storage module. When the control block device is applied to a block device, the processing module can be used to control and manage the actions of the block device. The storage module can be used to support block devices in executing mutual program code, etc. The processing module can be a processor or controller, which can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and microprocessors, etc., and the storage module can be a memory.

[0109] Furthermore, the control block device provided in the embodiments of this application may specifically be a chip, component, or module. The chip may include a connected processor and a memory. The memory stores instructions, and when the processor calls and executes the instructions, the chip can execute the program similarity analysis method provided in the above embodiments. The embodiments of this application also provide a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, it causes the computer to execute the aforementioned method steps to implement the program similarity analysis method provided in the above embodiments.

[0110] This application also provides a computer program product. When the computer program product is run on a computer, it causes the computer to execute the aforementioned related steps to implement the program similarity analysis method provided in the above embodiments. The control block device, computer-readable storage medium, computer program product, or chip provided in this application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they achieve can be referred to in the beneficial effects of the corresponding methods provided above, and will not be repeated here. Through the description of the above embodiments, those skilled in the art can understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the control block device can be divided into different functional modules to complete all or part of the functions described above. In the embodiments provided in this application, it should be understood that the disclosed control block device and method can be implemented in other ways. For example, the control block device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another control block device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, control block device or unit, and can be electrical, mechanical or other forms.

[0111] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multiple task processing and parallel processing are possible or may be advantageous. The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. The above content is only a specific implementation of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A method for program similarity analysis, characterized in that, The method includes: The instructions of the first program sample and the second program sample are subjected to structured parsing and semantic normalization to construct a first instruction semantic feature sequence and a second semantic feature sequence that can reflect the local behavioral characteristics of the program. Based on the first instruction semantic feature sequence and the second semantic feature sequence, a first multiset and a second multiset, including multiple instruction semantic fragment features, are generated at the class-level granularity. Determine the number of similar instruction semantic fragments between the first multiset and the second multiset; Based on the first type of valid instructions and the second type of valid instructions corresponding to the first program sample and the second program sample, respectively, a first-class similarity index and a second-class similarity index are determined based on the number of similar instruction semantic segments. The first type of valid instructions and the second type of valid instructions are determined based on the function of the instructions. The similarity between the first program sample and the second program sample at the behavioral structure level is determined based on the first class-level similarity index and the second class-level similarity index.

2. The method as described in claim 1, characterized in that, The step of performing structured parsing and semantic normalization on the instructions of the first and second program samples to construct a first instruction semantic feature sequence and a second semantic feature sequence that can reflect the local behavioral characteristics of the program includes: Identify the file format types of the first program sample and the second program sample; Based on the identified file format type, the first program sample and the second program sample are disassembled or parsed respectively to obtain the first instruction sequence of the first program sample and the second instruction sequence of the second program sample. The first valid instruction in the first instruction set and the second valid instruction in the second instruction set are normalized to obtain the first instruction semantic features and the second instruction semantic features. The normalization process is used to map valid instructions of different forms but with the same behavioral semantics into unified instruction semantic features. Based on the order in which the first valid instruction appears in the first program sample, the semantic features of the first instruction are constructed into a one-dimensional ordered sequence of first instruction semantic features; at the same time, based on the order in which the second valid instruction appears in the second program sample, the semantic features of the second instruction are constructed into a one-dimensional ordered sequence of second instruction semantic features.

3. The method as described in claim 2, characterized in that, The normalization process includes at least one of the following: Unify the mapping of conditional branch instructions to branch semantic identifiers; Map all return-type instructions to return semantic identifiers; Based on the invocation type of the invocation instruction and the category to which the invocation target belongs, the invocation instructions are semantically classified; String constants and numeric constants are bucketed according to preset ranges.

4. The method as described in claim 2, characterized in that, The method further includes: The one-dimensional ordered first instruction semantic feature sequence is converted into a numerical instruction feature first instruction semantic feature sequence by hash mapping. The one-dimensional ordered second instruction semantic feature sequence is converted into a second instruction semantic feature sequence with numerical instruction features through a hash mapping method.

5. The method as described in claim 1, characterized in that, The generation of a first multiset and a second multiset, comprising multiple instruction semantic fragment features, at the class-level granularity based on the first instruction semantic feature sequence and the second semantic feature sequence, respectively, includes: At the class level, the first instruction semantic feature sequence and the second instruction semantic feature sequence are traversed using a sliding window of length K, and K consecutive instruction semantic features within the sliding window are combined into a first instruction semantic fragment feature and a second instruction semantic fragment feature. A first multiset is constructed based on the semantic fragment features of the first instruction, and a second multiset is constructed based on the semantic fragment features of the second instruction.

6. The method as described in claim 1, characterized in that, The first multiset contains multiple first code classes, and the second multiset contains multiple second code classes. Determining the number of similar instruction semantic fragments between the first multiset and the second multiset includes: Identify the semantic features of all target instruction fragments that coexist in both the first and second target code classes; For each target instruction semantic fragment feature, the first occurrence count of the target instruction semantic fragment feature in the first target code class and the second occurrence count of the target second code class are obtained respectively; The smaller value between the first occurrence count and the second occurrence count is taken as the contribution value of the target instruction semantic fragment feature to the inter-class similarity between the target first code class and the target second code class; The contribution values ​​of the inter-class similarity of all the first code classes and the second code classes are summed to obtain the number of similar instruction semantic fragments between the first multiset and the second multiset.

7. The method as described in claim 1, characterized in that, The step of determining a first-class similarity index and a second-class similarity index based on the number of similar instruction semantic segments, using the first and second program samples respectively as benchmarks corresponding to the first and second types of valid instructions, includes: Divide the number of similar instruction semantic fragments by the total number of valid instructions in the first category to obtain the first category-level similarity index; Divide the number of similar instruction semantic fragments by the total number of valid instructions in the second category to obtain the second-level similarity index.

8. The method as described in claim 1, characterized in that, Determining the similarity between the first program sample and the second program sample at the behavioral structure level based on the first class-level similarity index and the second class-level similarity index includes: Identify code classes with the same classpath in the first program sample and the second program sample; For code classes with the same classpath, a similarity calculation is performed to determine the first matching result; For code classes with different class paths, if the class-level similarity index of either the first program sample or the second program sample exceeds a preset threshold, the corresponding code class will be identified as a candidate matching code class. The code class with the highest similarity among the candidate matching code classes is selected, and a greedy matching strategy with one-to-one correspondence is used to generate a second matching result; Based on the first matching result and the second matching result, the number of similar instruction semantic fragments for each matched code class is summarized; The overall similarity between program samples is determined based on the number of similar instruction semantic fragments in each matched code class and the number of valid instructions in the first program sample and the second program sample.

9. A program similarity analysis device, characterized in that, The device includes: The parsing and normalization module is used to perform structured parsing and semantic normalization on the instructions of the first program sample and the second program sample, and to construct a first instruction semantic feature sequence and a second semantic feature sequence that can reflect the local behavioral characteristics of the program. A multiset generation module is used to generate a first multiset and a second multiset, which include multiple instruction semantic fragment features, at the class-level granularity based on the first instruction semantic feature sequence and the second semantic feature sequence, respectively. The first determining module is used to determine the number of similar instruction semantic fragments between the first multiset and the second multiset; The second determining module is used to determine a first-class similarity index and a second-class similarity index based on the number of similar instruction semantic segments, using the first-class and second-class valid instructions corresponding to the first program sample and the second program sample as benchmarks, respectively. The first-class and second-class valid instructions are determined based on the function of the instructions. The third determining module is used to determine the similarity between the first program sample and the second program sample at the behavioral structure level based on the first class-level similarity index and the second class-level similarity index.

10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the program similarity analysis method as described in any one of claims 1-8.

Images