C / C + + source code third-party library reuse detection method based on function structure information

Abstract

The invention relates to the technical field of software development and software supply chain security, and discloses a C / C + + source code third-party library reuse detection method based on function structure information, which comprises the following steps: constructing a TPL vector index library based on an open source software function AST structure; in the vector index database, filtering common functions with reuse frequencies exceeding a normal frequency range by adopting an abnormal value detection method; and a dynamic proportion screening strategy is used for effectively filtering the reused TPL candidate items which are wrongly recognized, so that a more accurate reused TPL result is obtained. According to the method, the robustness and the accuracy of TPL reuse detection are remarkably improved, and missing report and false report are effectively reduced; meanwhile, compared with a traditional method, the method has higher detection efficiency and lower storage requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of software development and software supply chain security, and in particular to a C / C++ source code third-party library reuse detection method based on function structure information. Background Art

[0002] The number of open-source software (OSS) has expanded dramatically with the popularity of remote code repositories such as GitHub. The use of third-party libraries (TPL) has become more and more common through various code reuse methods such as installation dependencies, API calls, project forks, file copies, and code cloning. Especially in C / C++ projects, in order to avoid duplication of work, developers often use code cloning to introduce the required TPL and modify it to meet business needs. The above situation brings new challenges to the accurate identification of TPL and also expands the potential attack surface of software. It is crucial to comprehensively track the reuse of software TPL. The security of the software supply chain is directly related to the security and reliability of software products. Therefore, software composition analysis (SCA) technology for detecting the reuse of TPL in software has become a research hotspot as an effective means to timely discover software security risks and reduce hazards.

[0003] In SCA practice, OSSPOLICE attempts to use lightweight features such as directory structure to build a feature library, but when the project structure changes, detection failure and insufficient robustness may occur. CENTRIS proposed by Woo et al. improves the accuracy of TPL detection in reuse scenarios with small modifications by segmenting OSS source code, eliminating redundant functions and using fuzzy hashing technology; however, this method may fail and produce false negative results when facing more text modifications (for example, the functions of the tinyfiledialogs library in the julius project are different from the original functions on Github). OSSFP introduces a filtering method for public functions and trivial functions, and improves detection performance and accuracy by building a hash fingerprint index for each core function of TPL; however, OSSFP assumes that some functions in the reused TPL have not been changed, so when large-scale function modifications occur (for example, the SQLiteCpp project reuses the sqlite library through the sqlite3.c source file, in which all functions have new access modifiers added), it will also face the problem of insufficient robustness, resulting in detection failure.

[0004] In addition, it is observed that CENTRIS does not consider the impact of interference functions, which is prone to produce more false positives. OSSFP uses a fixed frequency threshold to filter public functions, which actually causes inaccurate filtering; for example, the Boost library contains a large number of public functions, while the xxhash library has almost none. OSSFP believes that the public functions of the two should be filtered in the same proportion.

[0005] By analyzing the existing static SCA technology, it is found that it still has shortcomings in detection robustness and interference function processing: (1) The robustness of TPL features is insufficient: For example, CENTRIS, OSSFP and other work rely on function text hash features, facing Type II and Type III code cloning (such as variable name or code line modification), which is prone to false negatives (FN); (2) Interference function filtering is inaccurate: CENTRIS does not fully consider interference functions, resulting in a large number of false positives (FP); and OSSFP screens public functions through a fixed frequency threshold, which actually has the problem of inaccurate filtering.

[0006] Therefore, a C / C++ source code third-party library reuse detection method based on function structure information is proposed to solve the above problems. Summary of the invention

[0007] In view of the above problems, the purpose of the present invention is to provide a C / C++ source code third-party library reuse detection method based on function structure information. By constructing a feature vector index library based on AST structure and combining anomaly detection technology with dynamic proportion screening strategy, the robustness and accuracy of TPL reuse detection are significantly improved, and omissions and false positives are effectively reduced. The technical solution is as follows:

[0008] The C / C++ source code third-party library reuse detection method based on function structure information includes the following steps:

[0009] Step S1: collecting source codes of different versions of large-scale open source software and preprocessing the source codes. The preprocessing includes the following steps:

[0010] S11, dividing the source code into functions and extracting the function AST structure of the source code;

[0011] S12, according to the degree and order of the node types in the function AST structure, parsing the function AST structure into two vectors with different structural features, and performing feature concatenation on the two vectors to generate a function feature vector;

[0012] Step S2: Filter the common structure functions in the function set segmented from the source code based on the structural complexity of the function AST structure;

[0013] Step S3: Use the function feature vector to construct a function feature dictionary for each TPL through hash matching; merge the function feature dictionaries of all TPLs to generate a vector index library; in the process of merging the vector index library, record all function clone pairs and store them as a function clone pair set;

[0014] Step S4, obtaining the reuse frequency of each function based on the function clone pair set; in the vector index library, filtering the public functions whose reuse frequency exceeds the normal frequency range to obtain the core vector index library;

[0015] Step S5: Based on the core vector index library, an approximate nearest neighbor algorithm is used to perform TPL reuse detection on the software under test to obtain reused TPL candidates; according to the function feature ratio of the reused TPL candidates, the misidentified reused TPL candidates are filtered out to determine the reused TPL detection result.

[0016] Furthermore, the step S12 specifically includes:

[0017] According to the degree and order of the node types in the function AST structure, the TreeCen algorithm is used to parse the function AST structure into two vectors with different structural features, and the two vectors are concatenated to generate the function feature vector. The calculation formula is as follows:

[0018] v=βv1+(1-β)v2,β∈[0,1]

[0019] Among them, v1 represents the vector obtained by the sum of the degrees of each node type, v2 represents the vector obtained by the sum of the orders of each node type, and β represents the weight of v1 in feature concatenation.

[0020] Furthermore, the step S2 comprises the following steps:

[0021] S21. The structural complexity C of a function is defined as:

[0022] C=|{t i |c i ≥1,i=1,2,…,n}|

[0023] Among them, t i is the i-th structure type in the function AST structure, c i is the number of occurrences of the i-th structural type in the function AST structure; the total number of structural types that appear more than or equal to 1 in the function AST structure is the structural complexity of the function;

[0024] S22. In the function set segmented from the source code, count the number of occurrences of different structure types in the AST structure of each function to obtain the structural complexity C of each function;

[0025] S23: Eliminate functions whose structural complexity C is less than a preset threshold in the set of functions The general structure function of

[0026] Furthermore, the step S3 comprises the following steps:

[0027] S31. Use the function text hash value as the dictionary key, use the function feature of the earliest submission time as the dictionary value, and build a function feature dictionary for each TPL through hash matching; wherein the function feature is defined as a tuple describing the function attribute information: {function text hash value, function address, submission time, function feature vector};

[0028] S32, extracting function feature vectors from the function feature dictionaries of all TPLs; comparing and removing duplicates of all function feature vectors by using an approximate nearest neighbor algorithm, giving priority to retaining function feature vectors with earlier submission times, and then merging the function feature dictionaries of all TPLs to generate a vector index library; in the process of merging the vector index library, all similar function pairs are merged. a 、f b Clone the pair (f a ,f b ) is stored as a set of function clone pairs, and the function with the earlier submission time is ensured to be f a , and f b It is its clone.

[0029] Furthermore, the step S4 comprises the following steps:

[0030] S41, constructing a union-find set DSU by using function clones, taking the function with the earliest submission time as the root node of each set in the union-find set, and taking the remaining functions that have a clone relationship with the function with the earliest submission time as the child nodes of each set in the union-find set; querying the number of child nodes in each set and in the union-find set by a path compression algorithm, and obtaining the reuse frequency of each function;

[0031] S42. The Z-score anomaly detection method based on statistical distribution is used to calculate the standardized distance of function reuse frequency. The calculation formula is as follows:

[0032] Z(X f )=(X f -μ) / σ

[0033] Among them, X f is the reuse frequency value of function f; Z(X f ) is the standardized distance; μ and σ are the mean and standard deviation of the function reuse frequency in the TPL to which function f belongs, respectively;

[0034] The filtering conditions for the core function feature set within the normal frequency range are as follows:

[0035] B ′ ={f∈B|Z(X f )≤τ}

[0036] Among them, B represents the function feature set of TPL to which function f belongs, B ′ represents the set of function features within the normal frequency range in B, and the threshold τ is set to 1;

[0037] In the vector index library, the common function feature set whose reuse frequency exceeds the normal frequency range is filtered according to the screening condition, the core function feature set within the normal frequency range is retained, and the core vector index library is obtained;

[0038] Furthermore, the step S5 comprises the following steps:

[0039] S51, preprocessing the source code of the software to be tested using the same method as step S1 to obtain a feature vector of the function to be tested;

[0040] S52, in the core vector index library, using the approximate nearest neighbor algorithm to find a function feature vector that approximately matches the function feature vector to be tested, and taking the third-party library set to which the function set corresponding to the found function feature vector belongs as a candidate for reuse of TPL;

[0041] S53. Calculate the proportion of function features corresponding to all reused TPL candidates. The calculation formula is as follows:

[0042]

[0043] Among them, p(T j ) is the function feature ratio corresponding to the jth reused TPL candidate, T j is the jth reuse TPL candidate, The software under test is related to T j The set of matching function features, T in the core vector index library j The corresponding function feature set;

[0044] The reused TPL candidates are sorted in ascending order according to the proportion of their function features, and the TPL candidates ranked in the top 10% of the sorted results and with a function feature proportion of less than 2% are removed to obtain the reused TPL detection results.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] The present invention obtains function feature vectors through the function AST structure of the source code of the open source software; uses the function feature vectors to build a function feature dictionary for each TPL; merges the feature dictionaries of all TPLs to generate a vector index library, and records a set of function clone pairs; obtains the reuse frequency of each function based on the set of function clone pairs, and filters the common functions with abnormal reuse frequency to obtain a core vector index library; based on the core vector index library, performs TPL reuse detection on the software to be tested, and filters the misjudged TPL candidates according to the function feature proportion of the reused TPL candidates, and determines the reused TPL function set. Based on the function AST structural features, the present invention significantly improves the robustness of TPL detection in the modified reuse scenario, effectively reducing missed reports; based on the anomaly detection technology, filters the common functions whose frequency exceeds the normal range, reduces the influence of the interference function, improves the accuracy of TPL reuse detection, and effectively reduces false positives; adopts a dynamic proportion screening strategy to effectively filter the misidentified reused TPL candidates, further improving the accuracy of TPL reuse detection; at the same time, compared with the traditional method, the present method has higher detection efficiency and lower storage requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Schematic diagram of the overall framework of this method.

[0048] Figure 2 Schematic diagram of structural vector encoding.

[0049] Figure 3 Construct pseudocode for the feature dictionary.

[0050] Figure 4 Pseudocode for vector indexing merge.

[0051] Figure 5 Flowchart of the common function filtering method based on outlier detection.

[0052] Figure 6 This is a comparison chart of the accuracy of detection results of different methods. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings in the present invention, so as to have a further understanding of the concept of the present invention, the technical problems solved, the technical features constituting the technical solution and the technical effects brought about.

[0054] The overall flow chart of this method is as follows Figure 1 As shown, the specific steps include:

[0055] Step S1, function feature extraction: collecting source codes of large-scale open source software OSS (Open Source Software, OSS) and its different versions, and preprocessing the source codes, including the following steps:

[0056] S11. OSS data collection: Use Git API or crawlers to widely collect valid open source code repo repositories; for example, use the git tag command to obtain the version tag list of the repository and its release time, and use the git diff command to identify file changes between adjacent versions.

[0057] S12, OSS data preprocessing, including the following steps:

[0058] S121. For each source code file, eliminate unnecessary comments and blank lines to avoid interference from invalid information;

[0059] S122, based on the public source code parsing tool Tree-sitter, perform function segmentation on the source code and extract the function AST structure of the source code;

[0060] S123, according to the degree and order of the node types in the function AST structure, parsing the function AST structure into two vectors with different structural features, and performing feature concatenation on the two vectors to generate a function feature vector;

[0061] like Figure 2 As shown, the TreeCen algorithm is used to efficiently complete the conversion of function AST structure information to fixed-length vectors; the feature vector dimension needs to be determined by the necessary number of C / C++AST node types, which can be obtained through the configuration file node-types.json of the parsing tool;

[0062] In order to deal with the limitations of node degree features in Type II and Type III code cloning problems (such as variable name modification and position swapping of adjacent code nodes), this method parses the degree and sequence of node types in the function AST structure of the source code into vectors of two different structural features, and then generates the final function feature vector v by feature concatenation. The calculation formula is as follows:

[0063] v=βv1+(1-β)v2,β∈[0,1]

[0064] Among them, v1 represents the vector obtained by summing the degrees of each node type; v2 represents the vector obtained by summing the orders of each node type; β represents the weight of v1 in feature concatenation;

[0065] Based on the function AST structural characteristics, the present invention solves the failure problem of Type II and Type III code cloning (such as variable name modification and swapping of adjacent code node positions), significantly improves the robustness of TPL detection in modified reuse scenarios, and effectively reduces underreporting.

[0066] Step S2: In the function set segmented from the source code, filter the common structure functions based on the structural complexity of the function AST structure to reduce the impact of false positives;

[0067] Referring to the maintainability index, this method designs a metric filtering mechanism based on structural complexity, which includes the following steps:

[0068] S21. For a function, the set of structure types in its AST structure is A = {t1, t2, ..., t n}, where each structure type t i The number of occurrences is c i (c i ≥0), then the structural complexity C of the function is defined as:

[0069] C=|{t i |c i ≥1,i=1,2,…,n}|

[0070] Among them, c i ≥1 indicates the structure type t i Appears at least once in the function's AST; if c i =0, the structure type is not included in the complexity calculation; therefore, when the function actually uses fewer types of structure types, its structural complexity C is lower, otherwise it is higher.

[0071] S22. In the function set segmented from the source code, count the number of occurrences of different structure types in the AST structure of each function to obtain the structural complexity C of each function.

[0072] S23. Set a predefined threshold When the structural complexity of the function , the function is marked as a general structure function or a simple function and removed from the function set;

[0073] The present invention filters interference functions (general structure functions or simple functions) based on the structural complexity of the function AST structure, improves the accuracy of TPL reuse detection, and effectively reduces false alarms.

[0074] Step S3, using the function feature vector, constructing a function feature dictionary for each TPL through hash matching; merging the function feature dictionaries of all TPLs to generate a vector index library; in the process of merging the vector index library, recording all function clone pairs and storing them as a function clone pair set, including the following steps:

[0075] S31, feature dictionary construction; Since there are a large number of duplicate functions in different versions of TPL, the present invention uses hash matching to quickly build an independent feature dictionary for each TPL to achieve function deduplication; wherein the key of the feature dictionary is the function text hash value, and the value is the function feature of the earliest submission time; wherein the function feature is defined as a tuple describing the function attribute information: {function text hash value, function address, submission time, function feature vector};

[0076] like Figure 3 As shown in the figure, the construction process of a single TPL function feature dictionary is described. Where D represents a feature dictionary of a TPL, F represents a specific function feature, VF is a set of function features of a certain version, and VFL is a list of function attribute information of all versions of the TPL; in order to ensure that the stored function features are the earliest submitted version, the algorithm performs operations through the following strategies: if the text hash value of the current function F has not been recorded in the dictionary D, then F is directly appended and stored in the dictionary D; otherwise, the submission time of the new function F is compared; if the submission time of F is earlier, the record in the dictionary D is updated.

[0077] S32, vector index merging; introduce ANN search based on vector database, extract function feature vectors from the function feature dictionary of all TPLs for comparison and deduplication; in the deduplication process, give priority to retaining function feature vectors with earlier submission time, simplify indexes, merge and generate vector index libraries, and effectively eliminate duplicate functions and similar functions between different TPLs; Figure 4 As shown in the figure, by traversing the function feature dictionaries of all TPLs, they are finally merged into a global vector index library IM; where DL is the list of the function feature dictionary D; RL represents the matching result of the function feature vector set in the function feature dictionary D in IM; FT and FM represent the compared function features and their similar matching results respectively; δ is the set vector similarity threshold;

[0078] In this process, the retrieval log of function clone pairs is recorded for the subsequent common function filtering stage. For example, for two similar functions f1 and f2, assuming that f1 is submitted earlier than f2, there may be two situations: f1 is detected first, or f2 is detected first, and the latter situation also requires updating IM. Regardless of the order, all function clone pairs (f1, f2) must be recorded and stored as a function clone pair set, and it is also necessary to ensure that the function submitted earlier is f1, and f2 is its clone.

[0079] Step S4, based on the function clone pair set, obtain the reuse frequency of each function; in the vector index library, filter the public functions whose reuse frequency exceeds the normal frequency range, and obtain the core vector index library, including the following steps:

[0080] S41, frequency statistics based on path compression; Figure 5 As shown in the figure, based on the function clone pair set, a frequency statistical structure of Union-find disjoint sets (DSU) is constructed; wherein the root node of each set is the function with the earliest submission time, called the root function; the child nodes of each set are the remaining functions that have a clone relationship with the root function; the path compression algorithm is used to efficiently query the number of members (child nodes) of each set in the DSU, so as to obtain the reuse frequency of each root function; then, the root function frequency table is statistically calculated for each TPL, thereby obtaining the reuse frequency of each function.

[0081] S42, public function filtering based on Z-score; In order to efficiently filter public functions, this method counts the reuse frequencies of all functions and transforms the public function filtering problem into an anomaly detection problem; that is, when the reuse frequency of a function significantly deviates from the normal frequency range, it is regarded as an outlier and determined to be a public function; Since this is a single-dimensional outlier detection, the Z-score anomaly detection method based on statistical distribution is selected. First, the standardized distance Z (X f ), the calculation formula is as follows:

[0082] Z(X f )=(X f -μ) / σ

[0083] Among them, X f is the reuse frequency value of function f; Z(X f ) is the standardized distance; μ and σ are the mean and standard deviation of the function reuse frequency in the TPL to which function f belongs, respectively;

[0084] If Z(X f ) exceeds the specified distance threshold τ, it is considered as an outlier and filtered; considering that common functions usually show high-frequency features, only functions with too high reuse frequency values ​​are focused on, and reuse frequency values ​​below the mean μ are also considered to be within a reasonable range; therefore, the core function feature set B within the normal frequency range is ′ The filter conditions are as follows:

[0085] B ′ ={f∈B|Z(X f )≤τ}

[0086] Among them, B represents the function feature set of TPL to which function f belongs, B ′represents the set of function features within the normal frequency range in B, and the threshold τ is set to 1;

[0087] In the vector index library, filter the common function feature set whose reuse frequency exceeds the normal frequency range according to the screening conditions, extract the core function feature set of each TPL, and obtain the core vector index library;

[0088] The present invention filters the common functions whose frequencies exceed the normal range based on the anomaly detection technology, reduces the influence of the interference function (common function), improves the accuracy of TPL reuse detection, and effectively reduces false alarms.

[0089] Step S5, TPL reuse detection of the software to be tested; the goal of this step is to identify the approximate nearest neighbor function set of function features from the software item to be tested based on the vector index library, so as to determine the reused TPL candidate items; due to the data scale limitation, the TPL candidate results may contain inaccurate detection items, so a dynamic proportion filtering strategy for the reused TPL candidate items is designed to obtain a more accurate reused TPL detection result, including the following steps:

[0090] S51, preprocessing the source code of the software to be tested using the same method as step S1 to obtain a feature vector of the function to be tested;

[0091] The source code of the software to be tested is input into the system as a whole and preprocessed using the same method as step S1. Each function of the source code of the software to be tested is segmented, and the AST structure of the function to be tested is extracted. The feature extraction of the AST structure of the function to be tested is performed to generate a feature vector of the function to be tested.

[0092] S52. In the core vector index library, an approximate nearest neighbor algorithm is used to search for a function feature vector that approximately matches the function feature vector to be tested, and a third-party library set to which the function set corresponding to the found function feature vector belongs is used as a TPL candidate for reuse.

[0093] S53, filtering misidentified reused TPL candidates according to the function feature proportions of the reused TPL candidates, and then determining the final reused TPL detection result;

[0094] A dynamic proportion threshold strategy is adopted to optimize the screening process of reused TPL candidates. First, the proportion of function features corresponding to all reused TPL candidates is calculated. The calculation formula is as follows:

[0095]

[0096] Among them, p(T j ) is the function feature ratio corresponding to the jth reused TPL candidate, T j is the jth reuse TPL candidate, The software under test is related to T j The set of matching function features, T in the core vector index library j The corresponding function feature set;

[0097] For reused TPL candidates, sort them in ascending order according to their function feature proportions; remove TPL candidates that rank in the top 10% of the sorted results and whose function feature proportions are less than 2%, obtain more accurate reused TPL detection results, and output a detailed TPL detection report;

[0098] Traditional methods remove all TPL candidates whose function features account for less than 2%, which easily filters out some correct TPL candidates, thereby reducing the accuracy of TPL reuse detection; therefore, this strategy aims to consider the minimum top 10% of function features, filter out TPL candidates that are misidentified due to accidental similarities as much as possible, achieve a balance between filtering and correctness, and retain reused TPL candidates with higher reliability and significant features, further improving the accuracy of TPL reuse detection.

[0099] Step S6, based on the validation data set of 100 well-known open source projects, the TPL reuse detection accuracy of the TPLADD, CENTRIS and OSSFP methods is evaluated; candidate result filtering modules are designed for TPLADD and CENTRIS, and the experiment is carried out with and without the filtering module enabled, and the results of the two scenarios are recorded; the result of enabling the filtering module is represented by "(f)", for example, TPLADD(f) represents enabling filtering of TPL candidates, and TPLADD represents not enabling filtering of TPL candidates;

[0100] like Figure 6 As shown in the figure, when all filtering modules are turned on, compared with the existing method CENTRIS, the precision and recall of the proposed method TPLADD are improved by 3.88% and 2.76% respectively. It can be seen that the proposed method significantly improves the accuracy of TPL reuse detection; where TP is true positive, PRE is precision, REC is recall, and F1 is F1 value;

[0101] At the same time, in terms of time, this method is 21.9% faster than the CENTRIS method, and it takes only 0.42 seconds to detect a TPL on average, which significantly improves the detection efficiency of TPL reuse detection; in terms of space, this method only needs to retain 0.41% of the overall function features. Compared with CENTRIS and OSSFP, the size of the vector index library is only 20% of CENTRIS, which significantly reduces the storage requirements of TPL reuse detection.

[0102] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A C / C++ source code third-party library reuse detection method based on function structure information, characterized in that: The following steps are involved: Step S1: collecting source codes of different versions of large-scale open source software and preprocessing the source codes. The preprocessing includes the following steps: S11, dividing the source code into functions and extracting the function AST structure of the source code; S12, according to the degree and order of the node types in the function AST structure, parsing the function AST structure into two vectors with different structural features, and performing feature concatenation on the two vectors to generate a function feature vector; Step S2: Filter the common structure functions in the function set segmented from the source code based on the structural complexity of the function AST structure; Step S3: Use the function feature vector to construct a function feature dictionary for each TPL through hash matching; merge the function feature dictionaries of all TPLs to generate a vector index library; in the process of merging the vector index library, record all function clone pairs and store them as a function clone pair set; Step S4, obtaining the reuse frequency of each function based on the function clone pair set; in the vector index library, filtering the public functions whose reuse frequency exceeds the normal frequency range to obtain the core vector index library; Step S5: Based on the core vector index library, an approximate nearest neighbor algorithm is used to perform TPL reuse detection on the software under test to obtain reused TPL candidates; according to the function feature ratio of the reused TPL candidates, the misidentified reused TPL candidates are filtered out to determine the reused TPL detection result.

2. The C / C++ source code third-party library reuse detection method based on function structure information as claimed in claim 1, characterized in that: The step S12 specifically includes: According to the degree and order of the node types in the function AST structure, the TreeCen algorithm is used to parse the function AST structure into two vectors with different structural features, and the two vectors are concatenated to generate the function feature vector. The calculation formula is as follows: v=βv1+(1-β)v2,β∈[0,1] Among them, v1 represents the vector obtained by the sum of the degrees of each node type, v2 represents the vector obtained by the sum of the orders of each node type, and β represents the weight of v1 in feature concatenation.

3. The C / C++ source code third-party library reuse detection method based on function structure information as claimed in claim 1, characterized in that: The step S2 comprises the following steps: S21. The structural complexity C of a function is defined as: C=|{t i |c i ≥1,i=1,2,…,n}| Among them, t i is the i-th structure type in the function AST structure, c i is the number of occurrences of the i-th structural type in the function AST structure; the total number of structural types that appear more than or equal to 1 in the function AST structure is the structural complexity of the function; S22. In the function set segmented from the source code, count the number of occurrences of different structure types in the AST structure of each function to obtain the structural complexity C of each function; S23: Eliminate functions whose structural complexity C is less than a preset threshold in the set of functions The general structure function of 4. The C / C++ source code third-party library reuse detection method based on function structure information as claimed in claim 1, characterized in that: The step S3 comprises the following steps: S31. Use the function text hash value as the dictionary key, use the function feature of the earliest submission time as the dictionary value, and build a function feature dictionary for each TPL through hash matching; wherein the function feature is defined as a tuple describing the function attribute information: {function text hash value, function address, submission time, function feature vector}; S32, extracting function feature vectors from the function feature dictionaries of all TPLs; comparing and removing duplicates of all function feature vectors by using an approximate nearest neighbor algorithm, giving priority to retaining function feature vectors with earlier submission times, and then merging the function feature dictionaries of all TPLs to generate a vector index library; in the process of merging the vector index library, all similar function pairs are merged. a 、f b Clone the pair (f a ,f b ) is stored as a set of function clone pairs, and the function with the earlier submission time is ensured to be f a , and f b It is its clone.

5. The C / C++ source code third-party library reuse detection method based on function structure information as claimed in claim 1, characterized in that: The step S4 comprises the following steps: S41, constructing a union-find set DSU by using function clones, taking the function with the earliest submission time as the root node of each set in the union-find set, and taking the remaining functions that have a clone relationship with the function with the earliest submission time as the child nodes of each set in the union-find set; querying the number of child nodes in each set and in the union-find set by a path compression algorithm, and obtaining the reuse frequency of each function; S42. The Z-score anomaly detection method based on statistical distribution is used to calculate the standardized distance of function reuse frequency. The calculation formula is as follows: Z(X f )=(X f -m) / s Among them, X f is the reuse frequency value of function f; Z(X f ) is the standardized distance; μ and σ are the mean and standard deviation of the function reuse frequency in the TPL to which function f belongs, respectively; The filtering conditions for the core function feature set within the normal frequency range are as follows: B ′ ={f∈B|Z(X f )≤τ} Among them, B represents the function feature set of TPL to which function f belongs, B ′ represents the set of function features within the normal frequency range in B, and the threshold τ is set to 1; In the vector index library, the common function feature set whose reuse frequency exceeds the normal frequency range is filtered according to the screening condition, the core function feature set within the normal frequency range is retained, and the core vector index library is obtained.

6. The C / C++ source code third-party library reuse detection method based on function structure information as claimed in claim 1, characterized in that: The step S5 comprises the following steps: S51, preprocessing the source code of the software to be tested using the same method as step S1 to obtain a feature vector of the function to be tested; S52, in the core vector index library, using the approximate nearest neighbor algorithm to find a function feature vector that approximately matches the function feature vector to be tested, and taking the third-party library set to which the function set corresponding to the found function feature vector belongs as a candidate for reuse of TPL; S53. Calculate the proportion of function features corresponding to all reused TPL candidates. The calculation formula is as follows: Among them, p(T j ) is the function feature ratio corresponding to the jth reused TPL candidate, T j is the jth reuse TPL candidate, The software under test is related to T j The set of matching function features, T in the core vector index library j The corresponding function feature set; the reused TPL candidates are sorted in ascending order according to their function feature proportions, and the TPL candidates ranked in the top 10% of the sorted results and whose function feature proportions are less than 2% are removed to obtain the reused TPL detection results.

Images