A file tree matching approximate retrieval method and system based on a locality sensitive hashing algorithm

Abstract

The application relates to a file tree matching approximate retrieval method and system based on a local sensitive hashing algorithm, which comprises the following steps: calculating the sha1 values of sub-file trees and sub-files under a file tree through a local sensitive hashing algorithm, and generating a hash vector of the file tree; judging the similarity between file trees by calculating the distance between the hash vectors of two file trees; performing multiple clustering on the hash values of the file trees through a hierarchical clustering algorithm; in the search, starting from the clustering center point of the highest layer, comparing to obtain the nearest point, then searching the nearest child node under the point, and traversing all nodes in the lowest layer to return the top-k most similar file tree nodes. According to the application, a user inputs a file tree path, the program is parsed, the local sensitive hash value of the file tree is calculated while the traditional hash signature is generated, then the hierarchical clustering algorithm is used to perform multiple-layer clustering on the local sensitive hash value, and in the retrieval, the similar open source component library is quickly obtained through the clustering center nodes of each layer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data security, and particularly relates to a file tree approximate matching retrieval method and system based on a local sensitive hashing algorithm. BACKGROUND

[0002] Software composition analysis (SCA) is an automatic process of identifying open source software in a code library, and file and file tree matching is an important technology in SCA; a traditional method is to map a file or a file tree into a plurality of hash signatures, such as mapping file content into a 160-bit or 128-bit 0-1 vector value through a sha1 or md5 algorithm as a file signature, and for a file tree, extracting the names, sizes and hash values of subfiles and subfile trees under the file tree as features, sorting and splicing, and then generating a sha1 or md5 value as a signature of the file tree. However, this method cannot be used for approximate query, and in the actual software development process, there are many phenomena of modifying references to open source components, and users hope to detect file trees with only a small number of file changes, which requires approximate query; and the traditional method is based on a bottom-up hash signature, and a small change in a file will cause all nodes on its parent path to have completely different hash signatures. In this case, the traditional method can only match the subfiles or subfile trees, and misses the overall similarity of the file tree.

[0003] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and provides a file tree matching approximate retrieval method and system based on a local sensitive hashing algorithm, which solves the problem that the traditional scheme can only perform accurate matching and cannot identify slightly changed component file trees.

[0005] The purpose of the present application is achieved by the following technical solution: a file tree matching approximate retrieval method based on a local sensitive hashing algorithm, the approximate retrieval method comprising:

[0006] S1, calculating the sha1 values of the subfile trees and subfiles under the file tree through a local sensitive hashing algorithm, and generating a hash vector of the file tree, and judging the similarity between the file trees by calculating the distance between the hash vectors of the two file trees;

[0007] S2. The hash value of the file tree is clustered multiple times using a hierarchical clustering algorithm. During the search, the nearest point is obtained by comparing the cluster center point of the highest level, and then the nearest child node under that point is searched. All nodes are traversed at the lowest level, and the top-k most similar file tree nodes are returned.

[0008] Step S1 specifically includes the following:

[0009] S11. Traverse the nodes under the file tree in sequence. If a node is a file (i.e., a subfile), calculate the SHA1 value based on the subfile content using the java.security.MessageDigest class. It then reads the contents of the sub-files in a loop and inputs them into the MessageDigest function, finally converting the sub-file contents of uncertain length into hash values ​​of fixed length;

[0010] S12. If the node is a file tree, i.e. a sub-file tree, check whether a local sensitive hash value has been generated. If not, return to step S11 to recursively traverse the nodes under the sub-file tree. If the node is a file, generate the SHA1 value according to step S11. If the node is a file tree, recursively execute step S12.

[0011] S13. Sum the file sizes of all child nodes: sum = ∑ i size i Then, the child node file size is normalized to 0-1 to form the weight w. i =size i / sum, where size represents the file size and i represents the i-th file;

[0012] S14. Perform a weighted summation of each bit of the hash value of the child nodes: get This represents the k-th element of the vector of the j-th child node; This represents the k-th element of the new vector;

[0013] S15. The distance between two file trees that are approximated by calculation. This is used to measure the similarity between file trees. The smaller the distance, the more similar the two file trees are; conversely, the larger the distance, the greater the differences between the file trees, meaning there are more dissimilar files. 'a' and 'b' represent two file trees being compared approximately. Representing vectors The l-th element, Representing vectors The l-th element.

[0014] Step S2 specifically includes the following:

[0015] S21, set the file tree set as S, one element in S corresponds to the local sensitive hash value of the file tree;

[0016] S22, perform k-means clustering algorithm on all sets in S, and divide all sets in S into k categories;

[0017] S23, on the basis of the k categories in step S22, perform k-means clustering algorithm again to generate k / m categories;

[0018] S24, repeat steps S22 and S23 until the number of categories is less than or equal to m;

[0019] S25, search from the highest layer category to find the nearest category center point with the searched hash value, and then recursively search the categories under the center point until the last layer category, and the file tree information under the category is included.

[0020] The k-means clustering algorithm includes the following contents:

[0021] A1, randomly select k points as the center of gravity as k categories, respectively calculate the distance between other samples and the k category points, and the sample category is the same as its nearest center type;

[0022] A2, in each category, calculate the average vector value of the vector of all samples to obtain a new center of gravity;

[0023] A3, repeat steps A1 and A2 for N rounds, or when the distance between the new and old centers of gravity is less than the set threshold, stop iteration.

[0024] A file tree matching approximate retrieval system based on a local sensitive hash algorithm includes a local sensitive hash calculation module and a hierarchical clustering calculation module;

[0025] The local sensitive hash calculation module is used to calculate the sha1 value of the sub-file tree and the sub-file under the file tree through the local sensitive hash algorithm, and generate the hash vector of the file tree, and judge the similarity between the file trees by calculating the distance between the hash vectors of the two file trees;

[0026] The hierarchical clustering calculation module is used to perform multiple clustering on the hash value of the file tree through the hierarchical clustering algorithm, and in the search, start from the highest layer clustering center point, compare to get the nearest point, then search the nearest child node under the point, and traverse all nodes in the lowest layer, and return the top-k most similar file tree nodes.

[0027] The specific calculation steps of the local sensitive hash calculation module include the following contents:

[0028] Step one, traversing the nodes under the file tree in turn, if the node is a file, i.e. a sub-file, then according to the sub-file content, the sha1 value is calculated by the java.security.MessageDigest class and the sub-file content of uncertain length is converted into a fixed length hash value;

[0029] Step two, if the node is a file tree, i.e. a sub-file tree, then it is checked whether the local sensitive hash value has been generated, if not, then it is returned to step one to recursively traverse the nodes under the sub-file tree, if the node is a file, then the sha1 value is generated according to step two, if the node is a file tree, then step two is recursively executed;

[0030] Step three, the file size sum of all sub-nodes is calculated i size i , and then the sub-node file size is normalized to 0-1 to form the weight w i =size i / sum, wherein size represents the file size, and i represents the i-th file;

[0031] Step four, the weighted sum of each bit of the hash value of the sub-node is calculated: , wherein represents the k-th element of the j-th sub-node vector; represents the k-th element of the new vector;

[0032] Step five, the distance between the two file trees for approximate comparison is calculated to measure the similarity between the file trees, the smaller the distance, the more similar the two file trees, and vice versa, the larger the distance, the greater the difference between the file trees, i.e. the more different files, a and b represent the two file trees for approximate comparison, represents the l-th element of the vector , and represents the l-th element of the vector .

[0033] The hierarchical clustering calculation module specifically calculates the following steps:

[0034] Step one, set the file tree set as S, and one element in S corresponds to the local sensitive hash value of the file tree;

[0035] Step two, perform the k-means clustering algorithm on all sets in S to divide all sets in S into k categories;

[0036] Step three, on the basis of the k categories in step two, the k-means clustering algorithm is performed again to generate k / m categories;

[0037] Step four, repeat steps S22 and S23 until the number of categories is less than or equal to m; m is an empirical value that needs to be manually set, and a suitable m value is conducive to search; if m is too small, the number of categories is large, and the number of nodes in each category is too small, and the number of search layers is small; if m is too large, the number of categories is small, and the number of search layers is large;

[0038] Step five, starting from the highest layer category, searching for the nearest category center point of the searched hash value, and then recursively searching the category under the center point until the last layer category, which contains the file tree information under the category.

[0039] A computer readable storage medium, having stored thereon a computer program, the computer program being executed by a processor to implement the steps of the approximate retrieval method.

[0040] A terminal device, comprising a memory and a processor, the memory having stored thereon a computer program, the processor executing the computer program to implement the steps of the approximate retrieval method.

[0041] The present application has the following advantages: a file tree matching approximate retrieval method and system based on a local sensitive hash algorithm, which can be integrated into an SCA system for similar search of file trees. The user inputs the file tree path, which is parsed by the program to calculate the local sensitive hash value of the file tree while generating the traditional hash signature. Then, the hierarchical clustering algorithm is used to perform multi-layer clustering on the local sensitive hash value. During retrieval, the similar open source component library is quickly obtained through the clustering center nodes of each layer. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 It is a flowchart of the method of the present application;

[0043] Figure 2 It is a flowchart of the local hash algorithm;

[0044] Figure 3 It is a schematic diagram of a local sensitive hash algorithm calculation example. DETAILED DESCRIPTION

[0045] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in combination with the drawings is not intended to limit the protection scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work belong to the protection scope of the present application. The present application will be further described below in combination with the drawings.

[0046] As shown in Figure 1 one of the embodiments of the present application relates to a file tree matching approximate search method based on a local sensitive hashing algorithm, the approximate search method comprising:

[0047] S1, through the local sensitive hashing algorithm: the file tree is mapped into a 160-dimensional hash vector, and each value in the vector ranges between 0 and 1. The similarity between the file trees is measured by the Manhattan distance of two vectors, and the smaller the distance is, the more similar they are. The hash vector of the file tree is generated by its child file tree and child file; for a file, the sha1 value is directly generated based on the content; for a file tree, the normalized weight is obtained according to the size of the child file tree or the child file, and then the weighted sum of each bit of its hash value is obtained, to obtain a value between 0 and 1. This newly generated vector is taken as the local sensitive hash value of the file tree. The hash value of the underlying file tree is obtained first, and then the hash value of each node of the entire file tree is generated from bottom to top;

[0048] As shown in Figure 2 , specifically comprising the following contents:

[0049] S11, the nodes under the file tree are traversed in turn, if the node is a file, i.e. a child file, the sha1 value is calculated according to the content of the child file through the java.security.MessageDigest class and the content of the child file is read into the MessageDigest function, and finally the content of the child file with uncertain length is converted into a hash value with fixed length;

[0050] S12, if the node is a file tree, i.e. a child file tree, it is checked whether the local sensitive hash value has been generated, if not, it is returned to step S11 to recursively traverse the nodes under the child file tree, if the node is a file, the sha1 value is generated according to step S11, if the node is a file tree, step S12 is recursively executed;

[0051] S13. Sum the file sizes of all child nodes: sum = ∑ i size i Then, the child node file size is normalized to 0-1 to form the weight w. i =size i / sum, where size represents the file size and i represents the i-th file;

[0052] S14. Perform a weighted summation of each bit of the hash value of the child nodes: get This represents the k-th element of the vector of the j-th child node; This represents the k-th element of the new vector;

[0053] S15. The distance between two file trees that are approximated by calculation. This is used to measure the similarity between file trees. The smaller the distance, the more similar the two file trees are; conversely, the larger the distance, the greater the differences between the file trees, meaning there are more dissimilar files. 'a' and 'b' represent two file trees being compared approximately. Representing vectors The l-th element, Representing vectors The l-th element.

[0054] Because the SHA1 algorithm is a random mapping, for a file tree, the more files that have been modified, the greater the difference in hash values ​​of the merged child nodes, and the larger the final Manhattan distance. Since the hash vector length is 160, the minimum distance is 0 and the maximum distance is 160.

[0055] like Figure 3 As shown, the input file tree root node h is not a file, so we traverse the child nodes under h; child node W1 is a sub-file tree, so we continue to traverse the nodes under W1. Child node W1 has two child nodes, w1 and w2. Since w1 and w2 are file types, we call the MessageDigest class to generate SHA1 values ​​h1 and h2; returning to node W1, since all its child nodes have generated hash values, we summarize and weight them to obtain the hash value h3 of W1; returning to node h, based on traversing the child nodes of h to W2, W2 is a file, so we directly generate the SHA1 value h4; returning to node h, we summarize the hash values ​​generated by its child nodes, and the hash value is now generated. This value can be used to calculate the distance with the hash values ​​of other nodes.

[0056] S2. The hash value of the file tree is clustered multiple times using a hierarchical clustering algorithm. During the search, the nearest point is obtained by comparing the cluster center point of the highest level, and then the nearest child node under that point is searched. All nodes are traversed at the lowest level, and the top-k most similar file tree nodes are returned.

[0057] Specifically includes the following contents:

[0058] S21, set the file tree set as S, one element in S corresponds to the local sensitive hash value of the file tree;

[0059] S22, perform k-means clustering algorithm on all sets in S, and divide all sets in S into k categories;

[0060] S23, on the basis of the k categories in step S22, perform k-means clustering algorithm again to generate k / m categories;

[0061] S24, repeat steps S22 and S23 until the number of categories is less than or equal to m; m is an empirical value that needs to be set manually, and a suitable m value is beneficial to search; if m is too small, the number of categories is too large, and the number of nodes in each category is too small, so the number of search layers is small; if m is too large, the number of categories is small, and the number of search layers is large

[0062] S25, start searching from the highest layer category, find the nearest category center point with the searched hash value, and then recursively search the category under the center point until the last layer category, which contains the file tree information under the category.

[0063] Further, the k-means clustering algorithm includes the following contents:

[0064] A1, randomly select k points as the center of gravity as k categories, respectively calculate the distance between other samples and the k category points, and classify the sample category with the nearest center of gravity type;

[0065] A2, in each category, calculate the average vector value of the vector of all samples to obtain a new center of gravity;

[0066] A3, repeat steps A1 and A2 for N rounds, or when the distance between the new and old centers of gravity is less than the set threshold, stop iteration.

[0067] Another embodiment of the application relates to a file tree matching approximate retrieval system based on a local sensitive hash algorithm, which comprises a local sensitive hash calculation module and a hierarchical clustering calculation module;

[0068] Further, the local sensitive hash calculation module is used to calculate the sha1 value of the sub-file tree and the sub-file under the file tree through the local sensitive hash algorithm, and generate the hash vector of the file tree, and judge the similarity between the file trees by calculating the distance between the hash vectors of the two file trees;

[0069] Further, the hierarchical clustering calculation module is configured to perform multiple clustering on the hash values of the file trees by a hierarchical clustering algorithm, and in the searching, a nearest point is obtained by comparing the clustering center points from the highest layer, and then a nearest child node under the point is searched, all nodes in the lowest layer are traversed, and top-k most similar file tree nodes are returned.

[0070] The local sensitive hash calculation module specifically includes the following steps in the calculation process:

[0071] Step one: traversing the nodes under the file tree in sequence, if the node is a file, i.e., a child file, then the sha1 value is calculated according to the content of the child file by using the java.security.MessageDigest class and the content of the child file is read into the MessageDigest function in a loop, and finally the content of the child file with an uncertain length is converted into a hash value with a fixed length;

[0072] Step two: if the node is a file tree, i.e., a child file tree, then it is checked whether the local sensitive hash value has been generated, if not, then it is returned to step one to recursively traverse the nodes under the child file tree, if the node is a file, then the sha1 value is generated according to step two, if the node is a file tree, then step two is recursively executed;

[0073] Step three: summing up the file sizes of all child nodes sum = ∑ i size i , and then normalizing the file sizes of the child nodes to 0-1 to form the weight w i = size i / sum, wherein size represents the file size, and i represents the i-th file;

[0074] Step four: performing weighted summation on each bit of the hash value of the child node: obtaining , wherein represents the k-th element of the j-th child node vector; represents the k-th element of the new vector;

[0075] Step five: measuring the similarity between the two file trees by calculating the distance between the two file trees , the smaller the distance, the more similar the two file trees, and vice versa, the larger the distance, the larger the difference between the file trees, i.e., the more different files, wherein a and b represent the two file trees for approximate comparison, represents the l-th element of the vector , and represents the l-th element of the vector .

[0076] The hierarchical clustering calculation module specifically includes the following steps in the calculation process:

[0077] Step one, set the file tree set as S, an element in S corresponds to the local sensitive hash value of the file tree;

[0078] Step two, perform k-means clustering algorithm on all sets in S, and divide all sets in S into k categories;

[0079] Step three, on the basis of the k categories in step two, perform k-means clustering algorithm again to generate k / m categories;

[0080] Step four, repeat steps S22 and S23 until the number of categories is less than or equal to m;

[0081] Step five, search from the highest layer category, find the nearest category center point with the searched hash value, and then recursively search the categories under the center point until the last layer category, and the file tree information under the category is contained.

[0082] Still another embodiment of the present application relates to a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the approximate search method.

[0083] The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0084] Still another embodiment of the present application relates to a terminal device, which includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of the approximate search method.

[0085] The foregoing is considered as illustrative only of the principles of the application. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the application to the exact construction and operation described. Accordingly, all such variations are intended to be included within the scope of the present application as defined in the claims below and their equivalents.

Claims

1. A file tree matching approximate retrieval method based on the locality-sensitive hashing algorithm, characterized in that: The approximate retrieval method includes: S1. Using the Locality Sensitive Hashing algorithm, the file tree is mapped into a 160-dimensional hash vector. Each value in the vector ranges from 0 to 1. The similarity between file trees is measured by the Manhattan distance between two vectors. The smaller the distance, the more similar the file trees are. The hash vector of a file tree is generated from its sub-file trees and sub-files. For files, the SHA1 value is generated directly based on the content; for file trees, normalized weights are obtained based on the size of sub-file trees or sub-files, and then each bit of the hash value is weighted and summed to obtain a value between 0 and 1. This newly generated vector is used as the local sensitive hash value of the file tree. First, the hash value of the underlying file tree is obtained, and then the hash value of each node of the entire file tree is generated from the bottom up. S2. The hash value of the file tree is clustered multiple times using a hierarchical clustering algorithm. During the search, the nearest point is obtained by comparing the cluster center point of the highest level, and then the nearest child node under that point is searched. All nodes are traversed at the lowest level, and the top-k most similar file tree nodes are returned.

2. The file tree matching approximate retrieval method based on the locality-sensitive hashing algorithm according to claim 1, characterized in that: Step S1 specifically includes the following: S11. Traverse the nodes under the file tree in sequence. If a node is a file (i.e., a subfile), calculate the SHA1 value based on the subfile content using the java.security.MessageDigest class. It then reads the contents of the sub-files in a loop and inputs them into the MessageDigest function, and finally converts the sub-file contents of uncertain length into hash values ​​of fixed length. S12. If the node is a file tree, i.e. a sub-file tree, check whether a local sensitive hash value has been generated. If not, return to step S11 to recursively traverse the nodes under the sub-file tree. If the node is a file, generate the SHA1 value according to step S11. If the node is a file tree, recursively execute step S12. S13. Sum the file sizes of all child nodes. Then, the size of the child node file is normalized to 0~1 to form the weight. Where size represents the file size, and i represents the i-th file; S14. Perform a weighted summation of each bit of the hash value of the child nodes: ,get , This represents the k-th element of the vector of the j-th child node; This represents the k-th element of the new vector; S15. The distance between two file trees that are approximated by calculation. This is used to measure the similarity between file trees. The smaller the distance, the more similar the two file trees are; conversely, the larger the distance, the greater the differences between the file trees, meaning there are more dissimilar files. 'a' and 'b' represent two file trees being compared approximately. Representing vectors The l-th element, Representing vectors The l-th element.

3. The file tree matching approximate retrieval method based on the locality-sensitive hashing algorithm according to claim 1, characterized in that: Step S2 specifically includes the following: S21. Let S be the set of file trees, where an element in S corresponds to the local sensitivity hash value of the file tree. S22. Perform k-means clustering on all sets in S to divide all sets in S into k categories; S23. Based on the k categories in step S22, perform the k-means clustering algorithm again to generate k / m categories; S24. Repeat steps S22 and S23 until the number of categories is less than or equal to m; S25. Start the search from the highest level category, find the category center point that is closest to the retrieved hash value, and then recursively search the categories under that center point until the last level category, which contains the file tree information under that category.

4. The file tree matching approximate retrieval method based on the locality-sensitive hashing algorithm according to claim 3, characterized in that: The k-means clustering algorithm includes the following: A1. Randomly select k points as centroids and k categories, calculate the distance between other samples and these k categories, and classify the sample category as the same as its nearest centroid type; A2. Under each category, calculate the average vector value of all its samples to obtain the new centroid; A3. Repeat steps A1 and A2 for N rounds, or stop iterating when the distance between the old and new centroids is less than the set threshold.

5. A file tree matching approximate retrieval system based on the locality-sensitive hashing algorithm, characterized in that: It includes a locality-sensitive hash calculation module and a hierarchical clustering calculation module; The Locality Sensitive Hash (LSH) calculation module is used to map a file tree into a 160-dimensional hash vector using the LSH algorithm. Each value in the vector is between 0 and 1. The similarity between file trees is measured by the Manhattan distance between two vectors. The smaller the distance, the more similar the file trees are. The hash vector of a file tree is generated from its sub-file trees and sub-files. For files, the SHA1 value is generated directly based on the content; for file trees, normalized weights are obtained based on the size of sub-file trees or sub-files, and then each bit of the hash value is weighted and summed to obtain a value between 0 and 1. This newly generated vector is used as the local sensitive hash value of the file tree. First, the hash value of the underlying file tree is obtained, and then the hash value of each node of the entire file tree is generated from the bottom up. The hierarchical clustering calculation module is used to perform multiple clusterings on the hash value of the file tree using a hierarchical clustering algorithm. During the search, it starts from the cluster center point of the highest level, compares to obtain the nearest point, and then searches for the nearest child node under that point. It traverses all nodes at the lowest level and returns the top-k most similar file tree nodes.

6. A file tree matching approximate retrieval system based on the locality-sensitive hashing algorithm according to claim 5, characterized in that: The specific calculation steps of the locality-sensitive hash calculation module include the following: Step 1: Traverse the nodes in the file tree sequentially. If a node is a file (i.e., a subfile), calculate the SHA1 value based on the subfile's content using the java.security.MessageDigest class. It then reads the contents of the sub-files in a loop and inputs them into the MessageDigest function, and finally converts the sub-file contents of uncertain length into hash values ​​of fixed length. Step 2: If the node is a file tree, i.e. a sub-file tree, check if a local sensitive hash value has been generated. If not, return to Step 1 and recursively traverse the nodes under the sub-file tree. If the node is a file, generate the SHA1 value according to Step 2. If the node is a file tree, recursively execute Step 2. Step 3: Sum the file sizes of all child nodes. Then, the size of the child node file is normalized to 0~1 to form the weight. Where size represents the file size, and i represents the i-th file; Step 4: Perform a weighted summation of each bit of the hash value of the child nodes: ,get , This represents the k-th element of the vector of the j-th child node; This represents the k-th element of the new vector; Step 5: Calculate the distance between the two file trees for approximate comparison. This is used to measure the similarity between file trees. The smaller the distance, the more similar the two file trees are; conversely, the larger the distance, the greater the differences between the file trees, meaning there are more dissimilar files. 'a' and 'b' represent two file trees being compared approximately. Representing vectors The l-th element, Representing vectors The l-th element.

7. A file tree matching approximate retrieval system based on the locality-sensitive hashing algorithm according to claim 5, characterized in that: The specific calculation steps of the hierarchical clustering calculation module include the following: Step 1: Let the set of file trees be S, where an element in S corresponds to the local sensitive hash value of the file tree; Step 2: Perform k-means clustering on all sets in S to divide all sets in S into k categories; Step 3: Based on the k categories in Step 2, perform the k-means clustering algorithm again to generate k / m categories; Step 4: Repeat steps S22 and S23 until the number of categories is less than or equal to m, where m is a set empirical value; Step 5: Start the search from the highest level category, find the category center point that is closest to the retrieved hash value, and then recursively search the categories under that center point until the last level category, which contains the file tree information under that category.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the approximate retrieval method according to any one of claims 1-4.

9. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the approximate retrieval method according to any one of claims 1-4.

Images