A software and process matching method based on similarity calculation

Abstract

The present invention discloses a software and process matching method based on similarity calculation, including: S1, constructing a software and process matching framework; S2, data synchronization, using the IP-guard terminal security management system to regularly collect software installation and application process operation information; S3, data cleaning the source data, and then formatting the data; S4, rough similarity pre-screening and precise similarity calculation; S5, generating software program rules based on the software and process matching framework. The present invention constructs a software and process matching framework to intelligently and efficiently match and associate the software with the application process running in the task manager to obtain the real operation data of the software; secondly, the rough similarity algorithm is used to evaluate the similarity between the string in the output set and the string to be detected, quickly filter out the dissimilar strings, and reduce the number of strings that need to be accurately calculated for similarity, thereby improving the efficiency of the entire query process.

Description

Technical Field

[0001] The present invention belongs to the technical field of terminal security management, and more specifically, particularly relates to a software and process matching method based on similarity calculation. Background Art

[0002] In the information age, software is an essential tool for corporate office work. Compliance inspection of software installation is an important part of enterprise terminal management. Employees have different knowledge of IT computers, and employees will bring various problems to the enterprise when installing software, such as virus infection, data leakage, and software copyright risks. In order to achieve unified management of terminal software, the first thing is to be able to quickly and accurately grasp the software installation and use of the current terminal computer. However, most application software is constantly iterating and updating, resulting in many software versions, and even the software names or application processes corresponding to some versions have changed. This has caused great trouble to software installation and usage statistics, especially for large-scale enterprises with a large number of terminal users, making this statistical work even more difficult. Summary of the invention

[0003] In view of the problems existing in the prior art, the purpose of the present invention is to provide a software and process matching method based on similarity calculation. By constructing a software and process matching framework, the software can be intelligently and efficiently matched and associated with the application processes running in the task manager, and the real software operation data can be obtained, thereby assisting enterprises in unifying the software usage of terminal computers.

[0004] To achieve the above object, the present invention provides the following technical solution: a software and process matching method based on similarity calculation, comprising the following steps:

[0005] S1. Build a software and process matching framework to calculate the similarity between the software name and the process name, as well as the similarity between the software path and the process path;

[0006] S2. Data synchronization: Use the IP-guard terminal security management system to regularly collect software installation and application process operation information as source data for subsequent services;

[0007] S3. Clean the source data, format the data, and convert it into a string in a unified format;

[0008] S4, matching the similarity of the character strings through rough similarity pre-screening and precise similarity calculation;

[0009] S5. Generate software program rules based on the software and process matching framework.

[0010] Optionally, in step S1, the specific process of the software and process matching framework includes:

[0011] S11, obtaining software text information and converting the text information into a string format;

[0012] S12, segmenting and grouping the character string by segmentation marks to form class bucket identifiers;

[0013] S13, converting the class bucket identifier into a conventional format using a binary method;

[0014] S14, build classification index map ;

[0015] S15, remove risky common sequences and single-digit buckets to filter out oversized buckets;

[0016] S16, performing similarity character query and deduplication through bucket identifiers;

[0017] S17, optimization of precise similarity calculation based on rough similarity pre-screening.

[0018] Optionally, the specific steps of step S12 are:

[0019] 1) receiving an input string containing multiple substrings, wherein the substrings are separated by split markers;

[0020] 2) Split the input string into multiple substrings according to the segmentation markers;

[0021] 3) Adoption n Alphabetical sorting, starting from the end of the substring, each consecutive n characters form a bucket identifier, which is not enough n characters directly form a bucket-like identifier;

[0022] 4) The class buckets formed by all groups are merged to generate a set, which is recorded as the class bucket identifier of the original string.

[0023] Optionally, the specific steps of step S13 are:

[0024] 1) Determine the base number as base 27, which is used to represent a character set consisting of 0 and characters;

[0025] 2) receiving the class bucket identifier of the original string generated in step S12;

[0026] 3) Traverse the characters in the bucket identifier of the original string and determine the value in base 27 based on the position of the character in the character set sequence;

[0027] 4) From left to right, multiply the character values ​​by the corresponding power of 27, add the products, and output the unique integer identifier of the bucket-like identifier of the original string.

[0028] Optionally, in step S14, the specific process of building the classification index includes: defining the data structure, loading the database, calculating the bucket identifier, updating the mapping relationship, and outputting the classification index. map ;

[0029] Defining a data structure includes defining a mapping structure map ;

[0030] Among them: the key is an unsigned 32-bit integer used to represent the bucket identifier;

[0031] The value is an unsigned 32-bit integer array, the elements in the integer array represent the serial numbers of the original strings, and are used to point to the corresponding items in the original string set;

[0032] The database is a matching library storing a large number of character strings;

[0033] The calculation of the class bucket identifier: traverse each string in the database, and calculate the class bucket identifier according to the characteristics of the string, which is recorded as map Keys;

[0034] The updating of the mapping relationship includes: detecting map Whether there is a duplicate key of the bucket identifier in the

[0035] 1) If it exists, add the sequence number of the string to the integer array of the corresponding bucket identifier;

[0036] 2) If it does not exist, then map Create a new key-value pair in , where the key is the bucket identifier of the string and the value is a single-element array containing the string sequence number;

[0037] The output classification index map : Traverse the strings of the entire database and generate a complete classification index map , where each class bucket identifier is mapped to an integer array, which contains the serial numbers of all strings assigned to the class bucket identifier.

[0038] Optionally, step S15 includes: common sequence exclusion and single digital bucket exclusion;

[0039] The public sequence exclusion: before allocating the string to the corresponding class bucket according to the class bucket identifier, exclude the string that will lead to the formation of an oversized class bucket, and such string is recorded as a risk public sequence;

[0040] The single number bucket exclusion: excludes strings consisting of one or two characters.

[0041] Optionally, in step S16, the specific process of similarity character query and deduplication includes:

[0042] 1) Select the string to be processed as the query target;

[0043] 2) Calculate the class bucket identifier based on the string to be processed;

[0044] 3) Using the class bucket identifier obtained above, from the constructed classification index map Query and find all strings that can map to the bucket identifier of this type, and generate an output set;

[0045] 4) Remove duplicate strings from the output set;

[0046] 5) Output the deduplicated output set as the query result.

[0047] Optionally, the specific process of step S17 is:

[0048] 1) Select the string to be processed A As the target of the query;

[0049] 2) Match the string according to the database A Similar string output collection;

[0050] 3) Use The rough similarity algorithm is used for pre-screening;

[0051] 4) Use exact similarity to calculate string similarity;

[0052] 5) Output the matching results.

[0053] Optionally, the similarity calculation formula for the rough similarity pre-screening is:

[0054]

[0055] Where, LCS is the length of the common subsequence;

[0056] String A is the string to be matched;

[0057] String B The string that matches.

[0058] Optionally, in step S3, the data cleaning includes:

[0059] 1) Conduct a preliminary check on the source data to understand issues such as data structure, missing values, and outliers;

[0060] 2) Based on the data inspection results, define cleaning rules for specific problems in the data;

[0061] 3) Perform cleaning operations;

[0062] 4) Verify the cleaned data by rechecking the data quality or comparing it with the source data to detect whether new errors or problems have been introduced;

[0063] 5) The cleaned data that meets the requirements is returned to the data source for similarity matching operations.

[0064] The technical effects and advantages of the present invention are as follows: by using the IP-guard terminal security management system to collect data on software installation and application process operation, and to build a software and process matching framework, the software can be intelligently and efficiently matched with the application processes running in the task manager, the real software operation data can be obtained, and software program rules can be generated, so as to help enterprises unify the software usage of terminal computers.

[0065] Secondly, a rough similarity algorithm is used to quickly evaluate the similarity between each string in the output set and the string to be detected. The rough similarity algorithm is based on easy-to-calculate features such as string length, number of common characters, and specific substrings. It can quickly filter out those strings that are obviously dissimilar. Through rough similarity pre-screening, the number of strings that need to be accurately calculated can be greatly reduced, thereby improving the efficiency of the entire query process.

[0066] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 is a flow chart of the similarity matching method provided by the present invention;

[0068] Figure 2 It is a matching process design architecture diagram provided by the present invention;

[0069] Figure 3 It is a similarity calculation flow chart provided by the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0071] like Figures 1 to 3 As shown, a software and process matching method based on similarity calculation provided by an embodiment of the present invention includes the following steps:

[0072] S1. Build a software and process matching framework to calculate the similarity between the software name and the process name, as well as the similarity between the software path and the process path;

[0073] S2. Data synchronization: Use the IP-guard terminal security management system to regularly collect software installation and application process operation information as source data for subsequent services;

[0074] S3. Clean the source data, format the data, and convert it into a string in a unified format;

[0075] S4, matching the similarity of the character strings through rough similarity pre-screening and precise similarity calculation;

[0076] S5. Generate software program rules based on the software and process matching framework.

[0077] It should be noted that according to experimental statistics, after software statistical analysis in a small and micro enterprise, the number of software-related records after deduplication is as high as 20,000, and the number of process records is as high as 70,000. The software and processes need to be associated, and the traditional solution requires 2w*7w=1.4 billion similarity calculations.

[0078] Experimental results show that this process takes nearly 8 hours. Therefore, the present invention builds a software and process matching framework for matching software and processes, and generates software program rules based on this, so as to detect newly installed software or newly started processes in real time and match them immediately;

[0079] Based on the matching results, detect possible abnormal behaviors and automatically execute predefined response measures, such as blocking malicious processes and sending alert notifications;

[0080] Furthermore, software program rules include software installation rules, process operation rules, similarity thresholds, rule implementation, maintenance and optimization;

[0081] Among them, software installation rules and process running rules both define a list of software that is allowed to be installed or run, as well as software that is prohibited from being installed and processes that are prohibited from being run. At the same time, software that is not on the list needs to be declared and approved. For processes that cannot be run, a monitoring and response mechanism is set up, including automatic termination, logging, sending alarms, etc.

[0082] Specifically, in step S1, the specific process of the software and process matching framework includes:

[0083] S11, obtaining software text information and converting the text information into a string format;

[0084] S12, segmenting and grouping the character string by segmentation marks to form class bucket identifiers;

[0085] S13, converting the class bucket identifier into a conventional format using a binary method;

[0086] S14, build classification index map ;

[0087] S15, remove risky common sequences and single-digit buckets to filter out oversized buckets;

[0088] S16, performing similarity character query and deduplication through bucket identifiers;

[0089] S17, optimization of precise similarity calculation based on rough similarity pre-screening.

[0090] Furthermore, by converting the software text information into a unified format, matching problems caused by inconsistent formats can be eliminated, and the character string can be converted into a bucket identifier and a conventional format through segmentation marks and binary method, which greatly improves the efficiency of subsequent processing. The introduction of bucket identifiers enables similar or related character strings to be quickly classified, which is convenient for subsequent query and matching;

[0091] Further, the classification index map The query process is further accelerated, allowing the system to quickly locate the target information. Since super-large buckets contain a large amount of irrelevant information, the amount of calculation will increase, which will burden the system and extend the calculation time. Therefore, eliminating risk common sequences and single digital buckets can effectively filter out the existence of super-large buckets, making the index more accurate and efficient, helping to reduce the waste of storage space and improve query efficiency.

[0092] Furthermore, similar character query and deduplication can be performed through bucket-like identifiers, which can quickly find similar or identical software and process information in a large amount of data, thereby improving the accuracy of matching;

[0093] Furthermore, the design of this process takes into account the scalability of the system. As the business develops and the software and process information increases, the system can adapt to new needs by adjusting the generation rules of class bucket identifiers and optimizing the index structure.

[0094] Furthermore, through the above-mentioned optimization measures, the process can significantly improve the processing speed and response capability of the system. Whether it is software installation approval, process monitoring or anomaly detection, it can be completed more quickly and accurately.

[0095] Specifically, the specific steps of step S12 are:

[0096] 1) receiving an input string containing multiple substrings, wherein the substrings are separated by split markers;

[0097] 2) Split the input string into multiple substrings according to the segmentation markers;

[0098] 3) Adoption n Alphabetical sorting, starting from the end of the substring, each consecutive n characters form a bucket identifier, which is not enough n characters directly form a bucket-like identifier;

[0099] 4) The class buckets formed by all groups are merged to generate a set, which is recorded as the class bucket identifier of the original string.

[0100] It should be noted that the split mark is “-”, and the long string is split into substrings by the split mark;

[0101] Furthermore, according to the experimental results, n Alphabetical classification, n The larger the value, the more buckets there are, the sparser the string distribution is, and the higher the matching efficiency is, but at the same time, the more memory is required;

[0102] Therefore, it is necessary to find a solution that suits the needs;

[0103] For example, experiments are conducted based on actual data to explore the best solution:

[0104] Classified by 1 to 5 letters, a total of about 70,000 strings were assigned to different buckets;

[0105] The following table shows the distribution of strings in different types of buckets:

[0106] Table 1. Distribution of one-letter classification (distribution of 7w test strings matching letter buckets)

[0107]

[0108] Table 2. Distribution of two-letter classification (partial distribution of 7w test strings matching letter buckets)

[0109]

[0110] Table 3 Three-letter classification distribution (partial distribution of 7w test strings matching letter buckets)

[0111]

[0112] Table 4 Four-letter classification distribution (partial distribution of 7w test strings matching letter buckets)

[0113]

[0114] Table 5 Five-letter classification distribution (partial distribution of 7w test strings matching letter buckets)

[0115]

[0116] Time consumption of different classification methods after processing 2w*7w test data sets:

[0117] Table 6 Time consumption for completing matching

[0118]

[0119] Therefore, the classification model of "classification by 5 letters" was selected;

[0120] Exemplary:

[0121] “disk-program-files-xeisi-guangfazhengjuanwebjiaoyianquankongjian”

[0122] First, the string is divided and grouped according to the "-" symbol. The above string will be divided into 5 groups: "disk", "program", "files", "xeisi", "guangfazhengjuanwebjiaoyianquankongjian". Then, each group is traversed from the head of the string. Every 5 letters are taken to form a bucket. For groups with less than 5 letters, a bucket is directly formed.

[0123] For example, "disk" is classified into the "disk" bucket;

[0124] "program" is divided into three buckets: "progr", "rogra", and "ogram";

[0125] By analogy, the class buckets formed by all groups are finally merged into the class bucket assigned to the original string, which means that the original string can be found from these class buckets;

[0126] Specifically, the specific steps of step S13 are:

[0127] 1) Determine the base number as base 27, which is used to represent a character set consisting of 0 and characters;

[0128] 2) receiving the class bucket identifier of the original string generated in step S12;

[0129] 3) Traverse the characters in the bucket identifier of the original string and determine the value in base 27 based on the position of the character in the character set sequence;

[0130] 4) From left to right, multiply the character values ​​by the corresponding power of 27, add the products, and output the unique integer identifier of the bucket-like identifier of the original string.

[0131] Furthermore, each bucket is actually identified by a unique number. The model uses the idea of ​​base 26 to map each bucket to a unique integer identifier. In order to prevent some characters from being mapped to repeated numbers, the number 0 is introduced, so base 27 is used.

[0132] Exemplary: represents " progr " will be mapped to 8868492.

[0133] Specifically, in step S14, the specific process of building the classification index includes: defining the data structure, loading the database, calculating the bucket identifier, updating the mapping relationship, and outputting the classification index. map ;

[0134] Defining a data structure includes defining a mapping structure map ;

[0135] Among them: the key is an unsigned 32-bit integer used to represent the bucket identifier;

[0136] The value is an unsigned 32-bit integer array, the elements in the integer array represent the serial numbers of the original strings, and are used to point to the corresponding items in the original string set;

[0137] The database is a matching library storing a large number of character strings;

[0138] The calculation of the class bucket identifier: traverse each string in the database, and calculate the class bucket identifier according to the characteristics of the string, which is recorded as map Keys;

[0139] The updating of the mapping relationship includes: detecting map Whether there is a duplicate key of the bucket identifier in the

[0140] 1) If it exists, add the sequence number of the string to the integer array of the corresponding bucket identifier;

[0141] 2) If it does not exist, then map Create a new key-value pair in , where the key is the bucket identifier of the string and the value is a single-element array containing the string sequence number;

[0142] The output classification index map : Traverse the strings of the entire database and generate a complete classification index map , where each class bucket identifier is mapped to an integer array, which contains the serial numbers of all strings assigned to the class bucket identifier.

[0143] Furthermore, when processing large databases, consider using more efficient data structures (such as balanced trees, skip lists, etc.) to store mapping relationships, or use parallel processing technology to speed up string processing and bucket identifier calculations to improve performance;

[0144] In practical applications, it is also necessary to consider error handling mechanisms, such as handling invalid input, handling memory overflow and other abnormal situations, and consider the scalability of the system so that new functions can be easily added or larger databases can be supported.

[0145] Specifically, step S15 includes: common sequence exclusion and single digital bucket exclusion;

[0146] The public sequence exclusion: before allocating the string to the corresponding class bucket according to the class bucket identifier, exclude the string that will lead to the formation of an oversized class bucket, and such string is recorded as a risk public sequence;

[0147] The single number bucket exclusion: excludes strings consisting of one or two characters.

[0148] Furthermore, a threshold needs to be set first to determine whether a class bucket becomes an "oversized class bucket" because it contains too many strings. The threshold is adjusted according to the actual application requirements and the size of the database.

[0149] The purpose of single-digit bucket exclusion is to exclude strings consisting of only one or two characters. Such strings do not contain enough information to form meaningful bucket identifiers, so excluding them can improve the efficiency and accuracy of classification indexing.

[0150] Furthermore, by excluding risky common sequences, the number of class buckets that need to be traversed during the search process can be reduced, thereby improving search efficiency. Moreover, due to the large number of strings in super-large class buckets, the discrimination of the classification index will be reduced and too much storage space will be required. Reducing the number of super-large class buckets can also optimize the utilization of storage resources.

[0151] Specifically, in step S16, the specific process of similarity character query and duplicate removal includes:

[0152] 1) Select the string to be processed as the query target;

[0153] 2) Calculate the class bucket identifier based on the string to be processed;

[0154] 3) Using the class bucket identifier obtained above, from the constructed classification index map Query and find all strings that can map to the bucket identifier of this type, and generate an output set;

[0155] 4) Remove duplicate strings from the output set;

[0156] 5) Output the deduplicated output set as the query result.

[0157] Furthermore, by using the classification index map Fast search can significantly improve the efficiency of similar string query. Computational bucket identification and deduplication are used to ensure the accuracy and uniqueness of query results. In addition, the process can be expanded to handle larger databases and more complex query requirements.

[0158] It should be noted that when building a classification index map When performing similarity queries, the algorithm and data source used need to be consistent.

[0159] Specifically, the specific process of step S17 is:

[0160] 1) Select the string to be processed A As the target of the query;

[0161] 2) Match the string according to the database A Similar string output collection;

[0162] 3) Use The rough similarity algorithm is used for pre-screening;

[0163] 4) Use exact similarity to calculate string similarity;

[0164] 5) Output the matching results.

[0165] Specifically, the similarity calculation formula of the rough similarity pre-screening is:

[0166]

[0167] Where, LCS is the length of the common subsequence;

[0168] String A is the string to be matched;

[0169] String B The string that matches.

[0170] Furthermore, the similarity calculation algorithm adopts similar_text The function is to find the common subsequence of two strings to measure the similarity. The performance is relatively low. In order to further improve the matching efficiency, the model first performs a rough similarity algorithm on this algorithm to remove the strings that do not meet the similarity requirements.

[0171] Furthermore, the process of finding LCS is optimized. LCS contains the order of the sequence. We can ignore the order first. As long as the same letters are included in the set, they are considered to constitute the LCS here and defined as CLCS. (For example, abc and bca ,That ,but ) CLCS is an unordered LCS. CLCS can only form LCS when it forms a certain order. Therefore, the CLCS value of the two strings must be greater than LCS.

[0172] Exemplary: " cversqver "and" averbrve ", in the original algorithm, LCS can only be " ver " sequence is equal to 3, but in the coarse similarity, CLCS has ver " also includes the following " ver "and" rve "The matched sequence, ;

[0173] When the similarity between two strings is greater than a certain value k When , then its LCS must be greater than a certain value m, and their CLCS must be greater than LCS;

[0174] That is to say, when the coarse similarity algorithm is less than k hour, ,So ;

[0175] but similar_text The similarity must be less than k .

[0176] Based on this idea, the time complexity of the rough similarity algorithm is approximately O(n+m). n and m The length of two strings.

[0177] After testing: first use a rough similarity algorithm to remove the strings that do not meet the similarity requirements, and then perform precise similarity calculations. The time can be shortened by dozens of seconds for every 1,000 matches.

[0178] Specifically, in step S3, the data cleaning includes:

[0179] 1) Conduct a preliminary check on the source data to understand issues such as data structure, missing values, and outliers;

[0180] 2) Based on the data inspection results, define cleaning rules for specific problems in the data;

[0181] 3) Perform cleaning operations;

[0182] 4) Verify the cleaned data by rechecking the data quality or comparing it with the source data to detect whether new errors or problems have been introduced;

[0183] 5) The cleaned data that meets the requirements is returned to the data source for similarity matching operations.

[0184] To summarize, the present invention constructs a software and process matching framework by using the data collected by the IP-guard terminal security management system on software installation and application process operation information. It can intelligently and efficiently match and associate the software with the application processes running in the task manager, obtain the real operation data of the software, and assist enterprises in unifying the software usage of terminal computers.

[0185] Secondly, a rough similarity algorithm is used to quickly evaluate the similarity between each string in the output set and the string to be detected. The rough similarity algorithm is based on easy-to-calculate features such as string length, number of common characters, and specific substrings. It can quickly filter out those strings that are obviously dissimilar. Through rough similarity pre-screening, the number of strings that need to be accurately calculated can be greatly reduced, thereby improving the efficiency of the entire query process.

[0186] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A software and process matching method based on similarity calculation, characterized in that: The steps include: S1. Construct a software and process matching framework, calculate the similarity between the software name and the process name, and the similarity between the software path and the process path, wherein the specific process of the software and process matching framework includes: S11, obtaining software text information and converting the text information into a string format; S12, split the string into groups through the split markers to form class bucket identifiers, specifically: 1) receiving an input string containing multiple substrings, wherein the substrings are separated by split markers; 2) Split the input string into multiple substrings according to the segmentation markers; 3) Adoption n Alphabetical sorting, starting from the end of the substring, each consecutive n characters form a bucket identifier, which is not enough n characters directly form a bucket-like identifier; 4) The class buckets formed by all groups are merged to generate a set, which is recorded as the class bucket identifier of the original string; S13, converting the class bucket identifier into a conventional format using a binary method; S14, build classification index map ; S15, remove risky common sequences and single-digit buckets to filter out oversized buckets; S16, performing similarity character query and deduplication through bucket identifiers; S17, precise similarity calculation optimization based on rough similarity pre-screening; S2. Data synchronization: Use the IP-guard terminal security management system to regularly collect software installation and application process operation information as source data for subsequent services; S3. Clean the source data, format the data, and convert it into a string in a unified format; S4, matching the similarity of the character strings through rough similarity pre-screening and precise similarity calculation; S5. Generate software program rules based on the software and process matching framework.

2. The software and process matching method based on similarity calculation according to claim 1, characterized in that: The specific steps of step S13 are: 1) Determine the base number as base 27, which is used to represent a character set consisting of 0 and characters; 2) receiving the class bucket identifier of the original string generated in step S12; 3) Traverse the characters in the bucket identifier of the original string and determine the value in base 27 based on the position of the character in the character set sequence; 4) From left to right, multiply the character values ​​by the corresponding power of 27, add the products, and output the unique integer identifier of the bucket-like identifier of the original string.

3. The software and process matching method based on similarity calculation according to claim 1, characterized in that: In step S14, the specific process of building the classification index includes: defining the data structure, loading the database, calculating the bucket identifier, updating the mapping relationship, and outputting the classification index. map ; Defining a data structure includes defining a mapping structure map ; Among them: the key is an unsigned 32-bit integer used to represent the bucket identifier; The value is an unsigned 32-bit integer array, the elements in the integer array represent the serial numbers of the original strings, and are used to point to the corresponding items in the original string set; The database is a matching library storing a large number of character strings; The calculation of the class bucket identifier: traverse each string in the database, and calculate the class bucket identifier according to the characteristics of the string, which is recorded as map Keys; The updating of the mapping relationship includes: detecting map Whether there is a duplicate key of the bucket identifier in the 1) If it exists, add the sequence number of the string to the integer array of the corresponding bucket identifier; 2) If it does not exist, then map Create a new key-value pair in , where the key is the bucket identifier of the string and the value is a single-element array containing the string sequence number; The output classification index map : Traverse the strings of the entire database and generate a complete classification index map , where each class bucket identifier is mapped to an integer array, which contains the serial numbers of all strings assigned to the class bucket identifier.

4. The software and process matching method based on similarity calculation according to claim 1, characterized in that: Step S15 includes: common sequence exclusion and single digital bucket exclusion; The public sequence exclusion: before allocating the string to the corresponding class bucket according to the class bucket identifier, exclude the string that will lead to the formation of an oversized class bucket, and such string is recorded as a risk public sequence; The single number bucket exclusion: excludes strings consisting of one or two characters.

5. The software and process matching method based on similarity calculation according to claim 1, characterized in that: In step S16, the specific process of similarity character query and deduplication includes: 1) Select the string to be processed as the query target; 2) Calculate the class bucket identifier based on the string to be processed; 3) Using the class bucket identifier obtained above, from the constructed classification index map Query and find all strings that can map to the bucket identifier of this type, and generate an output set; 4) Remove duplicate strings from the output set; 5) Output the deduplicated output set as the query result.

6. The software and process matching method based on similarity calculation according to claim 1, characterized in that: The specific process of step S17 is: 1) Select the string to be processed A As the target of the query; 2) Match the string according to the database A Similar string output collection; 3) Use The rough similarity algorithm is used for pre-screening; 4) Use exact similarity to calculate string similarity; 5) Output the matching results.

7. The method for matching software and processes based on similarity calculation according to claim 6, characterized in that: The similarity calculation formula of the rough similarity pre-screening is: Where, LCS is the length of the common subsequence; String A is the string to be matched; String B The string that matches.

8. The method for matching software and processes based on similarity calculation according to claim 1, characterized in that: In step S3, the data cleaning includes: 1) Conduct a preliminary check on the source data to understand the data structure, missing values, and outlier issues; 2) Based on the data inspection results, define cleaning rules for specific problems in the data; 3) Perform cleaning operations; 4) Verify the cleaned data by rechecking the data quality or comparing it with the source data to detect whether new errors or problems have been introduced; 5) The cleaned data that meets the requirements is returned to the data source for similarity matching operations.

Images