Method and device for screening list

Abstract

The present invention discloses a method and device for screening a list, and relates to the field of computer technology. A specific implementation of the method includes: determining the adjusted edit distance LD similarity and LD similarity score between the list data of the list to be screened and the screening data according to the screening data in the screening request; screening out a candidate list whose LD similarity exceeds a first threshold from the list to be screened, and determining the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data; if the maximum LCS similarity of the candidate list exceeds a second threshold, screening out a list that meets the requirements from the candidate list according to the LD similarity score and the LCS similarity score. This implementation combines the LD similarity algorithm and the LCS algorithm to more accurately calculate the similarity, and then can obtain effective features to achieve accurate matching of the list.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for screening a list. Background Art

[0002] List screening and matching is often used for anti-money laundering and anti-terrorist financing list monitoring. It is an important means for financial institutions to prevent their financial services from being exploited by criminals, and helps the business departments of institutions to determine whether customers are restricted by the regulatory authorities. However, the existing screening list method has a low screening accuracy and cannot meet business needs. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a method and device for filtering a list, which can solve the problem of low filtering accuracy in the existing method for filtering a list.

[0004] To achieve the above objective, according to one aspect of an embodiment of the present invention, a method for screening a list is provided.

[0005] The method for screening a list according to an embodiment of the present invention includes:

[0006] Determine, according to the screening data in the screening request, an adjusted edit distance LD similarity and an LD similarity score between the list data of the to-be-screened list and the screening data;

[0007] Screening out a candidate list whose LD similarity exceeds a first threshold from the to-be-screened list, and determining the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data;

[0008] If the maximum LCS similarity value of the candidate list exceeds a second threshold, a list meeting the requirements is screened out from the candidate list according to the LD similarity score and the LCS similarity score.

[0009] Optionally, before the step of determining, according to the screening data in the screening request, an adjusted edit distance LD similarity score between the list data of the to-be-screened list and the screening data, the method further comprises:

[0010] Based on the Solr engine, an index of the list to be filtered is created according to the list data of the list to be filtered.

[0011] Optionally, based on the Solr engine, creating an index of the to-be-filtered list according to the list data of the to-be-filtered list includes:

[0012] Structuring the list data of the list to be screened;

[0013] Removing punctuation marks, ASCII codes and stop words from the list data;

[0014] Based on the language mode corresponding to the list data, the list data is deduplicated;

[0015] Create an index of the list to be filtered based on the list data after deduplication processing.

[0016] Optionally, based on the language corresponding to the list data, performing deduplication processing on the list data includes:

[0017] If the language corresponding to the list data is English, converting the English words in the list data into lowercase English words;

[0018] If the language corresponding to the list data is Traditional Chinese, converting the Traditional Chinese words in the list data into Simplified Chinese words;

[0019] De-duplicate the converted inventory data.

[0020] Optionally, the screening data is a customer name; and determining, according to the screening data in the screening request, an adjusted edit distance LD similarity and an LD similarity score between the list data of the to-be-screened list and the screening data comprises:

[0021] If the language corresponding to the screening data is English or a language other than Chinese and English, determining the LD similarity between the list data of the to-be-screened list and the screening data;

[0022] If the LD similarity exceeds the first threshold, an LD similarity score is determined according to the LD similarity.

[0023] Optionally, the screening data is a customer name; and determining, according to the screening data in the screening request, an adjusted edit distance LD similarity and an LD similarity score between the list data of the to-be-screened list and the screening data comprises:

[0024] If the language corresponding to the screening data is Traditional Chinese, converting the Traditional Chinese words in the screening data into Simplified Chinese words;

[0025] If the language corresponding to the screening data is Simplified Chinese or after converting the Traditional Chinese words in the screening data into Simplified Chinese words, separate the surname and the given name in the converted screening data, and perform a full match between the surname and the given name of the screening data and the list data respectively;

[0026] The adjusted edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data are determined according to the full matching result.

[0027] Optionally, determining the adjusted edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data according to the full matching result includes:

[0028] If the first full match results of the surname and the first name of the screening data and the list data do not exceed the first hit rate, converting the screening data into pinyin words, and performing full match between the converted pinyin words and the list data;

[0029] If the second full match result between the converted pinyin word and the list data exceeds a second hit rate, determining an LD similarity score according to the second full match result;

[0030] If the full match result between the converted pinyin word and the list data does not exceed the second hit rate, and there is an English name in the screening data, then the LD similarity is calculated;

[0031] If the LD similarity exceeds the first threshold, an LD similarity score is determined according to the LD similarity.

[0032] Optionally, determining the adjusted edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data according to the full matching result further includes:

[0033] If the surname and the given name of the screening data respectively exceed a first hit rate with the first full match result of the list data, an LD similarity score is determined according to the first full match result.

[0034] Optionally, determining the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data includes:

[0035] Merge the list data of the candidate list into a string according to the last name and the first name, the first name and the last name, and the alias of the list;

[0036] Determining, based on the character string, the LCS similarity between the list data of the candidate list and the screening data;

[0037] If the LCS similarity exceeds a second threshold, determining an LCS similarity score according to the LCS similarity.

[0038] Optionally, screening out a list that meets the requirements from the candidate list according to the LD similarity score and the LCS similarity score includes:

[0039] Determining a fusion decision score according to the LD similarity score and the LCS similarity score;

[0040] The lists whose fusion decision scores exceed a third threshold are screened out from the candidate lists to form a list hit set.

[0041] Optionally, determining a fusion decision score according to the LD similarity score and the LCS similarity score includes:

[0042] The difference obtained by subtracting the LD similarity score from the LCS similarity score is determined as the fusion decision score; wherein the LD similarity score is the deduction in the LD similarity calculation process, and the LCS similarity score is the score in the LCS similarity calculation process.

[0043] To achieve the above objective, according to another aspect of an embodiment of the present invention, a device for screening a list is provided.

[0044] The device for screening a list according to an embodiment of the present invention includes:

[0045] A first determination module, configured to determine, according to the screening data in the screening request, an adjusted edit distance LD similarity and an LD similarity score between the list data of the to-be-screened list and the screening data;

[0046] A second determination module is used to select a candidate list whose LD similarity exceeds a first threshold from the to-be-screened list, and determine the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data;

[0047] A screening module is used to screen out a list that meets the requirements from the candidate list according to the LD similarity score and the LCS similarity score if the maximum LCS similarity value of the candidate list exceeds a second threshold.

[0048] To achieve the above objective, according to another aspect of an embodiment of the present invention, a server is provided.

[0049] The server of the embodiment of the present invention includes:

[0050] one or more processors;

[0051] a storage device for storing one or more programs,

[0052] When the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0053] To achieve the above objective, according to another aspect of an embodiment of the present invention, a computer readable medium is provided.

[0054] The computer-readable medium of the embodiment of the present invention stores a computer program, and when the program is executed by a processor, the method described above is implemented.

[0055] One embodiment of the above invention has the following advantages or beneficial effects:

[0056] The method for screening a list in an embodiment of the present invention combines the LD similarity algorithm and the LCS algorithm to more accurately calculate the similarity, thereby obtaining effective features to achieve accurate matching of the list.

[0057] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific implementation examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0059] Figure 1 is a flowchart of a method for screening a list according to a first embodiment of the present invention;

[0060] Figure 2 is a flowchart of a method for screening a list according to a second embodiment of the present invention;

[0061] Figure 3 is a schematic diagram of the number of customers hit by full screening according to an embodiment of the present invention;

[0062] Figure 4 is a schematic diagram of the time consumption of full screening according to an embodiment of the present invention;

[0063] Figure 5 is a flowchart of a method for screening a list according to a third embodiment of the present invention;

[0064] Figure 6 It is a schematic diagram of the Solr index quantity of the Reuters list according to an embodiment of the present invention;

[0065] Figure 7 It is a schematic diagram of the time consumption of creating a Solr index according to an embodiment of the present invention;

[0066] Figure 8 is a flowchart of a method for screening a list according to a fourth embodiment of the present invention;

[0067] Fig. 9 is a flowchart of a method for screening a list according to a fifth embodiment of the present invention;

[0068] Fig.10 is a schematic diagram of a module of a device for screening a list according to an embodiment of the present invention;

[0069] Fig.11is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0070] Fig.12 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0072] Generally, in order to filter the list, it is necessary to determine the similarity between the filter data and the list data. There are many ways to calculate the similarity, and two of them are exemplified below.

[0073] Method 1: Levenshtein Distance (LD) similarity algorithm.

[0074] The LD similarity algorithm was proposed by Russian mathematician Vladimir Levenshtein in 1965. The method is a quantitative measurement of the degree of difference between two strings (such as English words), and the measurement method is to analyze how many times it takes to convert a string A into another string B. Edit distance can be used in natural language processing, such as spelling check, which one (or which ones) is more likely to be the edit distance between a misspelled word and other correct words.

[0075] If the LD similarity algorithm is applied alone to list screening and matching, only the similarity of words can be considered, but the order of multiple word combinations cannot be considered. As a result, there may be a situation where the word combination order is inconsistent but the word analysis results are consistent. For example, the two names zhong rong qing and zhong qing rong are not actually the same person for domestic customers.

[0076] Method 2: Longest Common Subsequences (LCS) similarity algorithm.

[0077] The LCS similarity algorithm is to find the longest identical character sequence obtained by deleting zero or more characters from two given strings without changing the order of the remaining characters. The length of the longest common substring can be calculated by recursion, or by dynamic programming.

[0078] 1. Given two sequences X =<x1,x2,...,xm> ,Y<y1,y2,...,yn> , find the longest common subsequence of X and Y.

[0079] 2. Introduce a two-dimensional array c[][], use c[i][j] to record the length of the LCS of X[i] and Y[j], and b[i][j] to record the value of the subproblem through which c[i][j] is obtained, so as to determine the direction of the search.

[0080] 3. Using bottom-up recursive calculation, before calculating c[i,j], c[i-1][j-1], c[i-1][j] and c[i][j-1] have all been calculated. At this time, we can calculate c[i][j] based on whether X[i]=Y[j] or X[i]!=Y[j], and we can get the following formula:

[0081]

[0082] 4. Calculate the similarity according to the following formula:

[0083]

[0084] The time complexity of this algorithm is O(m+n) and the space complexity is O(m.log(n)).

[0085] When the similarity is calculated using the second method, if the number of strings to be compared differs greatly, the calculated similarity value may also differ greatly.

[0086] In order to achieve matching accuracy and take into account the algorithm measurement time complexity, an embodiment of the present invention proposes a method for screening list names by combining the LD similarity algorithm and the LCS similarity algorithm. At the same time, the method can also comprehensively consider the influence of nationality and language.

[0087] Based on the above analysis, an embodiment of the present invention provides a method for screening a list, which is a method for implementing list screening based on decision fusion of LD and LCS, that is, the process of screening a list is completed by combining the LD similarity algorithm and the LCS algorithm. And this method can be applied to the business field of screening out hit customers based on matching customer factors with list customer data. Figure 1 FIG. 1 is a flow chart of a method for screening a list according to an embodiment of the present invention. Figure 1 As shown, the method for filtering a list may specifically include the following steps S101 to S103.

[0088] Step S101: according to the screening data in the screening request, determining the adjusted Levenshtein Distance (LD) similarity and LD similarity score between the list data of the to-be-screened list and the screening data.

[0089] Before step S101, a screening request is received, and the screening request at least includes screening data. Then, the screening data is obtained from the received screening request, and the list data of the list to be screened is obtained to determine the LD similarity and the LD similarity score. The screening data is the screening parameters input by the user according to actual needs, and the screening data can be customer name and / or address data, etc.

[0090] In order to facilitate the acquisition of the list to be screened, before step S102, based on the Solr engine, an index of the list to be screened is created according to the list data of the list to be screened and the language mode corresponding to the list data. In order to solve the problem of inaccurate matching due to different nationality language data in list screening and matching, an index of the list to be screened can be created according to the list data of the list to be screened and the language mode corresponding to the list data. This method can obtain effective features, can effectively perform similarity calculation, and finally achieve accurate matching of the list.

[0091] The process of creating an index is as follows: first, the list data of the list to be screened is structured. Then, punctuation marks, ASCII codes (American Standard Code for Information Interchange) and stop words in the list data are removed. Then, based on the language corresponding to the list data, the list data is deduplicated. Finally, an index of the list to be screened is created based on the deduplicated list data.

[0092] Furthermore, the language corresponding to the list data may be English, Chinese or other languages. In order to facilitate deduplication processing of the list data, the list data may be converted into a language before deduplication processing. Specifically, if the language corresponding to the list data is English, the English words in the list data are converted into lowercase English words; if the language corresponding to the list data is Traditional Chinese, the Traditional Chinese words in the list data are converted into Simplified Chinese words.

[0093] In step S101, the LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data can be determined based on the language corresponding to the screening data. The language corresponding to the screening data can be Chinese, English or other languages, and the LD similarity score can be understood as the deduction value in the LD similarity calculation process. Taking the screening data as a customer name as an example, the Solr engine can be used to calculate the LD similarity score between each word (including Chinese characters) of the customer name to be matched and the list data.

[0094] Furthermore, the process of determining the LD similarity score can be roughly divided into the following situations:

[0095] Scenario 1: The language corresponding to the screening data is English or other languages ​​​​except Chinese and English.

[0096] If the language corresponding to the screening data is English or other languages ​​except Chinese and English, the adjusted edit distance LD similarity between the list data of the to-be-screened list and the screening data is determined; if the LD similarity exceeds the first threshold, the LD similarity score is determined according to the LD similarity.

[0097] Scenario 2: The language corresponding to the screening data is Chinese, which includes Traditional Chinese and Simplified Chinese.

[0098] If the language corresponding to the screening data is Traditional Chinese, the Traditional Chinese words in the screening data are converted into Simplified Chinese words; if the language corresponding to the screening data is Simplified Chinese or after the Traditional Chinese words in the screening data are converted into Simplified Chinese words, the surname and the first name in the converted screening data are separated, and the surname and the first name of the screening data are fully matched with the list data respectively; according to the full matching result, the adjusted edit distance LD similarity and the LD similarity score between the list data of the list to be screened and the screening data are determined.

[0099] Furthermore, if the first full match result of the surname and first name of the screening data with the list data does not exceed the first hit rate, the screening data is converted into pinyin words, and the converted pinyin words are fully matched with the list data; if the second full match result of the converted pinyin words with the list data exceeds the second hit rate, the LD similarity score is determined according to the second full match result; if the full match result of the converted pinyin words with the list data does not exceed the second hit rate, and there is an English name in the screening data, the LD similarity is calculated; if the LD similarity exceeds the first threshold, the LD similarity score is determined according to the LD similarity. If the first full match result of the surname and first name of the screening data with the list data exceeds the first hit rate, the LD similarity score is determined according to the first full match result.

[0100] Step S102: selecting a candidate list whose LD similarity exceeds a first threshold from the list to be screened, and determining the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data.

[0101] In step S102, the LCS similarity score can be understood as the score value in the LCS similarity calculation process. According to the first matching result, a candidate list whose LD similarity exceeds a first threshold is screened from the to-be-screened list to form a candidate list set.

[0102] Step S103: If the maximum LCS similarity value of the candidate list exceeds the second threshold, a list that meets the requirements is screened out from the candidate list according to the LD similarity score and the LCS similarity score.

[0103] In step S103, lists that meet the requirements are screened out from the candidate lists and a list hit set is formed as a list screening and matching result. Specifically, the fusion decision score can be first determined based on the LD similarity score and the LCS similarity score. Then, lists whose fusion decision scores exceed the third threshold are screened out from the candidate lists to form a list hit set. Among them, the fusion decision score is the difference between the LCS similarity score and the LD similarity score, the LD similarity score is the deduction in the LD similarity calculation process, and the LCS similarity score is the score in the LCS similarity calculation process.

[0104] Taking the screening data being the customer name as an example, the LD similarity algorithm is used to calculate the similarity between each element in the customer name and the list data to obtain the initial decision matching score, and the lists whose similarity fails to meet the first threshold are filtered out to obtain the candidate list; then, based on the dimension of word combination, the LCS similarity algorithm is used to calculate the similarity between the customer name and the list data of the candidate list to obtain the final decision matching score and the list screening matching result.

[0105] The method for screening a list in an embodiment of the present invention combines the LD similarity algorithm and the LCS algorithm to more accurately calculate the similarity, thereby obtaining effective features to achieve accurate matching of the list.

[0106] Based on the above embodiment, the embodiment of the present invention provides another method for filtering a list. Figure 2 FIG. 1 is a flow chart of a method for screening a list according to an embodiment of the present invention. Figure 2 As shown, the method for filtering a list may specifically include the following steps S201 to S210.

[0107] Step S201: Based on the Solr engine, an index of the list to be filtered is created according to the list data of the list to be filtered.

[0108] Get your list faster and more accurately by creating an index in your filter list.

[0109] Step S202: according to the screening data in the screening request, determine the LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data.

[0110] Prior to step S202, a screening request is received, wherein the screening request includes at least screening data. The screening data is obtained from the screening request, and the list data of the list to be screened is obtained to determine the LD similarity and the LD similarity score. The list data includes: information such as customer ID type, ID number, nationality, gender, date of birth, etc. It should be noted that the LD similarity score is calculated based on the LD similarity algorithm, and the LD similarity score is a deduction in the LD similarity calculation process.

[0111] Step S203: Determine whether the LD similarity exceeds the first threshold; if so, the list is deemed to be hit and step S204 is executed. Otherwise, the process ends and the list screening is completed.

[0112] Step S204: Filter out candidate lists whose LD similarity exceeds a first threshold from the to-be-filtered lists to form a first candidate list set.

[0113] It can be understood that the processed screening data and list data are obtained through step S204, that is, the candidate list with LD similarity exceeding the first threshold is obtained to form a first candidate list set.

[0114] Step S205: Determine the LCS similarity and LCS similarity score between the list data of the candidate list and the screening data.

[0115] In step S205, the LCS similarity score is the score in the LCS similarity calculation process.

[0116] Step S206: Determine whether the maximum LCS similarity of the candidate list exceeds the second threshold; if so, execute step S207. Otherwise, end the process and complete the list screening.

[0117] Step S207: If the maximum LCS similarity value of the candidate list exceeds the second threshold, a fusion decision score is calculated according to the LD similarity score and the LCS similarity score.

[0118] Step S208: Return the result set of the fusion decision scores of the candidate list.

[0119] Step S209: determine whether the fusion decision score in the result set exceeds the third threshold; if so, execute step S210. Otherwise, end the process and complete the list screening.

[0120] It can be understood that the lists whose fusion decision scores exceed the third threshold are screened out from the candidate lists to form a list hit set.

[0121] Step S210: Filter out the lists whose fusion decision scores exceed a third threshold from the result set to form a list hit set.

[0122] The implementation principle of the above implementation is: the first matching score is calculated by the LD (Levenshtein Distance) similarity algorithm, and then the second matching score is calculated by the LCS (Longest Common Subsequences) similarity algorithm, and finally the similarity matching score calculated by the fusion LD and LCS algorithms is obtained. The specific algorithm implementation process is as follows:

[0123]

[0124]

[0125] See also Figure 3 and Figure 4, the 4C8G machine is adopted to test the screening and matching of 2.5 million screening data and Reuters list data. 868 customers are suspected to be hit, and all results are accurate. Since this solution mainly uses the customer name as the list screening result, and the customer list data contains customer document types, document numbers, nationalities, genders, birth dates, etc., after performing other element matching calculations based on this list screening, the customers who are hit can be more accurately located.

[0126] The method for screening the list in the embodiment of the present invention combines the LD similarity algorithm and the LCS algorithm to calculate the similarity more accurately, and then effective features can be obtained to achieve accurate matching of the list. At the same time, this method can also perform accurate screening based on the language mode corresponding to the list data, and can solve the nationality language data problem.

[0127] Generally, Solr is a high-performance full-text search server developed in Java and based on Lucene. At the same time, it is extended to provide a richer query language than Lucene, and at the same time realizes configurability, extensibility and optimized query performance, and provides a perfect function management interface. It is an excellent full-text search engine. Based on Solr, the scalability of in-site search is better and the workload of programmers can be reduced. The implementation methods for creating an index and searching an index using Solr are as follows:

[0128] 1. Create an index: The client (which can be a browser or a Java program) sends a JSON, CSV, or XML document describing the Field and its content to the Solr server using the POST method. The Solr server adds, deletes, or updates the index according to the JSON, CSV, or XML document.

[0129] 2. Search the index: The client (which can be a browser or a Java program) sends a request to the Solr server using the GET method, and then parses the query results in JSON, CSV, XML and other formats returned by the Solr server to organize the page layout. Solr does not provide the function of building the page UI, but provides a management interface. Through the management interface, the configuration and running status of Solr can be queried.

[0130] The disadvantages of the above implementation are as follows: directly creating a Solr index for the list data mainly does not analyze the nature of the list customer data, resulting in a large increase in the index volume of the list data and the appearance of duplicate list data indexes. For example, for "Wang Yuyu", index data such as "wang xu xu", "wang yu yu", "wang xu yu", "wang yu xu" are created, which may lead to mis-matching of homophonic characters in the subsequent list matching process.

[0131] In order to solve the above problems, based on the above embodiments, an embodiment of the present invention provides another method for filtering a list, which is used to describe the process of creating an index for list data, for example, using the Solr engine to create a list index for the screening data (for example, customer name). Figure 5 FIG. 1 is a flow chart of a method for screening a list according to an embodiment of the present invention. Figure 5 As shown, the method for filtering the list may specifically include the following steps S501 to S507.

[0132] Step S501: Structuring the list data of the to-be-screened list.

[0133] In step S501, unstructured Reuters all list agency list data may be converted into structured data.

[0134] Step S502: The list is stored in the warehouse.

[0135] Step S503: remove punctuation marks, ASCII codes and stop words in the list data.

[0136] In step S503, punctuation marks, ASCII codes, and stop words are removed from the customer names and aliases of the list entities.

[0137] Step S504: Determine the language corresponding to the list data; the language corresponding to the list data may be Chinese, English or other languages. If the language corresponding to the list data is English, convert the English words in the list data into lowercase English words.

[0138] It is understandable that for English words, all words are converted to lowercase words (Lowercase); for Chinese characters, traditional Chinese needs to be converted to simplified Chinese. The converted customer names are deduplicated to remove the duplication of the native language and English aliases, and then the Solr index is built. The above implementation method can effectively reduce the amount of Solr index data of the Reuters list.

[0139] Step S505: If the language corresponding to the list data is Traditional Chinese, convert the Traditional Chinese words in the list data into Simplified Chinese words.

[0140] Step S506: De-duplicate the converted list data.

[0141] Step S507: Create a Solr index of the list to be screened based on the list data after deduplication processing.

[0142] See also Figure 6 and Figure 7Through the above implementation, the amount of list data indexing can be effectively reduced by nearly one-fifth of the original amount (at the same time, if the 152 list agencies are executed according to business requirements, the amount of list data indexing is reduced to 1.4%). At the same time, the index creation time is reduced by nearly two-thirds (the operating environment is 4C8G), (at the same time, if the 152 list agencies are executed according to business requirements, the execution time is reduced to 15%). In addition, there is no need to convert Chinese customer names to pinyin and polyphones when building list indexes, because during screening and matching, the customers to be screened will be converted to pinyin. Chinese names are not converted to pinyin according to polyphones. Through the above implementation, the amount of Solr index data of Reuters list customer names can be greatly reduced. In addition, the traditional Chinese characters of the customer names in the list data are converted to simplified Chinese, and the uppercase English characters are uniformly converted to lowercase. After removing customers with the same names, the Solr index of the list customer names can be greatly reduced.

[0143] In order to facilitate understanding of the implementation process of the above implementation, the specific implementation code of the above implementation is used as an example. The specific implementation code of the above implementation is as follows:

[0144]

[0145]

[0146]

[0147] Based on the above embodiment, the embodiment of the present invention provides another method for screening a list, which is used to describe the process of determining the LD similarity score. It should be noted that the screening data may be a customer name or address data, etc. Next, taking the screening data as an example of a customer name, the Solr engine may calculate the LD similarity between each word (including Chinese characters) of the customer name to be matched and the list data.

[0148] Since the language mode corresponding to the screening data is different, the process of determining LD similarity and LD similarity score is also different. After determining the language mode corresponding to the screening data, it can be divided into the following three situations according to the language mode:

[0149] Scenario 1: The language corresponding to the screening data is English or other languages ​​​​except Chinese and English.

[0150] See also Figure 8 If the language corresponding to the screening data is English or other languages ​​except Chinese and English, the edit distance LD similarity between the list data of the to-be-screened list and the screening data is determined; if the LD similarity exceeds a first threshold, the LD similarity score is determined according to the LD similarity.

[0151] Scenario 2: The language corresponding to the screening data is Chinese, which includes Traditional Chinese and Simplified Chinese.

[0152] See also Figure 8 The process of determining LD similarity and LD similarity score may specifically include the following steps S801 to S811.

[0153] Step S801: determine whether the language corresponding to the screening data is Traditional Chinese; if so, execute step S802. Otherwise, execute step S803.

[0154] Step S802: If the language corresponding to the screening data is Traditional Chinese, convert the Traditional Chinese words in the screening data into Simplified Chinese words.

[0155] Step S803: If the language corresponding to the screening data is Simplified Chinese or after converting the Traditional Chinese words in the screening data into Simplified Chinese words, separate the surname and the given name in the converted screening data, and perform a full match between the surname and the given name of the screening data and the list data respectively.

[0156] After step S803, the adjusted edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data are determined according to the full matching result.

[0157] Step S804: Determine whether the first full match result of the surname and the first name of the screening data respectively exceeds the first hit rate with the list data. If so, the first full match result is deemed to be a hit, and step S811 is executed; otherwise, the first full match result is deemed to be a miss, and step S805 is executed.

[0158] Step S805: If the first complete matching result of the surname and the first name of the screening data with the list data respectively does not exceed the first hit rate, the screening data is converted into a pinyin word.

[0159] Step S806: Fully match the converted pinyin words with the list data.

[0160] Step S807: Determine whether the second full match result between the converted pinyin word and the list data exceeds the second hit rate; if so, determine that the second full match result is a hit, and execute step S811; otherwise, determine that the second full match result is a miss, and execute step S808.

[0161] It is understandable that the converted customer name is separated into the last name and the first name, and a full match is performed with the list data. If a match is found, it is used as a match score.

[0162] Step S808: Check whether there is an English name in the screening data; if so, execute step S809; otherwise, end the process and the LD similarity calculation is completed.

[0163] Step S809: If the full match result between the converted pinyin word and the list data does not exceed the second hit rate, and there is an English name in the screening data, then the LD similarity is calculated.

[0164] Taking into account that the list data may not be in the native language, the Chinese in the screening data is converted into pinyin, polyphone conversion is performed, and then the full match is calculated with the list data according to the converted pinyin name. If it hits, it is used as a feature matching score.

[0165] Step S810: determine whether the LD similarity exceeds the first threshold; if so, execute step S811; otherwise, end the process and the LD similarity calculation is completed.

[0166] Taking the screening data as customer names as an example, the LD similarity between the English name and the list data is calculated, and the score of the fuzzy matching word is deducted as a feature matching score; if there is no hit, it means that the customer does not match the list data, and the subsequent LCS secondary calculation is not performed.

[0167] Step S811: Determine the LD similarity score based on the matching result.

[0168] If the LD similarity exceeds the first threshold, the LD similarity score is determined according to the LD similarity. If the surname and first name of the screening data respectively exceed the first hit rate with the first full match result of the list data, the LD similarity score is determined according to the first full match result. If the converted pinyin word exceeds the second hit rate with the second full match result of the list data, the LD similarity score is determined according to the first full match result or the second full match result.

[0169] In summary, the embodiments of the present invention have the following advantages:

[0170] 1) This method can be used to calculate the similarity of each word or Chinese character of the customer to be screened, and obtain the customer's overall similarity score.

[0171] 2) It can effectively avoid the situation where the list data lacks Chinese names and only has English names, resulting in the omission of Chinese-to-pinyin conversion, and effectively takes Chinese into account.

[0172] 3) Since Chinese is converted into pinyin by the system, it is impossible for pinyin similarity to occur. Therefore, the embodiment of the present invention uses Chinese or Chinese to pinyin to perform full matching to calculate the first matching score.

[0173] 4) Consider the situation of polyphones when converting Chinese to Pinyin to avoid missing polyphones.

[0174] Based on the above embodiment, the embodiment of the present invention provides another method for screening a list, which is used to describe the process of determining LCS similarity and LCS similarity score. Next, take the screening data as customer name as an example. Fig. 9 FIG. 1 is a flow chart of a method for screening a list according to an embodiment of the present invention. Fig. 9 As shown, the method for filtering the list may specifically include the following steps S901 to S906.

[0175] Step S901: Obtain a matching result, and obtain the screening customer name (or searchName) and the customer name corresponding to the list (or entityName) after LD similarity calculation from the matching result.

[0176] It is to be understood that one or more list data sets of suspected hits obtained by LD calculation are combined as a secondary screening list data source.

[0177] Step S902: taking entityName processed by LD similarity calculation as a data set.

[0178] It can be understood that the entityName in the candidate list is taken as a data set.

[0179] Step S903: Determine the LCS similarity between entityName and searchName in the data set by using an LCS similarity algorithm.

[0180] In step S903, considering that the customer's surname may be placed first or last, the customers of the list data source are screened again, corresponding to three situations: surname and first name, first name and surname, and alias of the Reuters list. After deduplication, two names may remain, which are merged and regarded as a string. Then, the LCS similarity between entityName and searchName in the data set is determined based on the string.

[0181] Step S904: Obtain secondary matching results.

[0182] It can be understood that the LCS similarity between entityName and searchName in the data set is calculated to obtain the list data with the highest matching score as the secondary hit result. If it exceeds the customer name hit threshold set by the business, the customer is a suspected hit customer.

[0183] Step S905: Determine whether the LCS similarity exceeds the second threshold; if so, determine that the secondary matching result is a hit, and execute step S906. Otherwise, determine that the secondary matching result is a miss, end the process, and complete the list screening.

[0184] Step S905: Return the hit result set, which is a set consisting of the list finally obtained by screening.

[0185] It is understandable that the list data of the first match hit obtained by Levenshtein Distance measurement is used as the data source of the second list match, and the similarity between the customer to be screened and the list data of the first match is calculated by Longest Common Subsequences. The specific algorithm implementation process is as follows:

[0186]

[0187]

[0188] In summary, the embodiments of the present invention have the following advantages:

[0189] 1) Since the customer names recorded in Reuters data are based on the customs of China and Western countries, the order of surnames and given names may be inconsistent. This screening and matching method takes this into account to avoid omissions.

[0190] 2) After calculating the secondary matching results through the LCS similarity algorithm, it can effectively filter out the cases where the similarity is calculated for each word and the results are completely consistent, but it does not take into account the situation where the order of the names is reversed: for example, "Zhang Sanfeng" and "Zhang Fengsan" are not the guests of the same customer, so accurate screening of customer names can be achieved.

[0191] 3) Effectively screen all matching data of Reuters list data to form a similarity matching set.

[0192] Fig.10 is a schematic diagram of a module of a device for screening a list according to an embodiment of the present invention, see Fig.10 The device 1000 for screening a list includes the following modules:

[0193] A first determination module 1001 is used to determine, according to the screening data in the screening request, the adjusted edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data;

[0194] The second determination module 1002 is used to select a candidate list whose LD similarity exceeds a first threshold from the to-be-screened list, and determine the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data;

[0195] The screening module 1003 is configured to screen out a list that meets the requirements from the candidate list according to the LD similarity score and the LCS similarity score if the maximum LCS similarity value of the candidate list exceeds a second threshold.

[0196] Optionally, the device 1000 for screening a list further includes:

[0197] The index module is used to create an index of the list to be filtered according to the list data of the list to be filtered based on the Solr engine.

[0198] Optionally, the index module is further used to:

[0199] Structuring the list data of the list to be screened;

[0200] Removing punctuation marks, ASCII codes and stop words from the list data;

[0201] Based on the language mode corresponding to the list data, the list data is deduplicated;

[0202] Create an index of the list to be filtered based on the list data after deduplication processing.

[0203] Optionally, the index module is further used to:

[0204] If the language corresponding to the list data is English, converting the English words in the list data into lowercase English words;

[0205] If the language corresponding to the list data is Traditional Chinese, converting the Traditional Chinese words in the list data into Simplified Chinese words;

[0206] De-duplicate the converted inventory data.

[0207] Optionally, the first determining module 1001 is further configured to:

[0208] If the language corresponding to the screening data is English or a language other than Chinese and English, determining the LD similarity between the list data of the to-be-screened list and the screening data;

[0209] If the LD similarity exceeds the first threshold, an LD similarity score is determined according to the LD similarity.

[0210] Optionally, the first determining module 1001 is further configured to:

[0211] If the language corresponding to the screening data is Traditional Chinese, converting the Traditional Chinese words in the screening data into Simplified Chinese words;

[0212] If the language corresponding to the screening data is Simplified Chinese or after converting the Traditional Chinese words in the screening data into Simplified Chinese words, separate the surname and the given name in the converted screening data, and perform a full match between the surname and the given name of the screening data and the list data respectively;

[0213] The adjusted edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data are determined according to the full matching result.

[0214] Optionally, the first determining module 1001 is further configured to:

[0215] If the first full match results of the surname and the first name of the screening data and the list data do not exceed the first hit rate, converting the screening data into pinyin words, and performing full match between the converted pinyin words and the list data;

[0216] If the second full match result between the converted pinyin word and the list data exceeds a second hit rate, determining an LD similarity score according to the second full match result;

[0217] If the full match result between the converted pinyin word and the list data does not exceed the second hit rate, and there is an English name in the screening data, then the LD similarity is calculated;

[0218] If the LD similarity exceeds the first threshold, an LD similarity score is determined according to the LD similarity.

[0219] Optionally, the first determining module 1001 is further configured to:

[0220] If the surname and the given name of the screening data respectively exceed a first hit rate with the first full match result of the list data, an LD similarity score is determined according to the first full match result.

[0221] Optionally, the second determining module 1002 is further configured to:

[0222] Merge the list data of the candidate list into a string according to the last name and the first name, the first name and the last name, and the alias of the list;

[0223] Determining, based on the character string, the LCS similarity between the list data of the candidate list and the screening data;

[0224] If the LCS similarity exceeds a second threshold, determining an LCS similarity score according to the LCS similarity.

[0225] Optionally, the screening module 1003 is further used to:

[0226] Determining a fusion decision score according to the LD similarity score and the LCS similarity score;

[0227] The lists whose fusion decision scores exceed a third threshold are screened out from the candidate lists to form a list hit set.

[0228] Optionally, the screening module 1003 is further used to:

[0229] The difference obtained by subtracting the LD similarity score from the LCS similarity score is determined as the fusion decision score; wherein the LD similarity score is the deduction in the LD similarity calculation process, and the LCS similarity score is the score in the LCS similarity calculation process.

[0230] The device for screening a list in the embodiment of the present invention combines the LD similarity algorithm and the LCS algorithm to more accurately calculate the similarity, thereby obtaining effective features to achieve accurate matching of the list.

[0231] Fig.11 An exemplary system architecture 1100 is shown to which the method for filtering a list or the apparatus for filtering a list according to an embodiment of the present invention can be applied.

[0232] like Fig.11 As shown, system architecture 1100 may include terminal devices 1101, 1102, 1103, a network 1104, and a server 1105. Network 1104 is used to provide a medium for communication links between terminal devices 1101, 1102, 1103 and server 1105. Network 1104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0233] Users can use terminal devices 1101, 1102, 1103 to interact with server 1105 through network 1104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 1101, 1102, 1103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0234] The terminal devices 1101, 1102, 1103 may be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, etc. The server 1105 may be a server that provides various services.

[0235] It should be noted that the method for filtering a list provided in the embodiment of the present invention is generally executed by the server 1105 , and accordingly, the device for filtering a list is generally disposed in the server 1105 .

[0236] It should be understood that Fig.11 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0237] Reference below Fig.12 , which shows a schematic diagram of the structure of a computer system 1200 of a terminal device suitable for implementing an embodiment of the present invention. Fig.12 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0238] like Fig.12 As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the system 1200 are also stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0239] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed, so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0240] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1209, and / or installed from the removable medium 1211. When the computer program is executed by the central processing unit (CPU) 1201, the above-mentioned functions defined in the system of the present invention are executed.

[0241] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0242] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0243] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: determining the adjusted edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data according to the screening data in the screening request; screening out a candidate list whose LD similarity exceeds a first threshold from the to-be-screened list, and determining the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data; if the maximum LCS similarity of the candidate list exceeds a second threshold, screening out a list that meets the requirements from the candidate list according to the LD similarity score and the LCS similarity score.

[0244] The method for screening a list in an embodiment of the present invention combines the LD similarity algorithm and the LCS algorithm to more accurately calculate the similarity, thereby obtaining effective features to achieve accurate matching of the list.

[0245] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for screening a list, characterized in that: include: Based on the Solr engine, an index of the list to be screened is created according to the list data of the list to be screened and the language mode corresponding to the list data; Determine, according to the screening data in the screening request, the edit distance LD similarity and the LD similarity score between the list data of the to-be-screened list and the screening data; Screening out a candidate list whose LD similarity exceeds a first threshold from the to-be-screened list, and determining the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data; If the maximum LCS similarity value of the candidate list exceeds a second threshold, screening out a list that meets the requirements from the candidate list according to the LD similarity score and the LCS similarity score; The screening data is the customer name; Determining the edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data according to the screening data in the screening request includes: if the language mode corresponding to the screening data is Traditional Chinese, converting the Traditional Chinese words in the screening data into Simplified Chinese words; if the language mode corresponding to the screening data is Simplified Chinese or after converting the Traditional Chinese words in the screening data into Simplified Chinese words, separating the surname and the given name in the converted screening data, and fully matching the surname and the given name of the screening data with the list data respectively; determining the edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data according to the full matching result; Determining the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data includes: merging the list data of the candidate list into a character string according to the surname and the first name, the first name and the surname, and the alias of the list; determining the LCS similarity between the list data of the candidate list and the screening data based on the character string; if the LCS similarity exceeds a second threshold, determining the LCS similarity score based on the LCS similarity.

2. The method according to claim 1, characterized in that: Based on the Solr engine, creating an index of the to-be-filtered list according to the list data of the to-be-filtered list includes: Structuring the list data of the list to be screened; Removing punctuation marks, ASCII codes and stop words from the list data; Based on the language mode corresponding to the list data, the list data is deduplicated; Create an index of the list to be filtered based on the list data after deduplication processing.

3. The method according to claim 2, characterized in that Based on the language mode corresponding to the list data, deduplication processing of the list data includes: If the language corresponding to the list data is English, converting the English words in the list data into lowercase English words; If the language corresponding to the list data is Traditional Chinese, converting the Traditional Chinese words in the list data into Simplified Chinese words; De-duplicate the converted inventory data.

4. The method according to claim 1, characterized in that The screening data is a customer name; according to the screening data in the screening request, determining the edit distance LD similarity and LD similarity score of the list data of the to-be-screened list and the screening data includes: If the language corresponding to the screening data is English or a language other than Chinese and English, determining the LD similarity between the list data of the to-be-screened list and the screening data; If the LD similarity exceeds the first threshold, an LD similarity score is determined according to the LD similarity.

5. The method according to claim 1, characterized in that Determining the edit distance LD similarity and LD similarity score of the list data to be screened and the screening data according to the full matching result includes: If the first full match results of the surname and the first name of the screening data and the list data do not exceed the first hit rate, converting the screening data into pinyin words, and performing full match between the converted pinyin words and the list data; If the second full match result between the converted pinyin word and the list data exceeds a second hit rate, determining an LD similarity score according to the second full match result; If the full match result between the converted pinyin word and the list data does not exceed the second hit rate, and there is an English name in the screening data, then the LD similarity is calculated; If the LD similarity exceeds the first threshold, an LD similarity score is determined according to the LD similarity.

6. The method according to claim 5, characterized in that Determining the edit distance LD similarity and LD similarity score of the list data to be screened and the screening data according to the full matching result also includes: If the surname and the given name of the screening data respectively exceed a first hit rate with the first full match result of the list data, an LD similarity score is determined according to the first full match result.

7. The method according to claim 1, characterized in that According to the LD similarity score and the LCS similarity score, the lists meeting the requirements are screened out from the candidate lists, including: Determining a fusion decision score according to the LD similarity score and the LCS similarity score; The lists whose fusion decision scores exceed a third threshold are screened out from the candidate lists to form a list hit set.

8. The method according to claim 7, characterized in that Determining the fusion decision score according to the LD similarity score and the LCS similarity score includes: The difference obtained by subtracting the LD similarity score from the LCS similarity score is determined as the fusion decision score; wherein the LD similarity score is the deduction in the LD similarity calculation process, and the LCS similarity score is the score in the LCS similarity calculation process.

9. A device for screening a list, characterized in that: include: An indexing module, used for creating an index of the list to be screened based on the Solr engine according to the list data of the list to be screened and the language mode corresponding to the list data; A first determination module, configured to determine, according to the screening data in the screening request, the edit distance LD similarity and the LD similarity score between the list data of the to-be-screened list and the screening data; A second determination module is used to select a candidate list whose LD similarity exceeds a first threshold from the to-be-screened list, and determine the longest common subseries LCS similarity and LCS similarity score between the list data of the candidate list and the screening data; A screening module, configured to screen out a list meeting the requirements from the candidate list according to the LD similarity score and the LCS similarity score if the maximum LCS similarity of the candidate list exceeds a second threshold; The first determination module is further used for: if the language mode corresponding to the screening data is Traditional Chinese, converting the Traditional Chinese words in the screening data into Simplified Chinese words; if the language mode corresponding to the screening data is Simplified Chinese or after converting the Traditional Chinese words in the screening data into Simplified Chinese words, separating the surname and the given name in the converted screening data, and fully matching the surname and the given name of the screening data with the list data respectively; Determine the edit distance LD similarity and LD similarity score between the list data of the to-be-screened list and the screening data according to the full matching result; The second determination module is further used to: merge the inventory data of the candidate list into a character string according to the surname and the first name, the first name and the surname, and the alias of the list; determine the LCS similarity between the inventory data of the candidate list and the screening data based on the character string; If the LCS similarity exceeds a second threshold, determining an LCS similarity score according to the LCS similarity.

10. A server, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images