Method, computer program, and computer system (string similarity determination)

Abstract

To solve the problem in which record matching involves computation of similarities between strings but there is a continuous need to improve distance measurement.SOLUTION: A sequence of edit operations to be performed on a first string in order to obtain a second string may be determined. Each edit operation is an edit operation of a first type or a second type. The first type operation comprises a character insertion operation or character removal operation. The second type operation comprises a character maintenance operation. The first type edit operation is associated with an operation score indicative of a cost for applying the edit operation. The first type edit operation is associated with a switching score indicative of whether it is immediately followed by a second type edit operation. The switching scores and / or operation scores associated with the sequence of edit operations are combined in order to obtain a combined score that is indicative of a similarity level between the first and second strings.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to the field of digital computer systems, and more particularly to a method for determining the similarity between two strings. [Background technology]

[0002] Record linkage requires linking elements of a source dataset to related data items in a target dataset. To do so, record matching can be performed between records of the datasets. Record matching involves calculating the similarity between strings. However, there is a continuous need to improve distance measures. Summary of the Invention [Problem to be solved by the invention]

[0003] Record matching involves calculating the similarity between strings, however there is a continuing need to improve distance measures. [Means for solving the problem]

[0004] Various embodiments provide a method, a computer system and a computer program product for determining the similarity between two strings as described by the subject matter of the independent claims. Advantageous embodiments are described in the dependent claims. The embodiments of the present disclosure can be freely combined with each other if they are not mutually exclusive.

[0005] In one aspect, the present disclosure relates to a method for determining a similarity between a string s1 having N1 characters, where N1 > 0, and a string s2 having N2 characters, where N2 > 0, the method comprising: a. providing a distance algorithm, said distance algorithm comprising: i. receiving a first string and a second string; ii. determining a sequence of one or more edit operations to perform on characters of the first string to obtain the second string, the edit operations being of a first type or a second type, the first type edit operation comprising a character insertion operation or a character deletion operation, and the second type edit operation comprising a character preserving operation, the first type edit operation being associated with an operation score indicating the cost of applying the edit operation, and the first type edit operation being associated with a switching score indicating whether it is immediately followed in the sequence by a second type edit operation; iii. combining the switching scores or the operation scores, or both, associated with the sequence of editing operations, resulting in a combined score indicative of the level of similarity between the first string and the second string; and a providing step configured to: b. inputting the first n1 characters of the string s1 as the first string and the first n2 characters of the string s2 as the second string, where 0≦n1≦N1 and 0≦n2≦N2, into the distance algorithm to obtain the combined score; c. determining the distance between the string s1 and the string s2 using the obtained combined score; Equipped with.

[0006] In another aspect, the present disclosure relates to a computer program product comprising a computer-readable storage medium having computer-readable program code embodied therein, said computer-readable program code being configured to implement all steps of the method according to the preceding embodiments.

[0007] In another aspect, the present disclosure relates to a computer system for determining a similarity between a string s1 having N1 characters, where N1 >= 0, and a string s2 having N2 characters, where N2 >= 0. The computer system comprises: a. providing a distance algorithm, said distance algorithm comprising: i. receiving a first string and a second string; ii. determining a sequence of one or more edit operations to be performed on characters of the first string to obtain the second string, the edit operations being of a first type or a second type, the first type edit operation comprising a character insertion operation or a character deletion operation, and the second type edit operation comprising a character preserving operation, the first type edit operation being associated with an operation score indicative of the cost of applying the edit operation, and the first type edit operation being associated with a switching score if it is immediately followed in the sequence by an edit operation of a second type; iii. combining the switching scores or the operation scores, or both, associated with the sequence of editing operations, resulting in a combined score indicative of the level of similarity between the first string and the second string; and a providing step configured to: b. inputting the first n1 characters of the string s1 as the first string and the first n2 characters of the string s2 as the second string, where 0≦n1≦N1 and 0≦n2≦N2, into the distance algorithm to obtain the combined score; c. determining the distance between the string s1 and the string s2 using the obtained combined score; The device is configured to: [Brief explanation of the drawings]

[0008] In the following, embodiments of the present disclosure will be described in more detail, by way of example only, with reference to the drawings, in which:

[0009] [Figure 1] FIG. 1 is a block diagram of a computer system according to an example of the present subject matter.

[0010] [Figure 2] 1 is a flowchart of a method for determining similarity between two strings according to an example of the present subject matter.

[0011] [Figure 3] 1 is a flowchart of a method for determining similarity between two strings according to an example of the present subject matter.

[0012] [Figure 4] 1 is a flowchart of a method for determining similarity between two strings according to an example of the present subject matter.

[0013] [Figure 5A] 1 is a flowchart of a method for determining similarity between two strings according to an example of the present subject matter.

[0014] [Figure 5B] FIG. 10 illustrates the evolution of the contents of an edit distance matrix, according to an example of the present subject matter.

[0015] [Figure 6A] 1 is a flowchart of a method for determining similarity between two strings according to an example of the present subject matter.

[0016] [Figure 6B] FIG. 10 illustrates the evolution of the contents of an edit distance matrix, according to an example of the present subject matter.

[0017] [Figure 7] 1 is pseudocode for determining the similarity between two strings according to an example of the present subject matter.

[0018] [Figure 8] FIG. 1 illustrates a computerized system suitable for implementing one or more method steps as included in the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0019] The description of various embodiments of the present disclosure is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been selected to best explain the principles of the embodiments, practical applications of, or technical improvements to, the technology found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0020] The similarity between two strings can be measured by the distance between the two strings. However, accurately determining the distance that works for a wide variety of compared strings can be a difficult task. To that end, the present subject matter can provide a string similarity function that provides a number that indicates a distance measure specific to the algorithm. The number may be a combined score, as described herein. The term "string," as used herein, may refer to a sequence of zero or more characters, where a character may be a number, a letter, or any special character. Accurately determining the similarity of two strings can have implications for several fields of application in which similarity analysis may be used. For example, the string distance algorithm can be used in areas including fraud detection, fingerprint analysis, plagiarism detection, ontology merging, DNA analysis, RNA analysis, image analysis, evidence-based machine learning, database data deduplication, data mining, incremental search, data integration, malware detection, semantic knowledge integration, and natural language processing (where automatic spelling correction can determine candidate corrections for a misspelled word by selecting multiple words from a dictionary that have a low distance to the word).

[0021] The subject matter may be able to calculate edit distance by counting the number of edit operations required to transform one string into another. Edit operations are scored according to their type. Additionally, the subject matter may score the operations based on the order in which they are applied. This may provide an accurate comparison and overcome the following problems of existing edit distance measures: For example, the Levenshtein similarity between "Textile" and "Textile Company" is the same as the Levenshtein similarity between "Textile" and "TCeoxm tpialney" simply because the "Company" inserted together has the same weight as a character inserted in a random place. The same may occur in the case of "Wachter AG" versus "Wechsler AG" and "Wachter Bau AG." Levenshtein similarity may penalize word substitutions very heavily. For example, "Chaussures Michel" and "Michel Chassures" have a greater distance than "Chaussures Michel" and "Chic Chaussures", simply because many insertion and deletion operations are required due to substitutions.

[0022] The present subject matter may provide a distance algorithm. The distance algorithm may receive as input a first string and a second string and determine a sequence of one or more edit operations to perform on characters of the first string to obtain the second string. The distance algorithm may apply a score assignment rule to score the sequence of one or more edit operations. The result of the scoring is a combined score. The combined score may be the edit distance between the first string and the second string. The edit operations may be of a first type or a second type. The first type of edit operation includes a character insertion operation (referred to as "I") or a character deletion operation (referred to as "D"). The second type of edit operation includes a character preserving operation (referred to as "M"); for example, a character preserving operation may not involve editing and is therefore named an "edit operation" simply for purposes of naming. The distance algorithm may apply a score assignment rule to each operation in the sequence of operations. The score assignment rules assign to a first type of editing operation an operation score indicating the cost of applying the first type of editing operation. The operation score may be equal to 1, for example. The score assignment rules may assign to a second type of editing operation an operation score equal to 0 because the second type of editing operation may not involve editing. In another example, the operation score assigned to a given editing operation may be weighted (e.g., multiplied by) a predefined weight associated with the character to which the editing operation is applied. Additionally, if a first type of editing operation is immediately followed in a sequence by a second type of editing operation, the score assignment rules assign to the first type of editing operation a switching score p (or penalty), where p=SC, where SC is the value of the penalty. The switching score may be equal to 1 or any number, for example, and preferably lower than the sum of the costs for inserting and deleting characters. A switching score may be termed a first type switching score if a first type editing operation is immediately followed in a sequence by a second type editing operation.An edit operation of a first type immediately followed by an edit operation of a second type in a sequence may be referred to as a first switching type. The first type switching score provides a penalty for changing the type of operation from the first type to the second type. Alternatively, or in addition, if an edit operation of a second type immediately precedes an edit operation of a first type in a sequence, the score assignment rule may assign a switching score p to the edit operation of the first type. The switching score may be named a second type switching score in this case. An edit operation of a first type immediately precedes an edit operation of a second type in a sequence may be referred to as a second switching type. The second type switching score may optionally be used. For this purpose, a different implementation of the application of the switching score may be used. In one embodiment, the switching score is p=w. sw × SC, where w sw may be set to a value that enables or disables the switching score. The first type of switching score and the second type of switching score are each assigned a weight w sw1 and w sw2 , and the weight of the first type of switching score may be set to 1 (w sw1 = 1), while the second type of switching score may be set to 1 if it is considered or 0 if it is not, e.g., w sw2 = 1 or 0. The first type of switching score is p=w sw1 × SC, and the second type of switching score is p=w sw2×SC. Therefore, application of the score assignment rules to a sequence of one or more edit operations may result in a switching score or a manipulation score, or both, for the sequence of edit operations. A distance algorithm may combine these switching scores or manipulation scores, or both, associated with a sequence of edit operations to obtain a combined score or edit distance that indicates the level of similarity between the first string and the second string. The combination may be performed, for example, by summing the scores. Distance algorithms may be advantageous because they may provide an accurate edit distance between the compared strings. Using a combination of switching scores and manipulation scores, score assignment rules according to the present subject matter may enable an accurate edit distance regardless of how the sequence of edit operations is determined. The sequence of edit operations may be obtained using different techniques. For example, different candidate sequences of edit operations may be determined, and the candidate sequence that provides the lowest combined score or lowest edit distance may be selected. For example, the string “shop” may be obtained from “soup” by four delete operations that delete the letters “s,” “o,” “u,” and “p,” and four insert operations that insert “s,” “h,” “o,” and “p,” resulting in a sequence of eight operations DDDDIIII that gives a distance of 8 because the two can be transformed using eight edit operations, and the operation score is 1. However, the minimum set of operations that can perform this transformation may have a smaller distance; for example, “shop” may be obtained from “soup” by keeping “s,” inserting “h,” keeping “o,” deleting “u,” and keeping “p,” which is a sequence of five operations MIMDM with a lower distance of 6, e.g., a switching score of 4 and an operation score of 2, where the switching score is 1 and the operation score is 1. In another example, the sequence of edit operations may be obtained using known techniques such as the Levenshtein edit distance technique.

[0023] In the following, the first two strings to be compared may be referred to as string s1 having N1 characters, where N1≧0, and string s2 having N2 characters, where N2≧0. The distance algorithm may be configured to calculate a combined score (distance) for two input strings of the algorithm, referred to as a first string and a second string, where the first string has n1 characters and the second string has n2 characters. Depending on how the distance algorithm is performed, n1 and n2 may or may not be equal to N1 and N2, respectively (0≦n1≦N1 and 0≦n2≦N2). The combined score determined for n1=0 and n2>0 may indicate the cost of converting a space character into n2 characters, which may correspond to n2 insertion operations. The combined score determined for n1>0 and n2=0 may indicate the cost of converting n1 characters into a space character, which may correspond to n1 deletion operations.

[0024] In one first implementation example, the distance algorithm may be configured to calculate the similarity between a first string and a second string at a time without relying on previously calculated scores; for example, it may calculate a combined score for two strings independently of any other previously calculated scores. In this case, to determine the similarity between string s1 and string s2, the distance algorithm may be invoked with one input pair (first string, second string) at a time. For example, two strings s1 and s2 may be compared by the distance algorithm (at a time) by inputting the two strings s1 and s2 into the distance algorithm (i.e., the first string is s1 and the second string is s2) to obtain a combined score.

[0025] According to one embodiment, when n1=N1 and n2=N2, the obtained combined score indicates the distance between string s1 and string s2. The obtained combined score may be the edit distance between string s1 and string s2. For example, if string s1 includes a sequence of letters "sp" and a second string s2 includes a sequence of letters "shop", the sequence of edit operations to be performed on the letters "sp" to obtain the sequence of letters "shop" is a keep operation (because "s" is kept), two consecutive insert operations to insert the letter "ho", and one keep operation to keep the letter "p". The sequence of operations in this case may be MIIM. Note that other sequences of operations (I, M, and D) may be determined using other techniques. The distance algorithm may apply a score assignment rule to the sequence MIIM. The first keep operation may receive an operation score OS1=0 because it is a second type operation. The second operation "I" may receive an operation score OS2=1 because it is a first type operation. The second operation "I" may receive a second type switching score w sw2 The third operation "I" may further receive an operation score OS3=1 because there was a switching from a first operation, which is an operation of the second type "M", to a second operation of the first type "I". The third operation "I" may receive an operation score OS3=1 because it is an operation of the first type. Furthermore, the third operation may receive a switching score w of the first type. sw1 ×SC because the operation immediately following "I" is an "M" operation, which is a second type of operation. The last operation may receive an operation score OS4=0 because it is a second type of operation. The combined score may be obtained, for example, by summing the scores or using other combining techniques, e.g., the combined score may be equal to the sum of: sw2 ×SC+OS1+OS2+OS3+w sw1 ×SC+OS4, where w sw1 =1 and w sw2 =1.

[0026] According to one embodiment, the method further comprises providing a character weight for each character of the string s1 and the string s2, wherein associating the operation score to the first type of editing operation includes weighting the operation score with the character weight of the character involved in the first type of editing operation, and associating the first type of switching score to the first type of editing operation includes weighting the switching score p with the weight w c This involves weighting with weight w c is a function of the combined score of a subsequence of operations having the first type of edit operation as the last operation and the number of first type of edit operations in the subsequence. For example, the function may be the ratio of the combined score to the number of first type of edit operations, but is not limited thereto, as other functions may be used. In this embodiment, the second type of switching score may not be applied. Following the above example of s1="sp" and s2="shop", the combined score may be equal to the following formula: w sw2 ×SC+w1×OS1+w2×OS2+w3×OS3+w c ×w sw1 × SC+w4 × OS4, where weights w1 through w4 are the weights associated with the four characters of string s2, respectively, and w c is the combination score assigned to a subsequence of operations, which is the switching score w divided by the number of operations in that subsequence. sw1 Contains and precedes an edit operation assigned with xSC, sw1 =1 and w sw2 = 0. c is sometimes referred to as the average character weight. In practice, the penalty may depend on the weight of the character to be inserted (or deleted), so the penalty does not have to be pre-assigned if there is a deviation from diagonal processing (identical strings) to off-diagonal progression, i.e., w sw2 = 0. Alternatively, a penalty may be assigned when going from an off-diagonal progression back to a diagonal progression (i.e., w sw1=1). Diagonal and non-diagonal progression refer to iterative implementations using matrices. Deviations from diagonal to non-diagonal progression refer to the second switching type, and deviations from non-diagonal to diagonal progression refer to the first switching type.

[0027] According to one embodiment, the distance algorithm is further configured to associate a switching score with each first type edit operation in the sequence of edit operations if it is immediately preceded in the sequence by a second type edit operation. Following the above example of s1="sp" and s2="shop", the second operation "I" would be associated with a switching score w according to this embodiment. sw2 ×SC because it is immediately preceded by the first operation of the second type. In this case, the combined score is sw2 ×SC+OS1+OS2+OS3+w sw1 ×SC+OS4, where w sw1 =1 and w sw2 = 1. This switching score w sw2 ×SC can be advantageously used for scores calculated without letter weights.

[0028] According to one embodiment, N1 >= 1 or N2 >= 1, or both. The similarity level between string s1 and string s2 may be modeled by the following function:

number

number

[0029] In a second implementation example that may reuse previous calculations, the distance algorithm may use previously calculated combined scores when it is repeatedly invoked to process characters in strings s1 and s2. In this case, the distance algorithm may be invoked multiple times to determine the distance between strings s1 and s2, with the result of the final iteration being a combined score indicating the distance / similarity between the initial string s1 and string s2. In this example, the distance algorithm may determine a sequence of operations for the current pair of first and second strings based on a previously calculated sequence in a previous iteration. This may save resources that would otherwise be required for unnecessary repeated operation determination. In this case, the distance algorithm was first invoked with a first string having n1=0 characters in string s1 and a second string having n2=0 characters in string s2 (which correspond to two space characters). The distance algorithm may determine that the space characters have a distance of 0. Using values ​​of n1 and n2 starting from 0 may be advantageous as it may be possible to set an initial value of the combined score for pairs (first string, second string) having values ​​n1=0 or n2=0. Thus, according to one embodiment, the method further comprises providing or determining or setting an initial value of the combined score for pairs of first and second strings having n1 and n2 characters, respectively, where n1=0 and n2=0,1,...N2, or n2=0 and n1=0,1,...N1. This is shown in Figures 5A-5B, where initial values ​​may be provided for the first row and first column of a matrix.

[0030] Thus, according to one embodiment of the second implementation, the first n1 characters of string s1 and the first n2 characters of string s2 may be repeatedly input into the distance algorithm, with new values ​​of n1 and n2 being selected at each iteration according to nested loops up to n1=N1 characters and n2=N2 characters, respectively, where n1 represents the outer loop and n2 represents the inner loop. That is, for a given iteration, n1 may be fixed at a value between 0 and N1, n2 may be incremented from 0 to N2, then n1 may be fixed at the next value between 0 and N1, and so on. This may enable this second implementation to be implemented in a matrix as illustrated in FIGS. 5A-5B. At each iteration, the distance algorithm: a first combined score has previously been determined (e.g., or set / initialized) for a first string having n1-1 (and n1-1≧0) characters and a second string having n2 characters using a first sequence of editing operations; and / or a second combination score was previously determined (e.g., or set / initialized) for a first string having n characters and a second string having n (and n > 0) characters using a second sequence of editing operations; and / or Using a third sequence of editing operations, a third combination score was previously determined (e.g., or set / initialized) for a first string having n1-1 (and n1-1≧0) characters and a second string having n2-1 (and n2-1≧0) characters, and whether the last characters of the first string and the second string are the same. You may judge / check the following.

[0031] If all checked combination scores have been previously calculated and / or set, the distance algorithm may select the lowest of the calculated / set combination scores. That is, the selected lowest score may be the first, second, or third combination score. The selected lowest score may be the combination score determined for the selected sequence of edit operations, where the selected sequence of edit operations is the first, second, or third sequence of edit operations whose combination score was selected as the lowest. The distance algorithm may determine an additional operation to be performed in addition to the selected sequence of edit operations to obtain the second string of the current iteration from the first string of the current iteration. Thus, the sequence of edit operations to transform the first string of the current iteration into the second string includes the selected sequence of edit operations plus the determined additional operation. The distance algorithm may apply a score assignment rule taking the additional operation into account. This may result in an operation score and, if the operation preceding the additional operation in the sequence is an operation of a different type, a switching score. The selected lowest score may be combined with the resulting operation score and the switching score induced by the additional operation. The combined score is the edit distance between the first and second strings of this iteration.

[0032] If at least one of the checked combined scores has not been previously calculated or set, the distance algorithm may calculate the at least one combined score before selecting the lowest score and determining the sequence of operations and edit distance as described above. However, this may only be done for pairs of first and second strings with n1=0 or n2=0 (e.g., n1=0 and n2=0 represent a space character), and no corresponding initial or set value is provided in advance.

[0033] According to one embodiment, the method further comprises obtaining a combined score calculated for each pair of a first string having n1=N1 characters and a second string having n2 characters varying from 0 to N2, and a combined score calculated for each pair of a first string having n1 characters and a second string having n2=N2 characters varying from 0 to N1. The method further comprises receiving a request to compare two strings s3 and s4, where s3=s1+m1 and s4=s2+m2, where m1 and m2 are strings of 0 or more characters. A second example implementation of the method may be applied to s3 and s4 using the scores secured by repeatedly inputting the first n1 characters of string s3 and the first n2 characters of string s4 into the distance algorithm by changing n1 and n2 to new values ​​in each iteration (as described above), where the values ​​of n1 iterate over the range 0..N1 while the values ​​of n2 iterate over the range N2+1..N4 (the right quadrant of the matrix), then the values ​​of n1 iterate over N1+1..N3 while the values ​​of n2 iterate over the range 0..N4 (the lower two quadrants of the matrix).

[0034] According to one embodiment, determining the sequence of operations and associating the operation scores with the switching scores is performed in parallel, character by character.

[0035] According to one embodiment, a distance algorithm may be used to perform record matching between two records. Record matching comprises comparing pairs of attribute values ​​of two records using a distance algorithm to yield individual similarity levels for the attributes, and combining the individual similarity levels to determine whether the two records are matched records. The distance algorithm may be performed according to any of the example implementations described above.

[0036] A data record or record is a collection of related data items, such as a particular user's name, date of birth, and class. A record represents an entity, which refers to a user, object, or concept about which information is stored in the record. The terms "data record" and "record" are used interchangeably. Data records may be stored, for example, in a graph database as entities with relationships, and each record may be assigned to a node or vertex of the graph, with properties that are attribute values ​​such as name, date of birth, etc. A data record, in another example, may be a record in a relational database.

[0037] Matching records involves comparing attribute values ​​of the records. For example, if a record includes a set of attributes a1 through an, a comparison between two records is performed by comparing n pairs of values ​​of the attributes a1 through an. Therefore, a comparison between two or more records may result in n individual similarity levels indicating the level of similarity of the values ​​of the respective attributes a1 through an. The similarity level (or matching level) between the compared records may be a combination (e.g., an average) of the individual similarity levels. The matching level of two records indicates the degree of similarity of the attribute values ​​of the two records. Each of the similarity levels, individual similarity levels, and word-level similarity may be provided as a normalized value (e.g., 0 to 1) or any other format that enables matching of records. If the matching level is higher than a predefined similarity threshold, this indicates that the two records match. In that case, a deduplication system constructed in the present disclosure may merge the records because they represent the same entity. Merging records is an operation that can be implemented in different ways. For example, merging two records may involve creating a golden record as a replacement for similar-looking records that were found to be duplicates of each other. This is known as data fusion, or physical collapse at the record or attribute level of survivorship. If the level of matching is less than or equal to a predefined similarity threshold, this indicates that the two records are not a match and therefore may be retained as separate data records.

[0038] According to one embodiment, N1 >= 1 or N2 >= 1, or both. The similarity level between string s1 and string s2 may be modeled by the following function:

number

[0039] FIG. 1 illustrates an exemplary computer system 100. The computer system 100 may be configured to perform, for example, master data management or data warehousing, or both; for example, the computer system 100 may enable a deduplication system. The computer system 100 includes a data integration system 101 and one or more client systems or data sources 105. The client systems 105 may include computer systems (e.g., computer systems such as those described with reference to FIG. 8). The client systems 105 may communicate with the data integration system 101 via a network connection, including, for example, a wireless local area network (WLAN) connection, a wide area network (WAN) connection, a local area network (LAN) connection, the Internet, or a combination thereof. The data integration system 101 may control access (e.g., read and write access) to a central repository 103.

[0040] Data records stored in the central repository 103 may have values ​​for a set of attributes 109A-P, such as a company name attribute. Although this example is described with respect to a small number of attributes, more or fewer attributes may be used. A data set 107 used in accordance with the present subject matter may include at least a portion of the records in the central repository 103.

[0041] Data records stored in the central repository 103 may be received from client systems 105 and processed by the data integration system 101 before being stored in the central repository 103. The received records may or may not have the same set of attributes 109A-P. For example, a data record received by the data integration system 101 from a client system 105 may not have all values ​​of the set of attributes 109A-P; e.g., the data record may have values ​​for a subset of attributes in the set of attributes 109A-P and no values ​​for the remaining attributes. In other words, the records provided by the client systems 105 may have different completeness, which is the ratio of the number of attributes of the data record that contain data values ​​to the total number of attributes in the set of attributes 109A-P. Additionally, the received records from the client systems 105 may have a structure that differs from the structure of the stored records in the central repository 103. For example, the client system 105 may be configured to provide records in XML format, JSON format, or any other format that allows for the association of attributes with corresponding attribute values.

[0042] In another example, the data integration system 101 may import data records from the central repository 103 from client systems 105 using one or more extract, transform, and load (ETL) batch processes, or via HyperText Transport Protocol ("HTTP") communications, or via other types of data exchange.

[0043] The data integration system 101 may be configured to process received records, such as to identify duplicate records, using a distance algorithm 120 that implements at least a portion of the method. For example, the data integration system 101 may process data records received from the client system 105 using the distance algorithm 120 to find matching records within the dataset 107.

[0044] 2 is a flowchart of a method for determining similarity between two strings according to one example of the present subject matter. For illustrative purposes, the method illustrated in FIG. 2 may be implemented in the system shown in FIG. 1, but is not limited to this implementation. A distance algorithm 120 may be configured to perform the method of FIG. 2.

[0045] In step 201, a first string and a second string may be received, the first string including a sequence of n1 characters and the second string including a sequence of n2 characters.

[0046] In step 203, a sequence of one or more edit operations to be performed on characters of the first string to obtain a second string may be determined. Determining the sequence of edit operations may be performed in different ways. For example, in the case of a second implementation of the distance algorithm, the sequence of edit operations may be determined as described with reference to FIG. 4. The sequence of edit operations may be determined using previously calculated scores, for example, as described in steps 403 to 409 of FIG. 4. This may be particularly advantageous when the distance algorithm is invoked iteratively to calculate distances. In the case of a first implementation of the distance algorithm, the sequence of edit operations may be determined as described with reference to FIG. 3. In another example, known techniques such as the Levenshtein edit distance technique may be used to determine the sequence of edit operations.

[0047] In stage 205, each operation in a sequence of one or more edit operations may be assigned an operation score and, possibly, an additional switching score depending on the type of edit operation. For example, if an operation is a first type of edit operation, it may be associated with an operation score indicating the cost of applying the edit operation. In addition, if an operation is a first type of edit operation that is immediately followed in the sequence by a second type of edit operation, it may further be associated with a switching score. In the case of an iterative implementation of the distance algorithm, in stage 205, previously assigned scores may be used rather than assigning new scores to previously processed edit operations. For example, if the sequence of edit operations for the current iteration is "DII," the combined scores obtained in a previous iteration for the sequence "DI" may be used to calculate the combined score for "DII" in this iteration.

[0048] In stage 207, the switching scores or operation scores, or both, associated with the sequences of edit operations may be combined. This may result in a combined score that indicates the level of similarity between the first string and the second string. The combined score may be the edit distance between the first string and the second string.

[0049] 3 is a flowchart of a method for determining similarity between two strings according to one example of the present subject matter. For illustrative purposes, the method illustrated in FIG. 3 may be implemented in the system shown in FIG. 1, but is not limited to this implementation.

[0050] In step 301, two strings s1 and s2 may be inputs to the distance algorithm. The string s1 of N1 characters may be the first string of the distance algorithm, and the string s2 of N2 characters may be the second string of the distance algorithm.

[0051] In step 303, the distance algorithm may determine a sequence of editing operations to obtain the second string s2 from the first string s1. This may be performed, for example, by sequentially processing the first string s1 character by character. Processing each current character of the first string is performed by determining a current subsequence of characters of the first string s1, including the first x characters of the first string s1 ending with the current character; for example, if the first string s1 is "abcdef" and the current character is "c", the determined current subsequence is "abc". Furthermore, an operation to be performed on the current subsequence of characters of the first string s1 to obtain a corresponding (same length) subsequence of the second string s2 may be determined. For example, the second string "shop" may be obtained from the first string "soup" by first processing the first subsequence, which is the character "s" of "soup". This would indicate that "s" will be kept because it is the same as the corresponding subsequence "s" in the second string "shop". The subsequent subsequence of the letters "so" associated with the letter "o" may be processed to determine an operation to obtain the corresponding subsequence "sh" from the second string "shop". This may result in inserting an "h", resulting in the edited first string "shoup". The subsequent subsequence of the letters "shou" associated with the letter "u" may be processed to determine an operation to obtain the corresponding subsequence "shop" of "shop". This may result in deleting the "u", resulting in the edited first string "shop". The subsequent subsequence of the letters "shop" associated with the last letter "p" may be processed to determine an operation to obtain the corresponding subsequence "shop" of the second string "shop". This may result in keeping the "p". Thus, the determined sequence of operations is a sequence of five operations MIMDM.

[0052] In step 305, the distance algorithm may apply a score assignment rule to each operation in the determined sequence of edit operations, which may result in each edit operation in the sequence of edit operations having an operation score and optionally an additional switching score.

[0053] In step 307, the distance algorithm calculates the distance between string s1 and string s 2と The edit distance may be calculated between the determined sequence of edit operations. The edit distance may be, for example, the sum of all scores assigned to the determined sequence of edit operations.

[0054] In step 309, the edit distance between string s1 and string s2 may be received, for example, as the output of a distance algorithm.

[0055] 4 is a flowchart of a method for determining similarity between two strings s1 and s2 according to an example of the present subject matter. String s1 has N1 characters, and string s2 has N2 characters. For illustrative purposes, the method illustrated in FIG. 4 may be implemented in the system shown in FIG. 1, but is not limited to this implementation.

[0056] In step 401, the distance algorithm may receive a first string having the first n1 characters of string s1 and a second string having the first n2 characters of string s2, where 0≦n1≦N1 and 0≦n2≦N2. In a first execution of step 401, the distance algorithm may receive a first string having the first n1=0 characters of string s1 and a second string having the first n2=0 characters of string s2. That is, the distance algorithm may receive two space characters.

[0057] In step 403, the distance algorithm may check whether it has determined or initialized in a previous iteration the combined scores for pairs of first and second strings (called surrounding pairs) each having n'1≧0 characters and n'2≧0 characters, where the surrounding pairs (n'1, n'2) may include the pair (n'1=n1, n'2=n2-1) or the pair (n'1=n1-1, n'2=n2), or both. The surrounding pairs (n'1, n'2) may further include the pair (n'1=n1-1, n'2=n2-1) if the last characters of the first and second strings are the same. This condition may not be met only for pairs (n'1, n'2) where n1=0 or n2=0. If one or more pairs of surrounding pairs (missing pairs) have not been previously processed or initialized with values, then in step 405 (from 0), the distance algorithm may determine, for each pair of missing pairs, one or more editing operations required to obtain the n'2 characters of the pair from the n'1 characters of the pair. A combined score may then be calculated for the missing pairs. Step 407 may then be executed.

[0058] If the distance algorithm has previously processed the surrounding pairs of n'1 characters and n'2 characters (which means that the distance algorithm has previously calculated the edit distance for the surrounding pairs (n1, n2-1) and / or (n1-1, n2) and / or (n1-1, n2-1) of the character sequence or initialized the pairs with a value), step 407 may be executed.

[0059] In step 407, the distance algorithm may select one of the surrounding pairs (n1, n2-1) and / or (n1-1, n2) and / or (n1-1, n2-1) of the character sequence that has the lowest edit distance. In a previous iteration on the selected pair (n'1, n'2), the distance algorithm may have determined a sequence of edit operations (termed the selected sequence of edit distances) to be performed on the first n'1 characters of string s1 to obtain the first n'2 characters of string s2. Therefore, in step 409, the distance algorithm may determine or assume that the sequence of edit operations to be performed on the first n1 characters of string s1 to obtain the first n2 characters of string s2 is the selected sequence of edit operations plus one additional edit operation. This one additional operation may depend on the selected pair (n'1, n'2). For example, if the selected pair is (n'1, n'2) = (n1, n2-1), the one additional edit operation is an insertion operation. If the selected pair is (n'1, n'2) = (n1-1, n2), one additional edit operation is a delete operation. If the selected pair is (n'1, n'2) = (n1-1, n2-1), one additional edit operation is a keep operation.

[0060] In step 411, the distance algorithm may determine the edit distance between the first n1 characters of string s1 and the first n2 characters of string s2 by applying the score assignment rule to the additional edit operations, and finally to the last edit operation in the selected sequence of edit operations, resulting in an additional score. The sum of the additional scores in the edit distance between the first n'1 characters of string s1 and the first n'2 characters of string s2 may then be provided as the edit distance between the first n1 characters of string s1 and the first n2 characters of string s2.

[0061] It may be determined whether n1=N1 AND n2=N2 (step 413). If so, the edit distance calculated in step 411 may be provided as the edit distance between string s1 and string s2 in step 415. If not, a new pair of (n1, n2) values ​​may be defined in step 414, and steps 401-415 may be repeated until n1=N1 and n2=N2 are reached. n1 and n2 may be incremented at each iteration according to the nested loops, where n1 represents the outer loop and n2 represents the inner loop.

[0062] FIG. 5A is a flowchart of an exemplary method for determining the similarity between a string s1=“soup” having N1=4 characters and a string s2=“shop” having N2=4 characters using a second implementation example. To do so, the method of FIG. 5A may use a matrix whose first dimension represents the characters of “soup” and whose second dimension represents the characters of “shop,” but is not limited to this matrix implementation. The matrix implementation may enable efficient use of processing resources. In practice, the method fills the matrix row by row, so that only a single row and updates to the current row are kept in memory rather than the entire matrix. For example, if the first row of the matrix is ​​currently stored in memory, the cells of the second row may be calculated successively until the second row is completely calculated. Next, with the second row in memory, the third row is filled, and so on. For example, assume that the manipulation score and switching score are equal to 1.

[0063] In step 501, the distance algorithm may create a matrix M 520A of size (N1+1)×(N2+1), as shown in FIG. 5B. The last N2 columns of the matrix represent the N2 characters of string s2. The last N1 rows of the matrix represent the N1 characters of string s1. The additional first column and first row represent the special character ε, which represents a blank string. The first row indicates that the cost value for obtaining the first n2 characters of string s2 from blank characters, e.g., the cost of obtaining "sho" from blank characters, is 3, corresponding to three insertion operations with a cost value of 1 each. The first column indicates that the cost value for obtaining blank characters ε from the first n1 characters of string s1, e.g., the cost of obtaining blank characters from "so" is 2, corresponding to two deletion operations with a cost value of 1 each. In other words, matrix M is initialized with initial cost values ​​that can be used when comparing strings s1 and s2.

[0064] Each cell of the matrix M has two corresponding first and second strings. As shown in FIG. 5B, 22 has a pair of first and second strings ("s", "s") and a cell M 23 has a pair of first and second strings ("s", "sh"), and the cell 55 has a pair of first and second strings ("soup", "shop"), etc. A second implementation of the distance algorithm performs a traversal of a cell, e.g., M, in one iteration to fill the cell with a cost value. 22 ~M 55 The cost value in each cell indicates the edit distance between the first and second strings associated with that cell.

[0065] The distance algorithm uses the corresponding upper cell M with pre-calculated / initialized values. i-1,j , left cell M i,j-1 and diagonal cell M i-1,j-1 Each current cell M ij(i is a row index and j is a column index) (e.g., the surrounding cells may include diagonal cells if the letters assigned to row i and column j are the same). For example, in matrix 520A, cell M 22 only have their surrounding cells filled with values, and therefore the distance algorithm will 22 You may start from

[0066] Therefore, the distance algorithm finds a cell M that has a pair of first and second strings ("s", "s"). 22 The distance algorithm may determine the cost of getting from "s" to "s", which is 0 since it involves a maintenance operation. This cost is calculated by dividing the cell M 22 For example, the distance algorithm may be derived from the three cell values ​​surrounding the upper cell value M 12 and the left cell value M 21 is equal to 1 and the diagonal cell value M 11 may be determined to be 0. In step 503, the distance algorithm 11 From M 22 To, M 12 From M 22 and M 21 From M 22 Determine the cost to move / transfer to and select the lowest cost. 12 From M 22 The cost of moving to Cell M 12 plus the cost of an additional operation to delete the letter 's', which is 1+1=2. 21 From M 22 The cost of moving to Cell M 21 plus the cost of an additional operation to insert the letter "s", which is 1+1=2. 11 From M 22 The cost of moving to Cell M 11 plus the cost of the extra operation to maintain the letter "s", which is 0+0. Therefore, in step 505, the distance algorithm calculates the distance between cell M22 may be assigned a minimum value of 0. Cell M 22 The value of this cell M 22 is marked by a plus sign "+" in the resulting matrix 520B to indicate that the movement / transition to reach was along the diagonal (i.e., the operation was a maintain operation). Matrix 520B contains the resulting contents after the first run of the distance algorithm.

[0067] 5B shows the matrix contents for different iterations of the distance algorithm. For example, matrix 520C is the matrix of the current cell M as shown in FIG. 34 This represents the status of the matrix before processing. Cell M 22 Similarly, the distance algorithm uses the upper cell value M 24 and the left cell value M 33 is equal to 3 and the diagonal cell value M 23 In step 503, the distance algorithm determines that M 23 From M 34 To, M 24 From M 34 and M 33 From M 34 The cost of moving to M may be determined and the lowest cost may be selected. 24 From M 34 The cost of moving to Cell M 24 plus the cost of an additional operation to delete the letter "o", which is 3+1=4. 33 From M 34 The cost of moving to Cell M 33 plus the cost of an additional operation to insert the letter "o", which is 3+1=4. 23 From M 34 The cost of moving to Cell M 23 , which is equal to the cost of the cell M plus the cost of an additional operation to keep the letter "o" (0) plus the switching score of 1 for switching to the keep operation, hence 2 + 1. Therefore, in step 505, the distance algorithm calculates 34may be assigned a minimum value of 3. 34 Since the value of is obtained from the corresponding diagonal cell, it is marked by a plus sign "+" in the resulting matrix 520D. FIG. 5B shows the contents of matrix M 520E after the final iteration of the distance algorithm. In one example, the last row and last column of matrix 520E may be reserved for reuse when comparing two strings containing "soup" and "shop," respectively. For example, to calculate the edit distance between two strings, "εsouppap" and "εshopping," having N characters and N characters, respectively, a new N×N matrix may be used, and the reserved rows and columns may be used so that only the cells in the last four columns and last three rows of the new matrix representing the two strings may be calculated. This may be done, for example, by repeatedly feeding the distance algorithm the first n1 characters of the string "εsouppap" and the first n2 characters of the string "εshopping" by changing n1 and n2 to new values ​​in each iteration (as described above), where the values ​​of n1 iterate over the range 0..N1 while the values ​​of n2 iterate over the range N2+1..N4 (the right quadrant of the new matrix), then the values ​​of n1 iterate over N1+1..N3 while the values ​​of n2 iterate over the range 0..N4 (the bottom two quadrants of the new matrix).

[0068] In step 507, the distance algorithm calculates the edit distance between string s1="soup" and string s2="shop" by calculating the bottom right cell M 55 may provide a value for

[0069] The method of Figure 5A may prioritize words where the editing operations are in the same location. Another way to do this is to prioritize words with longer spans of the same character, i.e., in matrix M, the longer the sequence, the more computational progress along the diagonal. |s1|+|s2|-1 and 3 |s1|+|s2|-1 There are different paths to get from the top left to the bottom right between|s1|+|s2|-1 (This gives possibilities.) Therefore, computing all paths and finding the one with the longest running diagonal is NP-complete. The method may instead rely on adding a penalty whenever there is a change to or from the diagonal progression through the matrix while building the matrix.

[0070] In another example, matrix 520F may be obtained using the method of FIG. 5A to compare string s1="shop" and string s2="shopping." Matrix 520F gives a distance of 5 between s1 and s2 (essentially, proceeding along the diagonal for "shop" as shown in FIG. 5B adds a penalty for deviation and four insertion operations: "p," "i," "n," and "g"). Because the calculated distance between s1 and s2 may have a value higher than the sum of the lengths of the two strings (|s1| + |s2|), the method may avoid this by employing the following: If both strings are blank, the similarity is 1. Otherwise, without loss of generality, it may be assumed that s1 is the shorter string. In that case, a penalty of up to 2|s1| may be introduced when all characters in s1 are contained in s2, and a penalty of up to |s2|-1 may be introduced when |s2|≦2|s1|, i.e., there are not enough characters in s1 to add two penalties per character.

[0071] FIG. 6A shows a second implementation example for generating a string s1="Durr" with N1=4 characters and a string s2="Du" with N2=5 characters. ·· 6A is a flowchart of an example method for determining the similarity between "Durr" and "Durrr." Similar to FIG. 5A, the method of FIG. 6A uses a first dimension representing the characters of "Durr" and a second dimension representing the characters of "Durrr." ··A matrix representing the characters of "rr" may be used, but is not limited to this matrix implementation. In this example, assume that the manipulation score and the first type of switching score are equal to 1 (since this example involves character weights, the second type of switching score may not be used in this example). The first type of switching score may be weighted by the average character weight. Additionally, each character in strings s1 and s2 may be associated with a respective weight. This is shown in FIG. 6B, where the character "rr" is associated with a weight of 1. ·· Each of the letters is associated with a weight of 10, except for "." However, to be able to use a matrix to assign penalties in this implementation, for each element in the matrix we record the number of insertion and deletion operations assigned, as well as the combined score. This is indicated for each cell of the matrix by the pair [cost / len], where "cost" represents the combined score and "len" is the number of insertion and / or deletion operations that have occurred so far. For example, for cell M of matrix 620A, 42 is associated with the pair [20 / 2], which indicates that the number of operations is 2 and the combined score calculated by the distance algorithm for the first string "Dur" and the second string "D" is 20. The number of edit operations of the first type is two delete operations, because the set of operations to get "D" from "Dur" includes one keep operation to keep "D" and two delete operations to delete "u" and "r". The penalty is calculated by dividing the total penalty "cost" by the number of characters "len" (to get the average character weight) and multiplying this by the constant p=w sw1 × SC. When a penalty is assigned, the number of insertion and / or deletion operations "len" and the penalty accumulated so far "cost" are reset to 0 because the corresponding penalty has already been integrated into the cost accumulated so far.

[0072] In step 601, the distance algorithm may create a matrix M 620A of size (N1+1)×(N2+1), as shown in FIG. 6B. The last N2 columns of the matrix represent the N2 characters of string s2. The last N1 rows of the matrix represent the N1 characters of string s1. An additional first column and first row represent a special character that represents a blank string. The first row represents a cost value for obtaining the first n2 characters of string s2 from a blank character, e.g., "Du" from a blank character. ·· The first column shows that the cost of obtaining "s1" from the first n1 characters of string s1, e.g., "Du", is 20, which corresponds to two deletion operations, each having a cost value of 1 and a weight of 10, i.e., 10*1+10*1+1*1. In other words, the matrix is ​​initialized with initial cost values ​​that can be used when comparing strings s1 and s2.

[0073] Each cell of the matrix M has two corresponding first and second strings. As shown in FIG. 6B, 22 has a pair of first and second strings ("D", "D") and cell M 23 has a pair of first and second strings ("D", "Du"), and cell M 56 is a pair of first and second strings ("Durr", "Du ·· An example implementation of the distance algorithm involves computing a cell, say M, in one iteration to fill the cell with a cost value. 22 ~M 56 The cost value in each cell indicates the edit distance between the first and second strings associated with that cell.

[0074] The distance algorithm uses the corresponding surrounding cells M with pre-calculated / initialized values. i-1,j , M i,j-1 and Mi-1,j-1 Each current cell M has ij (i is a row index and j is a column index), steps 603-605 may be performed for each row i and column j, e.g., if the letters assigned to row i and column j are the same, then the surrounding cells are the diagonal cells M i-1,j-1 For example, Figure 6B shows the status of matrix 620A after several iterations of steps 603-605. In matrix 620A, the distance algorithm operates row-wise, so that in the next iteration of steps 603-605, cell M 45 may be processed.

[0075] Current cell M ij For M, in step 603, the distance algorithm i-1,j-1 From M ij To, M i-1,j From M ij and M i,j-1 From M ij The cost of moving to M may be determined and the lowest cost may be selected. i-1,j-1 From M ij The cost of moving to may be determined / considered if the letters assigned to row i and column j are the same. i-1,j From M ij The cost of moving to Cell M i-1,j It is equal to the cost of M plus the cost of an additional operation to delete the character assigned to row i. i,j-1 From M ij The cost of moving to Cell M i,j-1 M i-1,j-1 From M ij The cost of moving to Cell M i-1,j-1 , plus the cost induced by the additional operation of maintaining the same character assigned to row i and column j. Thus, in step 605, the distance algorithm calculates the distance for cell M ij may be assigned the lowest cost value of the determined cost values. For example, if the lowest transition cost is M i-1,j-1 From M ijIf the weight of the additional operations is

number

[0076] For example, using the contents of matrix 620A in FIG. 6B, the distance algorithm calculates M 34 From M 45 To, M 35 From M 45 and M 44 From M 45 Determine the cost of transitioning to cell M by choosing the lowest cost. 45 You may process M 35 From M 45 The cost of moving to Cell M 35 It is equal to the cost of the operation M plus the cost of an additional operation to delete the character 'r' assigned to row 4. 44 From M 45 The cost of moving to Cell M 44 plus the cost of the additional operation to insert the letter "r" assigned to column 5. 3,4 From M 45 The cost of moving to Cell M 34 The cost of the first type of switching is equal to the cost of the second type of switching plus the cost induced by the additional operation to maintain the same letter "r" assigned to row 4 and column 5. The cost induced by the additional operation is the second type of switching and the last operation is the first type of switching. sw1 × SC, and the first type of switching score is 34weighted by the average character weight associated with c =cost / len=1 / 1=1. Therefore, in step 505, the distance algorithm calculates the distance between cell M 45 may be assigned the lowest cost value of 2.

[0077] The content of matrix 520B is the resulting content after processing all the cells of the matrix. In step 607, the distance algorithm finds the string s1="Durr" and the string s2="Du ·· The edit distance between the bottom right cell M and 56 6B shows a case where the string s1="Dunst" and the string s2="Du ·· 6 shows another matrix 620C that is the result of performing an iterative implementation of the distance algorithm between r and r.

[0078] In one example, multiple similarity metrics s1, s2, ..., sn may be combined. To do so, the following formula may be used: sc = 0.9max(s1, ..., sn) + 0.1min(s1, ..., sn). Using this approach, different similarity metrics can capture different aspects of the similarity between two strings. For example, the Levenshtein function may capture edit distance, while Jaccard similarity can account for word substitutions. On the other hand, the Jaccard similarity function can return a similarity of 1.0 for dissimilar strings (which may be undesirable). Therefore, instead of simply using the maximum value, the functions are combined by combining their maximum and minimum values. Using the approach presented in this disclosure can lead to a 7% increase in recall (from 85% to 92%).

[0079] Figure 7 shows, in pseudocode, elements of an example workflow for a matrix-based string comparison method for comparing strings s1 and s2. It is assumed that the strings have been preprocessed and that initial values ​​associated with blank string elements (i.e., values ​​in the first column and first row of the matrix) have already been added to the strings in an appropriate manner. It also uses two penalty functions, pen() and pend(). The pseudocode uses a matrix implementation to obtain the edit distance between strings s1 and s2. "(\e)row" refers to a blank row, such as the first row of matrix 520A. "prev" refers to the previous character. "cur" refers to the current row of the matrix. "top" refers to the top row. "ch1" and "ch2" refer to characters in strings s1 and s2, respectively, and "ch1" and "ch2" are assigned to the row and column of the current cell, respectively.

[0080] "s2_dist" and "s2_pen" are the distance and penalty for the current cell with the left cell, i.e., the cell in the same row but to the left of the current cell. lft_dist refers to the distance assigned to the left cell. lft_pen refers to the penalty assigned to the left cell.

[0081] "s1_dist" and "s1_pen" are the distance and penalty for the current cell using the top cell, i.e., the cell in the same column as the current cell but in the top row. top_dist refers to the distance assigned to the top cell. top_pen refers to the penalty assigned to the top cell.

[0082] "d2_dist" and "d2_pen" are the distance and penalty for the current cell using the diagonal cell, i.e., the cell in the column to the left of the current cell and in the top row. d_dist refers to the distance assigned to the diagonal cell. d_pen refers to the penalty assigned to the diagonal cell.

[0083] Penalty refers to the switching score described herein, and distance refers to the manipulation score.

[0084] The function pen() returns a penalty if the penalty objective passed to it has a length of 0, which represents a transition from diagonal movement (representing the second switching type) to lateral movement. Depending on whether a weighted version is implemented, this may return 0 or some penalty to be assigned. The function pend() returns the average penalty captured by a penalty objective representing a transition from lateral movement (representing the first switching type) to diagonal movement. Depending on whether a weighted version is implemented, this returns the accumulated distance in the penalty objective divided by the character in the penalty objective. In the weighted version of the code, depending on the possible penalty that occurs, it may be beneficial to trace the diagonal by changing the condition "ch1 == ch2" to the condition "ch1 == ch2 && d2_dist < s1_dist && && d2_dist < s2_dist".

[0085] FIG. 8 depicts an overall computerized system 800 (e.g., a data integration system) suitable for implementing at least a portion of the method steps as included in the present disclosure.

[0086] It is understood that the methods described herein are at least partially non-interactive and are automated by a computerized system such as a server or an embedded system. However, in an exemplary embodiment, the methods described herein can be implemented (partially) in an interactive system. These methods can be further implemented in software 812, 822 (including firmware 822), hardware (processor) 805, or a combination thereof. In an exemplary embodiment, the methods described herein are implemented in software as an executable program and are executed by a dedicated or general-purpose digital computer such as a personal computer, a workstation, a minicomputer, or a mainframe computer. Thus, the most common system 800 includes a general-purpose computer 801.

[0087] In an exemplary embodiment, in terms of hardware architecture, as shown in FIG. 8 , a computer 801 includes a processor 805, a memory (main memory) 810 coupled to a memory controller 815, and one or more input and / or output (I / O) devices (or peripherals) 10, 845 communicatively coupled via a local input / output controller 835. The input / output controller 835 can be, but is not limited to, one or more buses or other wired or wireless connections as known in the art. The input / output controller 835 may include additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communication, but these are omitted for brevity. Furthermore, the local interface may include address, control, or data connections, or a combination thereof, to enable appropriate communication between the aforementioned components. As described herein, the I / O devices 10, 845 may generally comprise any generalized cryptographic or smart card known in the art.

[0088] Processor 805 is a hardware device that executes software, particularly software stored in memory 810. Processor 805 can be any custom-made or commercially available processor, a central processing unit (CPU), a coprocessor among several processors associated with computer 801, a semiconductor-based microprocessor (in the form of a microchip or chipset), a microprocessor, or generally any device that executes software instructions.

[0089] The memory 810 can include any one or combination of volatile memory elements (e.g., random access memory (RAM, e.g., DRAM, SRAM, SDRAM, etc.)) and non-volatile memory elements (e.g., ROM, erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM)). It should be noted that the memory 810 can have a distributed architecture, with various components located remotely from each other but accessible by the processor 805.

[0090] The software in memory 810 may include one or more separate programs, each of which includes an ordered listing of executable instructions that implement logical functions, particularly functions associated with embodiments of the present disclosure. In the example of Figure 8, the software in memory 810 includes instructions 812, for example, instructions for managing a database, such as a database management system.

[0091] The software in memory 810 will typically also include a suitable operating system (OS) 811. The OS 811 essentially controls the execution of other computer programs, such as software 812, which may implement methods as described herein.

[0092] The methods described herein may be in the form of a source program 812, an executable program 812 (object code), a script, or any other entity comprising a set of instructions 812 to be executed. In the case of a source program, the program must be translated via a compiler, assembler, interpreter, etc., which may or may not be contained within memory 810, in order to operate properly in conjunction with the OS 811. Furthermore, the methods may be written as an object-oriented programming language with classes of data and methods, or a procedural programming language with routines, subroutines, or functions, or a combination thereof.

[0093] In an exemplary embodiment, a conventional keyboard 850 and mouse 855 may be coupled to the input / output controller 835. Other output devices, such as I / O devices 845, may include input devices, such as, but not limited to, printers, scanners, microphones, etc. Finally, I / O devices 10, 845 may further include devices that communicate both input and output, such as, but not limited to, network interface cards (NICs) or modulators / demodulators (for accessing other files, devices, systems, or networks), radio frequency (RF) or other transceivers, telephone interfaces, bridges, routers, etc. I / O devices 10, 845 may be any generalized cryptographic card or smart card known in the art. System 800 may further include a display controller 825 coupled to a display 830. In an exemplary embodiment, system 800 may further include a network interface coupled to a network 865. Network 865 may be an IP-based network for communication between computer 801 and any external servers, clients, etc. via a broadband connection. Network 865 transmits and receives data between computer 801 and external systems 30, which may be involved in performing some or all of the method steps discussed herein. In an exemplary embodiment, network 865 may be a managed IP network managed by a service provider. Network 865 may be implemented in a wireless manner using wireless protocols and technologies such as WiFi, WiMax, etc. Network 865 may also be a packet-switched network such as a local area network, a wide area network, a metropolitan area network, the Internet network, or other similar types of network environments.The network 865 may be a fixed wireless network, a wireless local area network (LAN), a wireless wide area network (WAN), a personal area network (PAN), a virtual private network (VPN), an intranet, or other suitable network system, and includes equipment for receiving and transmitting signals.

[0094] If computer 801 is a PC, workstation, intelligent device, etc., the software in memory 810 may further include a basic input / output system (BIOS) 822. The BIOS is a set of basic software routines that initializes and tests hardware at startup, starts the OS 811, and supports the transfer of data between hardware devices. The BIOS is stored in ROM so that the BIOS can be executed when computer 801 is booted.

[0095] When computer 801 is in operation, processor 805 is configured to execute software 812 stored in memory 810, to communicate data to memory 810, and to control the operation of computer 801 generally in accordance with the software. The methods and OS 811 described herein are read by processor 805, possibly buffered within processor 805, and then executed, in whole or in part, but typically the latter.

[0096] When the systems and methods described herein are implemented in software 812, the methods can be stored on any computer-readable medium, such as storage 820, for use by or in connection with any computer-related system or method, as shown in Figure 8. Storage 820 may include disk storage, such as HDD storage.

[0097] The subject matter may provide for the following provisions:

[0098] Clause 1. A method for determining the distance between a string s1 having N1 characters, where N1≧0, and a string s2 having N2 characters, where N2≧0, comprising: a. providing a distance algorithm, said distance algorithm comprising: i. receiving a first string and a second string; ii. determining a sequence of one or more edit operations to perform on characters of the first string to obtain the second string, the edit operations being of a first type or a second type, the first type edit operation comprising a character insertion operation or a character deletion operation, and the second type edit operation comprising a character preserving operation, the first type edit operation being associated with an operation score indicating the cost of applying the edit operation, and the first type edit operation being associated with a switching score indicating whether it is immediately followed in the sequence by a second type edit operation; iii. combining the switching scores or the operation scores, or both, associated with the sequence of editing operations, resulting in a combined score indicative of the level of similarity between the first string and the second string; and a providing step configured to: b. inputting the first n1 characters of the string s1 as the first string and the first n2 characters of the string s2 as the second string, where 0≦n1≦N1 and 0≦n2≦N2, into the distance algorithm to obtain the combined score; c. determining the distance between the string s1 and the string s2 using the obtained combined score; A method comprising:

[0099] Clause 2. The method of clause 1, wherein when n1=N1 and n2=N2, the obtained combined score indicates the distance between the string s1 and the string s2.

[0100] Clause 3. n1=0 and n2=0, The inputting step includes: repeatedly inputting the first n1 characters of the string s1 and the first n2 characters of the string s2 into the distance algorithm, where n1 and n2 are incremented according to nested loops, n1 representing the outer loop and n2 representing the inner loop. and wherein the distance algorithm, at each iteration, a first combined score has previously been determined for the first string having n1-1 characters and the second string having n2 characters using the first sequence of editing operations; and / or a second combined score was previously determined for the first string having n1 characters and the second string having n2-1 characters using the second sequence of editing operations; and / or A third combined score was previously determined for the first string having n1-1 characters and the second string having n2-1 characters using the third sequence of editing operations, and the last characters of the first string and the second string are the same or and determining determining a combined score for the first combined score, the second combined score, and the third combined score if it is determined that the combined score has not been previously determined, and selecting the lowest score of the determined combined scores; determining an additional operation to be performed in addition to the first, second, or third sequence of edit operations associated with the selected lowest cost to obtain the second string from the first string, wherein if the selected pair is (n1, n2-1), the additional operation is the insert operation, if the selected pair is (n1-1, n2), the additional operation is the delete operation, and if the selected pair is (n1-1, n2-1), the additional operation is the keep operation; and configured to determine the sequence of edit distances by the sequence of editing operations includes one of the first sequence, the second sequence, or the third sequence of editing operations associated with the selected lowest score and the determined additional operation; the distance algorithm is configured to combine, at each iteration, the switching scores or the operation scores or both associated with the sequence of editing operations by combining the lowest score with the switching score or the operation score or both associated with the additional operation; 2. The method of claim 1, wherein the determining the distance between the string s1 and the string s2 is performed using the resulting combined score of the last iteration.

[0101] Clause 4. The method of clause 3, further comprising providing an initial value of the combined score for a pair of a first string and a second string having n1 characters and n2 characters, respectively, where n1=0 and n2=0, 1, ... N2, or n2=0 and n1=0, 1, ... N1.

[0102] Clause 5. A step of obtaining the combined scores calculated for each pair of a first string having n1=N1 characters and a second string having n2 characters varying from 1 to N2, and the combined scores calculated for each pair of a first string having n1 characters varying from 1 to N1 and a second string having n2=N2 characters; receiving a request to compare two strings s3 and s4 having N3 and N4 characters, respectively, where s3=s1+m1 and s4=s2+m2, and m1 and m2 are strings of zero or more characters; repeating the method using the secured scores by repeatedly inputting the first n1 characters of the string s3 and the first n2 characters of the string s4 into the distance algorithm, wherein the value of n1 iterates over the range 0..N1 while the value of n2 iterates over the range N2+1..N4, then the value of n1 iterates over N1+1...N3 while the value of n2 iterates over the range 0...N4; 5. The method of clause 3 or 4, further comprising:

[0103] Clause 6. The method of any one of preceding clauses 1 to 5, further comprising a step of providing a character weight for each character of the strings s1 and s2, and wherein the associating of the operation score to the first type of editing operation comprises weighting the association score with the character weight of the character involved in the first type of editing operation.

[0104] Clause 7. The method further comprises providing a character weight for each character of the string s1 and the string s2, wherein the associating the operation scores to the first type of editing operation includes weighting the operation scores with the character weights of the characters involved in the first type of editing operation, and the associating the switching scores to the first type of editing operation includes weighting the switching scores with a weight w c weighting the weights w c 7. The method according to any one of the preceding clauses 1 to 6, wherein x is a predefined function of the combined score of a subsequence of operations having the first type of edit operation as the last operation and the number of first type of edit operations in the subsequence.

[0105] Clause 8. The method of clause 7, wherein the function is a ratio of the combined score to the number of edit operations of a first type in the subsequence.

[0106] Clause 9. The method of any one of the preceding clauses 1 to 8, wherein the switching score is referred to as a first type switching score, and wherein the distance algorithm is further configured to associate a second type switching score with each first type edit operation in the sequence of edit operations if it is immediately preceded in the sequence by an edit operation of a second type.

[0107] Clause 10. A method according to any one of the preceding clauses 1 to 9, wherein the steps of determining the sequence of operations and the association of the operation scores and the deviation scores are performed character-by-character in parallel.

[0108] Clause 11. N1 ≥ 1 and N2 ≥ 1, and the distance is determined by the following formula:

number

[0109] Clause 12. The method of any one of the preceding clauses 1 to 11, wherein the string s1 is shorter than the string s2.

[0110] Clause 13. N1 ≥ 1 and N2 ≥ 1, and the similarity levels are the following distances:

number

number

[0111] Clause 14. The method of any one of the preceding clauses 1 to 13, wherein the switching score is less than the sum of the operation scores for one character insertion operation and one character deletion operation.

[0112] Clause 15. The method of any one of the preceding clauses 1 to 14, wherein the distance algorithm is configured to determine the sequence of one or more edit operations by determining different candidate sequences of edit operations and selecting the candidate sequence that provides the lowest combined score.

[0113] A method for determining the distance between a string s1 having N1 characters, where N1≧1, and a string s2 having N2 characters, where N2≧1, is provided, the method comprising the steps of: a. providing a distance algorithm, said distance algorithm comprising: i. receiving a first string and a second string; ii. determining a sequence of one or more edit operations to perform on characters of the first string to obtain the second string, the edit operations being of a first type or a second type, the first type edit operation comprising a character insertion operation or a character deletion operation, and the second type edit operation comprising a character preserving operation, the first type edit operation being associated with an operation score indicating the cost of applying the edit operation, and the first type edit operation being associated with a switching score indicating whether it is immediately followed in the sequence by a second type edit operation; iii. combining the switching scores or the operation scores, or both, associated with the sequence of editing operations, resulting in a combined score indicative of the level of similarity between the first string and the second string; and a providing step configured to: b. inputting the first n1 characters of the string s1 as the first string and the first n2 characters of the string s2 as the second string, where 1≦n1≦N1 and 1≦n2≦N2, into the distance algorithm to obtain the combined score; c. determining the distance between the string s1 and the string s2 using the obtained combined score; Equipped with.

[0114] The present disclosure may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions that cause a processor to perform aspects of the present disclosure.

[0115] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves that record instructions, and any suitable combination of the foregoing. Computer-readable storage medium, as used herein, should not be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0116] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to a computer-readable storage medium in each computing / processing device for storage.

[0117] The computer-readable program instructions for carrying out the operations of the present disclosure may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, C++, etc., and procedural programming languages ​​such as the “C” programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, an electronic circuit, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer-readable program instructions to personalize the electronic circuit by utilizing state information of the computer-readable program instructions to perform aspects of the present disclosure.

[0118] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0119] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, whereby the instructions, executed by the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the flowcharts or block diagrams, or both. These computer-readable program instructions may also be stored on a computer-readable storage medium, whereby the instructions can instruct a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium having the instructions stored thereon comprises an article of manufacture including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts or block diagrams, or combination thereof.

[0120] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be executed on the computer, other programmable apparatus, or other device to generate a computer-implemented process, whereby the instructions executing on the computer, other programmable apparatus, or other device implement the functions / operations specified in one or more blocks of the flowcharts or block diagrams, or a combination thereof.

[0121] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions, that implement the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may actually be implemented as a single step, or may be executed concurrently, substantially concurrently, partially, or fully overlapping in time, or the blocks may even be executed in the reverse order depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

Claims

1. N １ N≧0 １ A string s having characters １ and N ２ N≧0 ２ A string s having characters ２ 1. A method for determining the distance between a computer system providing a distance algorithm, said distance algorithm comprising: receiving a first string and a second string; determining a sequence of one or more edit operations to perform on characters of the first string to obtain the second string, the one or more edit operations being of a first type or a second type, the first type edit operation comprising a character insertion operation or a character deletion operation, the second type edit operation comprising a character preserving operation, the first type edit operation being associated with an operation score indicative of a cost for applying the one or more edit operations, and the first type edit operation being associated with a switching score indicative of whether it is immediately followed in the sequence by a second type edit operation; combining the switching scores or the operation scores or both associated with the sequence of editing operations, resulting in a combined score indicative of a level of similarity between the first string and the second string; a providing step configured to: The computer system may provide the distance algorithm with the string s as the first string to obtain the combined score. １ The first n １ characters and the string s as the second string ２ The first n ２ inputting characters, wherein 0≦n １ ≦N １ and 0≦n ２ ≦N ２ and The computer system uses the obtained binding scores to １ and the string s ２ determining the distance between A method comprising:

2. n １ = N １ and n ２ = N ２ , the resulting combined score is １ and the string s ２ The method of claim 1 , wherein the distance between

3. n １ = 0 and n ２ = 0, The inputting step includes: The distance algorithm uses the string s １ The first n １ characters and the string s ２ The first n ２ repeatedly inputting n characters, １ and n ２ is incremented according to the nested loop, and n １ represents the outer loop, and n ２ represents the inner loop, the repeated input stage and wherein the distance algorithm, at each iteration, Using the first sequence of editing operations, n １ - the first string having one character and n ２ a first combined score has previously been determined for the second string having characters; and / or Using the second sequence of editing operations, １ the first string having n characters and ２ - a second combined score has previously been determined for the second string having one character; and / or Using the third sequence of editing operations, n １ - the first string having one character and n ２ - a third combined score was previously determined for the second string having one character, and the last character of the first string and the second string is the same; or and determining determining a combined score for the first combined score, the second combined score, and the third combined score if it is determined that the combined score has not been previously determined, and selecting the lowest score of the determined combined scores; determining additional operations to perform in addition to one of the first sequence, the second sequence, or the third sequence of editing operations associated with the selected lowest cost to obtain the second string from the first string, wherein the selected pair is (n １ , n ２ −1), the addition operation is the character insertion operation, and the selected pair is (n １ −1, n ２ ), the addition operation is the character deletion operation, and the selected pair is (n １ −1, n ２ -1), then the additional operation is the character-preserving operation; and and configured to determine the sequence of edit distances by the sequence of editing operations includes the one of the first sequence, the second sequence, or the third sequence of editing operations associated with the selected lowest score and the determined additional operation; the distance algorithm is configured to combine, at each iteration, the switching scores or the operation scores or both associated with the sequence of editing operations by combining the lowest score with the switching score or the operation score or both associated with the additional operation; The character string s １ and the string s ２ The method of claim 1 , wherein the determining the distance between is performed using the resulting combined score of the last iteration.

4. The computer system is １ letters and n ２ providing an initial value of the combined score for a pair of a first string and a second string having n characters; １ = 0 and n ２ = 0, 1, ..., N ２ , or n ２ = 0 and n １ = 0, 1, ..., N １ The method of claim 3, wherein

5. The computer system is １ = N １ a first string having characters and ２ fluctuates with n ２ the combined score calculated for each pair of second strings having characters, and １ fluctuates with n １ a first string having n characters and ２ = N ２ obtaining the calculated combined score for each pair of second strings having characters; The computer systems each include N ３ letters and N ４ Two strings s with characters ３ and s ４ receiving a request to compare s ３ =s １ +m １ and s ４ =s ２ +m ２ and m １ and m ２ is a string of zero or more characters; The computer system applies the distance algorithm to the string s ３ The first n １ characters and the string s ４ The first n ２ repeating the method using the secured score by repeatedly entering n characters, １ The value of is in the range 0..N １ Iterates over n ２ The value of is in the range N ２ +1...N ４ Iterate over n １ The value of N １ +1...N ３ Iterates over n ２ The values ​​of are in the range 0...N ４ Repeating the process over and over The method of claim 3 further comprising:

6. The computer system １ and the string s ２ 2. The method of claim 1 , further comprising providing a character weight for each character of the first type of editing operation, wherein the associating the operation score to the first type of editing operation comprises weighting the association score with the character weight of the character involved in the first type of editing operation.

7. The computer system according to claim 1, wherein the character string s １ and the string s ２ wherein the associating the operation scores to the first type of editing operation includes weighting the operation scores with the character weights of the characters involved in the first type of editing operation, and the associating the switching scores to the first type of editing operation includes weighting the switching scores with a weight w ｃ weighting by w ｃ 6. The method of claim 5, wherein σ is a predefined function of the combined score of a subsequence of operations having the first type of edit operation as the last operation and the number of first type of edit operations in the subsequence.

8. The method of claim 7 , wherein the function is a ratio of the combined score to the number of edit operations of the first type in the subsequence.

9. 2. The method of claim 1 , wherein the switching scores are referred to as first-type switching scores, and the distance algorithm is further configured to associate a second-type switching score with each first-type editing operation in the sequence of editing operations if it is immediately preceded in the sequence by an editing operation of a second type.

10. The method of claim 1 , wherein the determining the sequence of operations and the associating the operation scores and deviation scores are performed in parallel, character by character.

11. N １ ≧1 and N ２ ≧1, and the distance is [Equation 1] where p is the switching score and d ｇｌ (s １ , s ２ ) is the combined score.

12. The character string s １ is the string s ２ The method of claim 1 , wherein the

13. N １ ≧1 and N ２ ≧1, and the similarity level is [Equation 2] and further determined in accordance with: [Equation 3] is the average character weight of the strings s1 and s2, and d ｇｌ (s １ , s ２ 7. The method of claim 6, wherein p is the combination score and p is the switching score.

14. The method of claim 1 , wherein the switching score is less than the sum of the operation scores for one character insertion operation and one character deletion operation.

15. The method of claim 1 , wherein the distance algorithm is configured to determine the sequence of one or more edit operations by determining different candidate sequences of edit operations and selecting the candidate sequence that provides the lowest combined score.

16. A method for record matching, comprising: a step in which the computer system compares pairs of attribute values ​​of two records using the method of claim 1 to yield individual similarity levels for the attributes; and a step in which the computer system combines the individual similarity levels to determine whether the two records are matched records.

17. A computer program product causing a computer to carry out the method according to any one of claims 1 to 15.

18. N １ N≧0 １ A string s having characters １ and N ２ N≧0 ２ A string s having characters ２ 1. A computer system for determining a similarity between Memory and a processor; a local data storage having computer executable code stored thereon, the computer executable code including program instructions executable by the processor to cause the processor to perform a method; The method comprises: providing a distance algorithm, said distance algorithm comprising: receiving a first string and a second string; determining a sequence of one or more edit operations to perform on characters of the first string to obtain the second string, the one or more edit operations being of a first type or a second type, the first type edit operation comprising a character insertion operation or a character deletion operation, the second type edit operation comprising a character preserving operation, the first type edit operation being associated with an operation score indicative of a cost for applying the one or more edit operations, and the first type edit operation being associated with a switching score if it is immediately followed in the sequence by a second type edit operation; combining the switching scores or the operation scores or both associated with the sequence of editing operations, resulting in a combined score indicative of a level of similarity between the first string and the second string; a providing step configured to: To obtain the combined score, the distance algorithm uses the string s as the first string. １ The first n １ characters and the string s as the second string ２ The first n ２ inputting characters, wherein 0≦n １ ≦N １ and 0≦n ２ ≦N ２ and The obtained binding score is used to １ and the string s ２ determining the distance between A computer system comprising:

Images