System for comparing two strings and computer-readable storage medium

Abstract

System (200) for comparing two strings (11, 11') comprising the following: • a communication unit (210) designed to receive a first and a second input string (11, 11') each specifying a sequence of characters of an alphabet, in particular via an application interface over a network (300), e.g. via the Internet; • a processing unit (220) configured to insert a separator sequence (13) between the characters of the first and second input string (11, 11') to generate a corresponding first and second modified input string (14, 14'), wherein the separator sequence (13) specifies a stochastically rare string; • a tokenization unit (230) configured to tokenize the first and second modified input strings (14, 14') into a first and second token sequence (15, 15') such that a token (19, 19') of the first and second token sequence (15, 15') corresponds to a character of the first and second input strings (11, 11') or of the separator sequence (13), respectively; • a comparison unit (250) trained to compare the first and second input strings (11, 11') using a transformer-based inference model (17) with the first and second token sequence (15, 15'), excluding the tokens (19') corresponding to the separator sequence (13) from the comparison.

Description

[0001] The invention relates to a system for comparing two strings of characters and a computer-readable storage medium.

[0002] In many industries and application areas, it is necessary to compare concepts and words. Such a comparison can consider various aspects, such as character similarity, auditory similarity, visual similarity, or semantic similarity. Numerous solutions already exist for each of these comparison operations. For example, in information technology, comparing character strings is easily accomplished by comparing the bit sequences. Furthermore, taxonomies of concepts exist that allow for a comparison of the semantic meaning of terms. Solutions also exist for auditory and visual similarity. For instance, phonetic dictionaries exist that can be used to determine auditory similarity. Additionally, the visual representation of concepts can be solved using either image processing algorithms or multimodal models.

[0003] A large class of AI models that are also suitable for comparing concepts or strings are large language models (LLMs). Typically, such LLMs are built on the Transformer architecture. This architecture was first described in “Attention is all you need”, Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin, 2017, Crurran Associates, ISBN: 9781510860964.

[0004] Transformer-based models are a special type of artificial neural network that employs various mechanisms to efficiently process sequential input data, such as text, over very large amounts of data. Among other things, Transformer-based models utilize three key concepts to process large volumes of text: positional encoding, attention, and self-attention.

[0005] Due to the widespread use of LLMs, it is desirable to also employ them for string comparison. While LLMs deliver excellent results in many natural language processing applications, string comparison, particularly auditory and visual comparison, presents a challenge. LLMs output the next most probable character or token for a given input string, processing the input one token at a time. This process makes it difficult for LLMs to determine, for example, the number of characters in a string, as the model only works with one token at a time.

[0006] For example, for two strings of characters that differ only in one letter but are pronounced very differently, a very high auditory similarity is often determined.

[0007] Based on this background, the object of the invention is to improve the comparison of character strings. In particular, the object of the invention is to improve the auditory and / or visual comparison of character strings. Furthermore, it is a particular object of the invention to improve the comparison of character strings using transformer-based AI models. Finally, it is a particular object of the invention to specify a measure of similarity between two character strings.

[0008] The problem is solved by the subject matter of the independent claims.

[0009] Furthermore, the task is solved in particular by a system for comparing two strings, which includes the following: • a communication unit that is trained to receive a first and a second input string, each specifying a sequence of characters from an alphabet; • a processing unit trained to insert a separator sequence between the characters of the first and second input strings to generate a corresponding first and second modified input string, wherein the separator sequence specifies a stochastically rare string; • a tokenization unit configured to tokenize the first and second modified input strings into a first and second token sequence such that each token of the first and second token sequence corresponds to a character of the first and second input strings or the separator sequence, respectively; • a comparison unit trained to compare the first and second input strings using a transformer-based inference model based on the first and second token sequences, excluding tokens corresponding to the separator sequence from the comparison.

[0010] The invention is based on the idea of ​​forcing tokenization by inserting a separator sequence between the characters of the corresponding input strings, in which the input strings of individual characters are tokenized, i.e., each character of the input string is mapped to a single token.

[0011] Tokenization is typically a key component of processing text intended for use with a transformer-based inference model. This is because a transformer-based inference model does not operate on the input text itself. Instead, it maps an input sequence into tokens, where a token can represent a single character, a fragment of a word, or an entire word. These tokens are mapped to numerical identification numbers that the inference model can interpret. Examples of well-known tokenization algorithms include Byte-Pair Encoding (BPE) and WordPiece. In these approaches, words are divided into fragments. This has the advantage of allowing the model to better handle unknown words or variations. Furthermore, it makes it possible to process words that do not appear in the training data itself.

[0012] It is also possible to use so-called word embeddings. These are techniques for converting text into vectors. An embedding can be applied after tokenization to store relationships between words or concepts. Well-known methods include Word2Vec. The embeddings thus give meaning to the tokens by mapping them into a vector space where words / tokens with similar meanings are located close to each other.

[0013] However, if two input strings are to be compared, dividing the input strings into subwords can lead to words that differ by only a single letter being interpreted as very dissimilar, since different tokens are used. In this case, it is irrelevant for the comparison whether, for example, in a token comprising three characters, one, two, or three of the characters are different.

[0014] The invention takes advantage of this principle by enforcing the assignment of a single token to each character in the input strings. This is achieved by inserting stochastically rare characters between the individual characters of the input strings. Specifically, if a stochastically rare character is inserted, a tokenization algorithm will not create a token consisting of a stochastically very rare character and a character from the input string.

[0015] The proposed method thus makes it possible for a comparison operation to take into account the small difference between two strings that differ by only one character. This results in a significant improvement in accuracy compared to standard tokenization.

[0016] In one embodiment, the sequence of characters of the separation may be rare with respect to a reference corpus of tokens, in particular the probability of the occurrence of the separation sequence in the reference corpus may be below a threshold value, e.g. 0.1%, 0.01%, 0.001%, or 0.0001%.

[0017] The reference corpus of tokens can be learned, for example, during the training of the inference model. This means that a large amount of training data, which can be in the form of text, allows the construction of a reference corpus, where frequently occurring character combinations can form a token. Very rare characters are usually not grouped together but form a single token.

[0018] In one embodiment, the transformer-based inference model can be created through a training step. This training step can involve data collection, in which text corpora are gathered and prepared, such as the open-source "Common Crawl," which comprises approximately 2.4 billion web pages. Furthermore, the training can include tokenizing the collected data and building a corresponding lexicon. Alternatively, the training can be performed using self-supervised algorithms, which learn without predefined labels. These algorithms can predict the next word or letter for a given string, or they can fill in gaps in texts. Finally, the training can include optimizing a lost function using backpropagation and gradient-based optimization algorithms.

[0019] In one embodiment, the system may include a reduction unit configured to remove the tokens in the first and second token sequences corresponding to the separation sequence, to generate a purified first and a purified second token sequence, wherein the comparison unit is configured to perform the comparison using the purified first and second token sequences, in particular instead of the first and second token sequences.

[0020] It is therefore possible to actively remove the tokens from the separator sequence. This ensures that these tokens are not considered during the comparison. Furthermore, it is conceivable that an input parameter informs the inference model that the tokens or characters of the separator sequence should not be used for the comparison, for example, via so-called "prompt engineering".

[0021] In one embodiment, the system may include an output of a comparison result, wherein the comparison result may include an indication of an auditory and / or written similarity, preferably a percentage, wherein 100% may indicate an identity of the first and second input strings and 0% may indicate no similarity.

[0022] The system is particularly intuitive to use if the comparison result is displayed as a percentage, for example, on a screen. Alternatively, similarity classes such as "identity," "high similarity," "medium similarity," "low similarity," and "no similarity" could be defined. In all cases, however, the degree of similarity can be specified, allowing a user or a connected computing device to process the comparison result immediately.

[0023] In one embodiment, the output may include transmitting the comparison result via an application interface, e.g., via an API, via a communication network, e.g., the Internet, to a client device, e.g., an end device.

[0024] This implementation allows an end device, such as a smartphone or user computer, to provide the input strings via the internet, and a server application to perform the comparison. This has the advantage that complex calculations can be performed in a data center with appropriate hardware and utilization, thus conserving the user's computing resources.

[0025] In one embodiment, the tokenization can include subdividing the first and second modified input strings into tokens, wherein a token can comprise at least one, preferably a plurality, of characters from a natural language alphabet.

[0026] Tokens can generally contain at least one character from a natural language alphabet. As explained above, the number of characters per token varies, with frequently recurring character strings in a reference corpus resulting in them being represented as a single token.

[0027] The subdivision can include the first and second modified input strings, each consisting of one character of the first and second input strings, and the characters of the separator sequence.

[0028] Tokenization is thus performed in such a way that a token is generated for each character of the input strings and a token is also generated for each character of the separator sequence.

[0029] This mechanism forces the inference model to consider the individual characters of the input strings separately, rather than longer tokens.

[0030] In one embodiment, the comparison may include a trademark law assessment of a likelihood of confusion, which takes into account an auditory and / or visual similarity.

[0031] The problem is further solved in particular by a computer-readable storage medium which contains instructions that cause at least one processor to implement the functions of the system as described above when the instructions are executed by the at least one processor.

[0032] In one embodiment, the system may include a storage unit which may be configured to store a reference corpus, wherein the reference corpus may specify a plurality of tokens, in particular tokens of a natural language.

[0033] In one embodiment, the system may include a reduction unit configured to remove the tokens of the first and second token sequences, each corresponding to the separation sequence, to generate a purified first and a purified second token sequence, wherein the comparison unit is configured to perform the comparison using the purified first and second token sequences, in particular instead of the first and second token sequences.

[0034] In one embodiment, the comparison unit can be configured to perform the comparison with regard to an auditory and / or written similarity.

[0035] In one embodiment, the tokenization unit can be configured to perform a subdivision of the first and second input sequence into tokens, wherein a token can comprise at least one, preferably a plurality, of characters from a natural language alphabet.

[0036] In one embodiment, the tokenization unit can be configured to divide the first and second modified input strings into one character each of the first and second input strings and the characters of the separator sequence.

[0037] In one embodiment, the comparison unit can be configured to remove characters from the inserted separator sequence, so that the comparison can be performed for the characters of the first and second input strings.

[0038] This results in similar or identical advantages to those already described with regard to the system. Further embodiments are described in the dependent claims.

[0039] The invention will be explained in more detail below with reference to exemplary embodiments and figures.

[0040] This shows Fig. 1 a schematic representation of a procedure for comparing two strings; Fig. 2 an embodiment for generating cleaned token sequences; Fig. 3 a schematic representation of a procedure for outputting a comparison result; Fig. 4 a schematic representation of a system for comparing two strings; and Fig. 5 a computer system.

[0041] In the following, the same reference numbers are used for identical or equivalent elements.

[0042] The Fig. Figure 1 shows a schematic representation of a procedure 100 for comparing two strings 11, 11'. As can be seen from the Fig. As indicated in section 1, input strings 11, 11' are provided. These input strings can consist of two single words. It is also conceivable that the input strings 11, 11' contain sequences of several words. Preferably, however, the input strings 11, 11' are short, i.e., they comprise only a few words, e.g., fewer than 20 words, fewer than 10 words, fewer than five words, or fewer than three words. Furthermore, it is preferably provided that the input strings 11, 11' have a maximum character length of 512 characters, fewer than 256 characters, fewer than 128 characters, fewer than 64 characters, or fewer than 32 characters. The input strings 11, 11' can, for example, be provided as a string data type. It is also conceivable that other data formats are used to provide the input strings 11, 11'.

[0043] Furthermore, a separator sequence 13 is provided. In the illustrated embodiment, the separator sequence 13 comprises a stochastically rare character with respect to a reference corpus. For example, the separator sequence could be the character “‡”. In the method 100, the separator sequence 13 is first inserted between the respective characters of the first and second input strings 11, 11' in an insertion step 110. This means that the separator sequence is inserted between each individual character of the corresponding input string 11, 11'. Through this operation, a first and a second modified input string 14, 14' are generated by the insertion step 110, wherein the first modified input string corresponds to the first input string 11 with inserted separator sequences 13, and the second modified input string 14' corresponds to the second input string 11' with inserted separator sequences 13.

[0044] Subsequently, in tokenization step 120, the first and second modified input strings 14, 14' are tokenized. Well-known algorithms can be used for tokenization, such as Byte-Pair Encoding (BPE), WordPiece, SentencePiece, Unigram Language Model, or morpheme-based tokenization. However, tokenization algorithms that do not perform character-based tokenization are generally used; that is, the generated tokens also include tokens containing more than one character.

[0045] Tokenization 120 generates a first and second token sequence 15, 15'. The first token sequence 15 is generated by tokenizing the first modified input string 14. The second token sequence 15' is generated by tokenizing the second modified input string 14'. Tokenization 120 is executed such that each token corresponds to a character of the first or second input string, or to the separator sequence 13. In other words, each character of the first and second input strings 11, 11' is assigned a token, and each character of the separator sequence 13 is assigned a token. The tokens assigned to the first modified input string 14 form the first token sequence 15, and the tokens assigned to the second modified input string 14' form the second token sequence 15'. The tokens are mapped to a corpus of known tokens via identification numbers.This means that during tokenization, a substring of the first and second input string 11, 11' is mapped to a corpus of tokens, e.g. via an identification number that identifies the token.

[0046] In the subsequent optional removal step 130, the tokens corresponding to the characters of the separator sequence 13 are removed from the first and second token sequences 15, 15', resulting in clean first and second token sequences 16, 16'. The clean token sequences 16, 16' thus contain only tokens corresponding to the characters of the first and second input strings 11, 11'.

[0047] The cleaned first and second token sequences 16, 16' now form input parameters for a transformer-based inference model 17, which is used in comparison step 140 to compare the first and second input strings 11, 11'. It is possible to provide additional input parameters, such as instructions, to the inference model 17 for comparison step 140. For example, extensive instructions can be provided, such as examples of how the comparison should be performed, what formats the output should have, whether an auditory or a written similarity should be checked, whether multiple passes of the comparison should be performed, or whether certain word components, such as consonants or vowels, should have increased weight in the comparison.

[0048] In one embodiment, it is also possible for the removal step 130 to be performed by the inference model itself. For example, an input parameter can be used to inform the inference model that the separation sequence 13 should not be considered when comparing the token sequences. More generally, the application thus provides that the removal of the tokens in the first and second token sequences, which each correspond to the separation sequence, can be implemented as ignoring or disregarding the corresponding tokens during a comparison.

[0049] The Fig. Section 2 illustrates once again, using a concrete example, how a string 11, 11' is prepared for comparison. In the upper section of the Fig. In step 2, an initial input string 11 is read in, and a separator sequence 13 is inserted between the respective characters of the input string 11 to generate a modified input string 14. In a subdivision step 121, which is part of the tokenization step 120, each character of the modified input string 14 is assigned a token 19, 19'. In the example of the Fig. In Figure 2, tokens 19 and 19' are represented by the corresponding characters of the modified input string 14. In other embodiments, the tokens can be represented by identification numbers, and an inference model also uses corresponding identification numbers to work with tokens 19 and 19'. These identification numbers identify corresponding tokens in a list of tokens used by the reference model. The token sequence 15 generated by subdivision step 121 therefore includes tokens 19 that correspond to the characters of input string 11, as well as tokens 19' that correspond to the separator sequence 13. However, for comparison, only tokens 19 that correspond to the characters of input string 11 are considered, which is represented by the cleaned token sequence 16.As already explained, this could mean that either the inference model does not take into account the tokens 19' of the separation sequence 13, or that the corresponding tokens 19' are removed before the comparison.

[0050] An example of the second input string 11' is shown in the lower part of the Fig. 2 indicated. As shown in the Fig. As can be seen in Figure 2, the first input string 11 and the second input string 11' differ only in three positions. Specifically, the starting characters differ, with an additional letter inserted in the third position of the second input string 11'. Further processing of the second input string 11' is analogous to that of the first input string 11. Accordingly, a token sequence 16' is generated, containing only tokens corresponding to the characters of the second input string 11'. These token sequences 16 and 16' are then compared using the inference model.

[0051] The Fig. 3 illustrates a process 100', which is based on process 100 of the Fig. 1. In procedure 100', procedure 100 is executed in a first step, providing input strings 11 and 11'. However, procedure 100' includes the additional output step 150, in which a comparison result 18 is output. The comparison result 18 can, for example, include a percentage indicating the similarity of the first and second input strings 11 and 11' with respect to a feature.

[0052] For example, comparison result 18 could indicate that the first and second input strings 11, 11' are similar with respect to auditory features. It is also conceivable that comparison result 18 indicates whether the first and second input strings 11, 11' are similar with respect to visual features. Comparison result 18 could be represented as a percentage, where 0% indicates that the first and second input strings 11, 11' are completely dissimilar and 100% indicates identity. However, it is also conceivable that comparison result 18 could indicate similarity classes. For example, the classes could be dissimilar, somewhat similar, moderately similar, highly similar, and identical. Comparison result 18 could then indicate the corresponding class.

[0053] Output step 150 could, for example, include output to a display device. Furthermore, it is conceivable that output step 150 includes the transmission of the comparison result 18 via a communication network, such as the internet. Such a scenario is described in the Fig. 4 shown.

[0054] The Fig. Figure 4 shows a System 200, which is set up for comparing two input strings 11, 11', and a terminal 400.

[0055] The terminal device 400 is communicatively connected to the system 200 via a communication medium 300. The communication medium 300 can be a communication network 300, such as the internet. It is also conceivable that this is a private network. Likewise, it is conceivable that it is a mobile network or a wired network.

[0056] The terminal device 400 can, for example, be a mobile device, such as a smartphone. It is also conceivable that the terminal device 400 is a mobile computer or a stationary computer. The terminal device 400 includes a terminal device user interface 420, through which the terminal device 400 can receive user input. In the illustrated embodiment, the terminal device 400 receives a first and a second input string 11, 11' via the terminal device user interface 420, which are transmitted to a terminal device communication unit 410. The terminal device communication unit 410 forms the interface between the terminal device 400 and the communication medium 300. In this embodiment, the terminal device communication interface 410 is configured to transmit the first and the second input string 11, 11' to the system 200 via the communication medium 300.System 200 includes a communication unit 210 configured to receive the first and second input strings 11, 11'. The communication unit 210 transmits the received input strings 11, 11' to a processing unit 220, which is configured to insert the separator sequence 13 between the characters of the first and second input strings 11, 11'. This insertion generates corresponding first and second modified input strings 14, 14'. System 200 also includes a tokenization unit 230 configured to receive the modified input strings 14, 14' from the processing unit 220. Furthermore, the tokenization unit 230 is configured to tokenize the first and second modified input strings 14, 14' into a first and second token sequence 15, 15'.By inserting the separator sequences, tokenization is performed such that a token of the first and second token sequence 15, 15' corresponds to a character of the first and second input string 11, 11' or the separator sequence, respectively.

[0057] In the illustrated embodiment of the Fig. 4. The system 200 further comprises a reduction unit 240 configured to receive the token sequences 15, 15' and remove the tokens in the first and second token sequences 15, 15' that correspond to the separation sequence 13. This process generates clean first and second token sequences 16, 16'. In an embodiment not shown, the system 200 does not include the reduction unit 240 and passes the token sequences 15, 15' directly to the comparator unit 250 provided in the system 200.

[0058] The comparator 250 features a transformer-based inference model used to compare the first and second input strings 11, 11'. The inference model can be implemented as a Large Language Model. Depending on the implementation, the inference model uses either the cleaned token sequences 16, 16' or the token sequences 15, 15' to perform the comparison.

[0059] In a further embodiment, it is conceivable that the terminal user interface 420 also receives information indicating whether the comparison should be performed with regard to visual or auditory similarity. This information can be transmitted as a parameter via the transmission medium from the terminal communication unit 410 to the communication unit 210 and made available to the comparison unit 250 as a further parameter. The comparison unit 250 is thus configured to perform the comparison of the corresponding token sequences 15, 15' or 16, 16' and to provide a comparison result 18 to the communication unit 210, which transmits the comparison result 18 to the terminal communication unit 410 so that the comparison result 18 can be output via the terminal user interface 420.

[0060] The Fig.Figure 5 schematically shows a computer system 500, which is configured to implement the procedures and / or systems described above. For this purpose, the computer system 500 includes a processing unit 510, which can be, for example, a microprocessor, such as a CPU. Furthermore, the computer system 500 includes a communication unit 530, which can be, for example, a network interface, cellular module, WLAN, or other communication unit. The processing unit 510 is also communicatively connected to a storage unit 520, which can be configured to store a transformer-based inference model and necessary corpora, such as dictionaries or token lists containing a mapping of characters to identification numbers.The Computer System 500 can also be configured as a distributed system, where, for example, the storage unit 520 can be implemented on a first hardware unit and the processing unit 510 on a second hardware unit. Thus, the Computer System 500 can be implemented in one or more data centers. Reference symbol list: 11, 11' string 13 Separation sequence 14, 14' modified input string 15, 15' Token sequence 16, 16' cleaned token sequence 17 Inference model 18 Comparison result 19, 19' Token 100, 100' procedure 110 Insert step 120 Tokenization Step 121 Subdivision step 130 Removal Step 140 Compare Step 150 Output Step 160 transfer steps 200 System 210 Communication unit 220 processing units 230 tokenization units 240 reduction units 250 comparison units 260 output units 300 communication medium 400 terminal devices 410 End device communication unit 420 Endpoint user interface 500 computer systems 510 processing units 520 storage unit 530 Communication unit QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited non-patent literature

[0000] Attention is all you need”, Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin, 2017, Crurran Associates, ISBN: 9781510860964

[0003]

Claims

[1] System (200) for comparing two strings (11, 11') comprising the following: • a communication unit (210) designed to receive a first and a second input string (11, 11') each specifying a sequence of characters of an alphabet, in particular via an application interface over a network (300), e.g. via the Internet; • a processing unit (220) configured to insert a separator sequence (13) between the characters of the first and second input string (11, 11') to generate a corresponding first and second modified input string (14, 14'), wherein the separator sequence (13) specifies a stochastically rare string; • a tokenization unit (230) configured to tokenize the first and second modified input strings (14, 14') into a first and second token sequence (15, 15') such that a token (19, 19') of the first and second token sequence (15, 15') corresponds to a character of the first and second input strings (11, 11') or of the separator sequence (13), respectively; • a comparison unit (250) trained to compare the first and second input strings (11, 11') using a transformer-based inference model (17) with the first and second token sequence (15, 15'), excluding the tokens (19') corresponding to the separator sequence (13) from the comparison. [2] System (200) according to claim 1, characterized bya storage unit (520) configured to store a reference corpus, wherein the reference corpus specifies a plurality of tokens (19, 19'), in particular token 19, 19') of a natural language. [3] System (200) according to claim 1 or 2, characterized by a reduction unit (240) configured to remove the tokens (19') in the first and second token sequence (15, 15') corresponding to the separation sequence 13) to generate a purified first and a purified second token sequence (16, 16'), wherein the comparison unit (250) is configured to perform the comparison (140) using the purified first and second token sequence (16, 16'), in particular in place of the first and second token sequence (15, 15'). [4] System (200) according to any one of claims 1 to 3, characterized by, that the comparison unit (250) is designed to perform comparisons with respect to auditory and / or written similarity. [5] System (200) according to any one of claims 1 to 4, characterized by , that the tokenization unit (230) is configured to perform a subdivision of the first and second input sequence (11, 11') into tokens (19, 19'), wherein a token (19, 19') comprises at least one, preferably a plurality, of characters of a natural language alphabet. [6] System (200) according to any one of claims 1 to 5, characterized by , that the tokenization unit (230) is configured to divide the first and second modified input strings (14, 14') into one character each of the first and second input strings (11, 11') and the characters of the separator sequence (13). [7] System (200) according to any one of claims 1 to 6, characterized by, that the comparison unit (250) is configured to remove characters of the inserted separator sequences (13) so that the comparison is executable for the characters of the first and second input string (11, 11'). [8] System (200) according to any one of claims 1 to 7, characterized by an output unit (210) which is designed to transmit the comparison result (18) via an application interface, e.g. an API, via a communication network (300), e.g. the Internet, to a user facility (400). [9] Computer-readable storage medium (520) comprising instructions which cause at least one processor (510) to implement the functions of a system according to any of the preceding claims when the instructions are executed by the at least one processor (510).

Images