Method, system, and computer program product for normalizing embeddings for cross-embedding alignment

Abstract

A method for embedding normalization for aligning cross embeddings is provided. The method may include applying mean centering to at least one embedding set, applying spectral normalization to at least one embedding set, and / or applying length normalization to at least one embedding set. Spectral normalization may include decomposing at least one embedding set, determining an average singular value of at least one embedding set, determining a corresponding replacement singular value for each corresponding singular value of a diagonal matrix, and / or replacing at least one embedding set with a product of at least one embedding set, a right singular vector, and an inverse matrix of the replacement diagonal matrix. Mean centering, spectral normalization, and / or length normalization can be iteratively repeated for a configurable number of iterations. A system and computer program product are also disclosed.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 192,779, filed on May 25, 2021, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The disclosed subject matter relates generally to methods, systems, and computer program products for normalizing embeddings and, in some specific embodiments or aspects, to methods, systems, and computer program products for normalizing embeddings for cross-embedding alignment. Background Art

[0004] Cross-embedding alignment (e.g., aligning embedding vectors from two or more different embedding spaces) is useful in a variety of applications. For example, cross-lingual and multilingual natural language processing (NLP) have recently received attention. Learning a transfer function that maps word embedding vectors from multiple languages ​​to a shared cross-lingual word embedding vector space enables multilingual modeling of word meanings and cross-lingual migration of NLP models. Cross-lingual word embedding (CLWE) represents words from two or more languages ​​in a shared cross-lingual vector space, and CLWE is used as a valuable tool for migrating data across different languages, understanding cross-lingual differences, and cross-lingual migration for downstream tasks such as bilingual dictionary induction (BLI), cross-lingual information retrieval, cross-lingual document classification, cross-lingual dependency parsing, and machine translation. Certain CLWE techniques use a strictly orthogonal transformation to map one embedding to another, so that the embeddings are in a shared language space. For example, the Procrustes alignment technique provides a closed-form solution that minimizes the sum of all pairs of Euclidean distances across the alignment between pairs of objects in two embeddings. Preprocessing the embedding vectors before applying alignment techniques (e.g., Procrustes alignment, etc.) can improve the CLWE performance.

[0005] However, limitations of certain preprocessing techniques result in degraded performance (e.g., poor, suboptimal, etc. performance). For example, mean centering does not change the Euclidean distance between any pair of points in a set of embedding vectors, but does change the cosine distance between the pairs of points. Length normalization does not change the cosine distance between any pair of points in a set of embedding vectors, but does change the Euclidean distance between the pairs of points. Principal component analysis (PCA) removal reduces the top principal components or top singular vectors (e.g., makes the top singular values ​​exactly 0), and therefore, PCA may be too blunt. Summary of the invention

[0006] Thus, improved methods, systems, and computer program products are provided for normalizing embeddings for cross-embedding alignment.

[0007] According to non-limiting embodiments or aspects, a method for normalizing cross-embedding alignment embeddings is provided. In some non-limiting embodiments or aspects, a method for normalizing cross-embedding alignment embeddings may include receiving at least one embedding set, each embedding set in the at least one embedding set including a set of embedding vectors. Mean centering can be applied to the at least one embedding set. Spectral normalization can be applied to the at least one embedding set. For example, spectral normalization may include decomposing the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining an average singular value of the at least one embedding set; for each corresponding singular value of the diagonal matrix, determining whether the corresponding singular value is greater than a configurable multiple of the average singular value; for each corresponding singular value of the diagonal matrix, if the corresponding singular value is greater than the configurable multiple of the average singular value, determining a corresponding replacement singular value based on a quotient of the corresponding singular value divided by the configurable multiple of the average singular value, or determining the corresponding replacement singular value to be 1 if the corresponding singular value is not greater than the configurable multiple of the average singular value, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing the at least one embedding set with the product of the at least one embedding set, the right singular vector, and an inverse matrix of the replacement diagonal matrix. Length normalization can be applied to the at least one embedding set.

[0008] In some non-limiting embodiments or aspects, applying mean centering can include determining a mean based on all embedding vectors in the set of embedding vectors. Additionally or alternatively, the mean can be subtracted from each embedding vector in the set of embedding vectors.

[0009] In some non-limiting embodiments or aspects, decomposing the at least one embedding set may include performing a singular value decomposition on the at least one embedding set.

[0010] In some non-limiting embodiments or aspects, determining the average singular value may include determining the square root of the average squared singular value.

[0011] In some non-limiting embodiments or aspects, applying length normalization may include adjusting each embedding vector in the set of embedding vectors to have a 2-norm of 1.

[0012] In some non-limiting embodiments or aspects, applying mean centering, applying spectral normalization, and applying length normalization to the at least one embedding set can be iteratively repeated for a configurable number of iterations.

[0013] In some non-limiting embodiments or aspects, the at least one embedding set can include a first embedding set and a second embedding set. Additionally or alternatively, the first embedding set can be aligned with the second embedding set.

[0014] In some non-limiting embodiments or aspects, the at least one embedding set may include a first language embedding set and a second language embedding set. The first language embedding set may include a first set of word embedding vectors for a first language. The second language embedding set may include a second set of word embedding vectors for a second language.

[0015] In some non-limiting embodiments or aspects, the at least one embedding set may include a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period.

[0016] In some non-limiting embodiments or aspects, the entity may include at least one of a merchant, a customer, an issuer, an acquirer, or a payment gateway.

[0017] According to non-limiting embodiments or aspects, a system for normalizing cross-embedding alignment is provided. In some non-limiting embodiments or aspects, the system for normalizing cross-embedding alignment may include: at least one processor; and at least one non-transitory computer-readable medium including one or more instructions, which, when executed by the at least one processor, direct the at least one processor to receive at least one embedding set. Each embedding set in the at least one embedding set may include a set of embedding vectors. The instructions, when executed by the at least one processor, can further direct the at least one processor to apply mean centering to the at least one embedding set. The instructions, when executed by the at least one processor, can further direct the at least one processor to apply spectral normalization to the at least one embedding set. Applying spectral normalization to the at least one embedding set may include decomposing the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining an average singular value of the at least one embedding set; for each corresponding singular value of the diagonal matrix, determining whether the corresponding singular value is greater than a configurable multiple of the average singular value; for each corresponding singular value of the diagonal matrix, if the corresponding singular value is greater than the configurable multiple of the average singular value, determining a corresponding replacement singular value based on a quotient of the corresponding singular value divided by the configurable multiple of the average singular value, or determining the corresponding replacement singular value to be 1 if the corresponding singular value is not greater than the configurable multiple of the average singular value, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing the at least one embedding set with the product of the at least one embedding set, the right singular vector, and an inverse matrix of the replacement diagonal matrix. The instructions, when executed by the at least one processor, are capable of further directing the at least one processor to apply length normalization to the at least one embedding set.

[0018] In some non-limiting embodiments or aspects, applying mean centering can include determining a mean based on all embedding vectors in the set of embedding vectors and / or subtracting the mean from each embedding vector in the set of embedding vectors.

[0019] In some non-limiting embodiments or aspects, decomposing the at least one embedding set may include performing a singular value decomposition on the at least one embedding set.

[0020] In some non-limiting embodiments or aspects, determining the average singular value may include determining the square root of the average squared singular value.

[0021] In some non-limiting embodiments or aspects, applying length normalization may include adjusting each embedding vector in the set of embedding vectors to have a 2-norm of 1.

[0022] In some non-limiting embodiments or aspects, the instructions, when executed by the at least one processor, can further direct the at least one processor to iteratively repeat applying mean centering, applying spectral normalization, and applying length normalization to the at least one embedding set for a configurable number of iterations.

[0023] In some non-limiting embodiments or aspects, the at least one embedding set may include a first embedding set and a second embedding set. Additionally or alternatively, the instructions, when executed by the at least one processor, may further direct the at least one processor to align the first embedding set with the second embedding set.

[0024] In some non-limiting embodiments or aspects, the at least one embedding set may include a first language embedding set and a second language embedding set. The first language embedding set may include a first set of word embedding vectors for a first language. The second language embedding set may include a second set of word embedding vectors for a second language.

[0025] In some non-limiting embodiments or aspects, the at least one embedding set may include a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period.

[0026] In some non-limiting embodiments or aspects, the entity may include at least one of a merchant, a customer, an issuer, an acquirer, or a payment gateway.

[0027] According to non-limiting embodiments or aspects, a computer program product for embedding normalization of cross-embedding alignment is provided. The computer program product may include at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium including one or more instructions, the one or more instructions, when executed by at least one processor, causing the at least one processor to receive at least one embedding set. Each embedding set in the at least one embedding set may include a set of embedding vectors. The instructions, when executed by the at least one processor, can further cause the at least one processor to apply mean centering to the at least one embedding set. The instructions, when executed by the at least one processor, can further cause the at least one processor to apply spectral normalization to the at least one embedding set. Applying spectral normalization to the at least one embedding set may include decomposing the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining an average singular value of the at least one embedding set; for each corresponding singular value of the diagonal matrix, determining whether the corresponding singular value is greater than a configurable multiple of the average singular value; for each corresponding singular value of the diagonal matrix, if the corresponding singular value is greater than the configurable multiple of the average singular value, determining a corresponding replacement singular value based on a quotient of the corresponding singular value divided by the configurable multiple of the average singular value, or determining the corresponding replacement singular value to be 1 if the corresponding singular value is not greater than the configurable multiple of the average singular value, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing the at least one embedding set with the product of the at least one embedding set, the right singular vector, and an inverse matrix of the replacement diagonal matrix. The instructions, when executed by the at least one processor, are capable of further causing the at least one processor to apply length normalization to the at least one embedding set.

[0028] In some non-limiting embodiments or aspects, applying mean centering can include determining a mean based on all embedding vectors in the set of embedding vectors and / or subtracting the mean from each embedding vector in the set of embedding vectors.

[0029] In some non-limiting embodiments or aspects, decomposing the at least one embedding set may include performing a singular value decomposition on the at least one embedding set.

[0030] In some non-limiting embodiments or aspects, determining the average singular value may include determining the square root of the average squared singular value.

[0031] In some non-limiting embodiments or aspects, applying length normalization may include adjusting each embedding vector in the set of embedding vectors to have a 2-norm of 1.

[0032] In some non-limiting embodiments or aspects, the instructions, when executed by the at least one processor, can further cause the at least one processor to iteratively repeat applying mean centering, applying spectral normalization, and applying length normalization to the at least one embedding set for a configurable number of iterations.

[0033] In some non-limiting embodiments or aspects, the at least one embedding set may include a first embedding set and a second embedding set. Additionally or alternatively, the instructions, when executed by the at least one processor, may further cause the at least one processor to align the first embedding set with the second embedding set.

[0034] In some non-limiting embodiments or aspects, the at least one embedding set may include a first language embedding set and a second language embedding set. The first language embedding set may include a first set of word embedding vectors for a first language. The second language embedding set may include a second set of word embedding vectors for a second language.

[0035] In some non-limiting embodiments or aspects, the at least one embedding set may include a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period.

[0036] In some non-limiting embodiments or aspects, the entity may include at least one of a merchant, a customer, an issuer, an acquirer, or a payment gateway.

[0037] Additional non-limiting embodiments or aspects are set forth in the following numbered clauses:

[0038] Clause 1: A computer-implemented method, the method comprising: receiving, with at least one processor, at least one embedding set, each embedding set in the at least one embedding set comprising a set of embedding vectors; applying, with at least one processor, mean centering to the at least one embedding set; applying, with at least one processor, spectral normalization to the at least one embedding set, wherein applying spectral normalization to the at least one embedding set comprises: decomposing, with at least one processor, the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining, with at least one processor, an average singular value of the at least one embedding set; for each corresponding singular value of the diagonal matrix, determining, with at least one processor, whether the corresponding singular value is greater than a configurable multiple of the average singular value; for the for each corresponding singular value of a diagonal matrix, determining, by at least one processor, a corresponding replacement singular value based on a quotient of the corresponding singular value divided by the configurable multiple of the average singular value if the corresponding singular value is greater than the configurable multiple of the average singular value, or determining, by at least one processor, the corresponding replacement singular value to be 1 if the corresponding singular value is not greater than the configurable multiple of the average singular value, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing, by at least one processor, the at least one embedding set with the product of the at least one embedding set, the right singular vectors, and an inverse matrix of the replacement diagonal matrix; and applying, by at least one processor, length normalization to the at least one embedding set.

[0039] Clause 2: A method as described in Clause 1, wherein applying mean centering includes: determining a mean based on all embedding vectors in the set of embedding vectors using at least one processor; and subtracting the mean from each embedding vector in the set of embedding vectors using at least one processor.

[0040] Clause 3: A method as described in any preceding clause, wherein decomposing the at least one embedding set includes performing a singular value decomposition on the at least one embedding set.

[0041] Clause 4: The method of any preceding clause, wherein determining the mean singular value comprises determining the square root of the mean squared singular value.

[0042] Clause 5: A method as described in any preceding clause, wherein applying length normalization includes: adjusting each embedding vector in the set of embedding vectors to have a 2-norm of 1 with at least one processor.

[0043] Clause 6: A method as described in any preceding clause, further comprising: iteratively repeating, with at least one processor, applying mean centering, applying spectral normalization, and applying length normalization to the at least one embedding set for a configurable number of iterations.

[0044] Clause 7: A method as described in any preceding clause, wherein the at least one embedding set includes a first embedding set and a second embedding set, the method further comprising: aligning the first embedding set with the second embedding set with at least one processor.

[0045] Clause 8: A method as described in any preceding clause, wherein the at least one embedding set includes a first language embedding set and a second language embedding set, wherein the first language embedding set includes a first set of word embedding vectors for a first language and the second language embedding set includes a second set of word embedding vectors for a second language.

[0046] Clause 9: A method as described in any of the preceding clauses, wherein the at least one embedding set includes a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period.

[0047] Clause 10: The method of any preceding clause, wherein the entity comprises at least one of a merchant, a customer, an issuer, an acquirer, or a payment gateway.

[0048] Clause 11: A system, the system comprising: at least one processor; and at least one non-transitory computer-readable medium comprising one or more instructions, the one or more instructions, when executed by the at least one processor, directing the at least one processor to: receive at least one embedding set, each embedding set in the at least one embedding set comprising a set of embedding vectors; apply mean centering to the at least one embedding set; apply spectral normalization to the at least one embedding set, wherein applying spectral normalization to the at least one embedding set comprises: decomposing the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining an average singular value of the at least one embedding set; for each corresponding singular value of the diagonal matrix, determining the corresponding singular value; whether a singular value is greater than a configurable multiple of the average singular value; for each corresponding singular value of the diagonal matrix, if the corresponding singular value is greater than the configurable multiple of the average singular value, determining a corresponding replacement singular value based on the quotient of the corresponding singular value divided by the configurable multiple of the average singular value, or if the corresponding singular value is not greater than the configurable multiple of the average singular value, determining the corresponding replacement singular value to be 1, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing the at least one embedding set with the product of the at least one embedding set, the right singular vector and the inverse matrix of the replacement diagonal matrix; and applying length normalization to the at least one embedding set.

[0049] Clause 12: The system of clause 11, wherein applying mean centering comprises: determining a mean based on all embedding vectors in the set of embedding vectors; and subtracting the mean from each embedding vector in the set of embedding vectors.

[0050] Clause 13: The system of clause 11 or clause 12, wherein decomposing the at least one embedding set comprises performing a singular value decomposition on the at least one embedding set.

[0051] Clause 14: The system of any one of Clauses 11 to 13, wherein determining the average singular value comprises determining a square root of an average squared singular value.

[0052] Clause 15: The system of any one of clauses 11 to 14, wherein applying length normalization comprises: adjusting each embedding vector in the set of embedding vectors to have a 2-norm of 1.

[0053] Clause 16: A system as described in any of clauses 11 to 15, wherein the instructions, when executed by the at least one processor, further direct the at least one processor to: iteratively repeat applying mean centering, applying spectral normalization, and applying length normalization to the at least one embedding set for a configurable number of iterations.

[0054] Clause 17: A system as described in any of clauses 11 to 16, wherein the at least one embedding set includes a first embedding set and a second embedding set, wherein the instruction, when executed by the at least one processor, further directs the at least one processor to: align the first embedding set with the second embedding set.

[0055] Clause 18: A system as described in any of Clauses 11 to 17, wherein the at least one embedding set includes a first language embedding set and a second language embedding set, wherein the first language embedding set includes a first set of word embedding vectors for a first language and the second language embedding set includes a second set of word embedding vectors for a second language.

[0056] Clause 19: A system as described in any of clauses 11 to 18, wherein the at least one embedding set includes a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period, and wherein the entities include at least one of a merchant, a customer, an issuer, an acquirer, or a payment gateway.

[0057] Clause 20: A computer program product, the computer program product comprising at least one non-transitory computer-readable medium, the at least one non-transitory computer-readable medium comprising one or more instructions, the one or more instructions, when executed by at least one processor, causes the at least one processor to: receive at least one embedding set, each embedding set in the at least one embedding set comprising a set of embedding vectors; apply mean centering to the at least one embedding set; apply spectral normalization to the at least one embedding set, wherein applying spectral normalization to the at least one embedding set comprises: decomposing the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining an average singular value of the at least one embedding set; and for each corresponding singular value of the diagonal matrix, determining determining whether the corresponding singular value is greater than a configurable multiple of the average singular value; for each corresponding singular value of the diagonal matrix, determining a corresponding replacement singular value based on a quotient of the corresponding singular value divided by the configurable multiple of the average singular value if the corresponding singular value is greater than the configurable multiple of the average singular value, or determining the corresponding replacement singular value to be 1 if the corresponding singular value is not greater than the configurable multiple of the average singular value, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing the at least one embedding set with the product of the at least one embedding set, the right singular vectors, and an inverse matrix of the replacement diagonal matrix; and applying length normalization to the at least one embedding set.

[0058] Clause 21: The computer program product of clause 20, wherein applying mean centering comprises: determining a mean based on all embedding vectors in the set of embedding vectors; and subtracting the mean from each embedding vector in the set of embedding vectors.

[0059] Clause 22: The computer program product of clause 20 or clause 21, wherein decomposing the at least one embedding set comprises performing a singular value decomposition on the at least one embedding set.

[0060] Clause 23: The computer program product of any of Clauses 20 to 22, wherein determining the average singular value comprises determining a square root of an average squared singular value.

[0061] Clause 24: The computer program product of any of clauses 20 to 23, wherein applying length normalization comprises: adjusting each embedding vector in the set of embedding vectors to have a 2-norm of 1.

[0062] Clause 25: A computer program product as described in any of clauses 20 to 24, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to: iteratively repeat applying mean centering, applying spectral normalization, and applying length normalization to the at least one embedding set for a configurable number of iterations.

[0063] Clause 26: A computer program product as described in any of clauses 20 to 25, wherein the at least one embedding set includes a first embedding set and a second embedding set, wherein the instructions when executed by the at least one processor further cause the at least one processor to: align the first embedding set with the second embedding set.

[0064] Clause 27: A computer program product as described in any of clauses 20 to 26, wherein the at least one embedding set includes a first language embedding set and a second language embedding set, wherein the first language embedding set includes a first set of word embedding vectors for a first language and the second language embedding set includes a second set of word embedding vectors for a second language.

[0065] Clause 28: A computer program product as described in any of Clauses 20 to 27, wherein the at least one embedding set includes a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period.

[0066] Clause 29: The computer program product of any of clauses 20 to 28, wherein the entity comprises at least one of a merchant, a customer, an issuer, an acquirer, or a payment gateway.

[0067] These and other features and characteristics of the presently disclosed subject matter, as well as methods of operation and function of the related structural elements and combinations of parts and economies of manufacturing will become more apparent after considering the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals indicate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for illustration and description purposes only and are not intended as definitions of the limitations of the disclosed subject matter. As used in this specification and claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Additional advantages and details of the disclosed subject matter are explained in more detail below with reference to exemplary embodiments or aspects shown in the accompanying drawings, in which:

[0069] Figure 1Ais a diagram of an exemplary system for normalizing embeddings for cross-embedding alignment according to some non-limiting embodiments or aspects of the presently disclosed subject matter;

[0070] Figure 1B is a diagram of an exemplary environment in which methods, systems, and / or computer program products described herein may be implemented according to some non-limiting embodiments or aspects of the presently disclosed subject matter;

[0071] Figure 2 According to some non-limiting embodiments or aspects of the presently disclosed subject matter Figure 1A and Figure 1B diagrams of exemplary components of one or more devices;

[0072] Figure 3 is a flow chart of an exemplary process for normalizing cross-embedding alignment embeddings according to some non-limiting embodiments or aspects of the presently disclosed subject matter;

[0073] FIG. 4A to FIG. 4C is a diagram illustrating some non-limiting embodiments or aspects of the presently disclosed subject matter. Figure 3 a graph of the performance of an exemplary implementation of the process; and

[0074] Figure 5A and Figure 5B is a diagram illustrating some non-limiting embodiments or aspects of the presently disclosed subject matter. Figure 3 A graph of the performance of an exemplary implementation of the process. DETAILED DESCRIPTION

[0075] For the purpose of description below, the terms "end", "upper", "lower", "right", "left", "vertical", "horizontal", "top", "bottom", "lateral", "longitudinal" and their derivatives shall refer to the disclosed subject matter as it is oriented in the accompanying drawings. However, it should be understood that the disclosed subject matter can adopt various alternative variations and step sequences, unless expressly specified to the contrary. It should also be understood that the specific devices and processes shown in the drawings and described in the following specification are merely exemplary embodiments or aspects of the disclosed subject matter. Therefore, unless otherwise indicated, specific dimensions and other physical characteristics associated with the embodiments or aspects disclosed herein should not be considered as limiting.

[0076] The aspects, components, elements, structures, actions, steps, functions, instructions, etc. used herein should not be understood as critical or necessary unless explicitly described as such. Also, as used herein, the article "one" is intended to include one or more items, and can be used interchangeably with "one or more" and "at least one". In addition, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, a combination of related items and unrelated items, etc.), and can be used interchangeably with "one or more" or "at least one". In the case of wishing only one item, the term "one" or similar language is used. Also, as used herein, the term "having" etc. is intended to be an open term. In addition, unless otherwise explicitly stated, the phrase "based on" is intended to mean "based at least in part on".

[0077] As used herein, the terms "communication" and "transmission" may refer to the reception, acceptance, transmission, migration, provision, etc. of information (e.g., data, signals, messages, instructions, commands, etc.). A unit (e.g., a device, a system, a component of a device or system, a combination thereof, etc.) communicating with another unit means that the unit is able to directly or indirectly receive information from the other unit and / or send information to the other unit. This may refer to a direct or indirect connection (e.g., a direct communication connection, an indirect communication connection, etc.) that is wired and / or wireless in nature. In addition, although the information sent may be modified, processed, relayed, and / or routed between the first unit and the second unit, the two units may also communicate with each other. For example, the first unit may communicate with the second unit even if the first unit passively receives information and does not actively send information to the second unit. As another example, the first unit may communicate with the second unit if at least one intermediate unit (e.g., a third unit located between the first unit and the second unit) processes the information received from the first unit and transmits the processed information to the second unit. In some non-limiting embodiments or aspects, a message may refer to a network packet (eg, a data packet, etc.) that includes data. It should be appreciated that many other arrangements are possible.

[0078] As used herein, the terms "issuer institution," "portable financial device issuer," "issuer," or "issuer bank" may refer to one or more entities that provide accounts to customers for conducting transactions (e.g., payment transactions) (e.g., initiating credit and / or debit payments). For example, an issuer institution may provide customers with account identifiers, such as a primary account number (PAN), that uniquely identify one or more accounts associated with the customer. The account identifier may be implemented on a portable financial device, such as a physical financial instrument (e.g., a payment card), and / or may be electronic and used for electronic payments. The terms "issuer institution" and "issuer institution system" may also refer to one or more computer systems operated by or on behalf of the issuer institution, such as a server computer executing one or more software applications. For example, an issuer institution system may include one or more authorization servers for authorizing transactions.

[0079] As used herein, the term "account identifier" may include one or more types of identifiers associated with a user account (e.g., PAN, card number, payment card number, payment token, etc.). In some non-limiting embodiments or aspects, an issuer institution may provide an account identifier (e.g., PAN, payment token, etc.) to a user, which uniquely identifies one or more accounts associated with the user. The account identifier may be embodied on a physical financial instrument (e.g., a portable financial instrument, a payment card, a credit card, a debit card, etc.), and / or may be electronic information transmitted to a user so that the user can use it for electronic payments. In some non-limiting embodiments or aspects, the account identifier may be an original account identifier, wherein the original account identifier is provided to the user when an account associated with the account identifier is created. In some non-limiting embodiments or aspects, the account identifier may be an account identifier provided to the user after the original account identifier is provided to the user (e.g., a supplemental account identifier). For example, if the original account identifier is forgotten, stolen, etc., the supplemental account identifier may be provided to the user. In some non-limiting embodiments or aspects, the account identifier may be directly or indirectly associated with an issuing institution such that the account identifier may be a payment token mapped to a PAN or other type of identifier. The account identifier may be any combination of alphanumeric characters, characters, and / or symbols, etc. The issuing institution may be associated with a bank identification number (BIN) that uniquely identifies the issuing institution.

[0080] As used herein, the term "payment token" or "token" may refer to an identifier used as a substitute or replacement identifier for an account identifier such as a PAN. A token may be associated with a PAN or other account identifier in one or more data structures (e.g., one or more databases, etc.) so that the token can be used to conduct transactions (e.g., payment transactions) without directly using an account identifier such as a PAN. In some examples, an account identifier such as a PAN may be associated with multiple tokens for different individuals, different uses, and / or different purposes. For example, a payment token may include a series of numbers and / or alphanumeric characters that can be used as a substitute for the original account identifier. For example, a payment token "4900 0000 0000 0001" may be used in place of a PAN "4147 0900 0000 1234". In some non-limiting embodiments or aspects, a payment token may be "format-retained" and may have a digital format consistent with account identifiers used in existing payment processing networks (e.g., an ISO 8583 financial transaction message format). In some non-limiting embodiments or aspects, a payment token may be used in place of a PAN to initiate, authorize, settle, or resolve a payment transaction, or to represent an original credential in other systems where the original credential would normally be provided. In some non-limiting embodiments or aspects, a token value may be generated such that recovery of the original PAN or other account identifier from the token value may not be computationally possible (e.g., using a one-way hash or other cryptographic function). In addition, in some non-limiting embodiments or aspects, a token format may be configured to allow an entity receiving a payment token to identify it as a payment token and to recognize the entity that issued the token.

[0081] As used herein, the term "provision" may refer to the process of enabling a device to use a resource or service. For example, provisioning may involve enabling a device to use an account to perform a transaction. Additionally or alternatively, provisioning may include adding provisioning data associated with account data (e.g., a payment token representing an account number) to the device.

[0082] As used herein, the term "token requester" may refer to an entity that seeks to implement tokenization according to an embodiment or aspect of the presently disclosed subject matter. For example, a token requester may initiate a request to tokenize a PAN by submitting a token request message to a token service provider. Additionally or alternatively, once the requester has received a payment token in response to the token request message, the token requester may no longer need to store the PAN associated with the token. In some non-limiting embodiments or aspects, the requester may be an application, device, process, or system configured to perform actions associated with a token. For example, the requester may request to register with a network token system, request token generation, token activation, token deactivation, token exchange, other token lifecycle management related processes, and / or any other token related processes. In some non-limiting embodiments or aspects, the requester may connect to the network token system via any suitable communication network and / or protocol (e.g., using HTTPS, SOAP, and / or XML interfaces, etc.). For example, token requesters may include card-on-file merchants, acquirers, acquirer processors, payment gateways operated on behalf of merchants, payment enablers (e.g., original equipment manufacturers, mobile network operators, etc.), digital wallet providers, issuers, third-party wallet providers, payment processing networks, etc. In some non-limiting embodiments or aspects, token requesters may request tokens for multiple domains and / or channels. Additionally or alternatively, a token service provider within a tokenization ecosystem may uniquely register and identify token requesters. For example, during token requester registration, a token service provider may formally process the token requester's application to participate in the token service system. In some non-limiting embodiments or aspects, a token service provider may collect information about the nature of the requester and the relevant use of the token to verify and formally approve the token requester and establish appropriate domain restriction controls. Additionally or alternatively, a token requester identifier may be assigned to a successfully registered token requester, which may also be entered and maintained within a token library. In some non-limiting embodiments or aspects, a token requester identifier may be revoked, and / or a new token requester identifier may be assigned to a token requester. In some non-limiting embodiments or aspects, this information may be reported and audited by the token service provider.

[0083] As used herein, the term "token service provider" may refer to an entity including one or more server computers in a token service system that generates, processes and maintains payment tokens. For example, a token service provider may include a token library that stores generated tokens or communicates with the token library. Additionally or alternatively, the token library may maintain a one-to-one mapping between a token and a PAN represented by the token. In some non-limiting embodiments or aspects, the token service provider is able to reserve an authorized BIN as a token BIN to issue a token that can be submitted to the PAN of the token service provider. In some non-limiting embodiments or aspects, various entities of the tokenization ecosystem may assume the role of a token service provider. For example, a payment network and an issuer or its agent may become a token service provider by implementing a token service according to a non-limiting embodiment or aspect of the subject matter of the present disclosure. Additionally or alternatively, the token service provider may provide reports or data outputs to a reporting tool regarding approved, pending or rejected token requests, including any assigned token requester IDs. The token service provider may provide data outputs associated with token-based transactions to reporting tools and applications, and present tokens and / or PANs in the report outputs as needed. In some non-limiting embodiments or aspects, the EMVCo standards organization may publish specifications that define how a tokenization system may operate. For example, such specifications may be informational, but they are not intended to limit any currently disclosed subject matter.

[0084] As used herein, the term "token repository" may refer to a repository that maintains established token-to-PAN mappings. For example, the token repository may also maintain other attributes of the token requester that may be determined at registration, and / or may be used by the token service provider to apply domain restrictions or other controls during transaction processing. In some non-limiting embodiments or aspects, the token repository may be part of a token service system. For example, the token repository may be provided as part of a token service provider. Additionally or alternatively, the token repository may be a remote repository accessible to the token service provider. In some non-limiting embodiments or aspects, the token repository may be protected by strong underlying physical and logical security due to the sensitive nature of the data mappings stored and managed therein. Additionally or alternatively, the token repository may be operated by any suitable entity, including a payment network, an issuer, a clearing house, other financial institutions, a transaction service provider, and the like.

[0085] As used herein, the term "merchant" may refer to one or more entities (e.g., operators of retail businesses that provide goods and / or services and / or access to goods and / or services to users (e.g., customers, consumers, customers of merchants, etc.) based on transactions (e.g., payment transactions). As used herein, the term "merchant system" may refer to one or more computer systems operated by or on behalf of a merchant, such as a server computer that executes one or more software applications. As used herein, the term "product" may refer to one or more goods and / or services provided by a merchant.

[0086] As used herein, a "point of sale (POS) device" may refer to one or more devices that may be used by a merchant to initiate a transaction (e.g., a payment transaction), participate in a transaction, and / or process a transaction. For example, a POS device may include one or more computers, peripheral devices, card readers, near field communication (NFC) receivers, radio frequency identification (RFID) receivers and / or other contactless transceivers or receivers, contact-based receivers, payment terminals, computers, servers, input devices, etc.

[0087] As used herein, a "point of sale (POS) system" may refer to one or more computers and / or peripheral devices that a merchant uses to conduct transactions. For example, a POS system may include one or more POS devices, and / or other similar devices that can be used to conduct payment transactions. A POS system (e.g., a merchant POS system) may also include one or more server computers that are programmed or configured to process online payment transactions via a web page, mobile application, or the like.

[0088] As used herein, the term "transaction service provider" may refer to an entity that receives transaction authorization requests from merchants or other entities and, in some cases, provides payment guarantees through an agreement between the transaction service provider and an issuer organization. In some non-limiting embodiments or aspects, the transaction service provider may include a credit card company, a debit card company, etc. As used herein, the term "transaction service provider system" may also refer to one or more computer systems operated by or on behalf of the transaction service provider, such as a transaction processing server that executes one or more software applications. The transaction processing server may include one or more processors and, in some non-limiting embodiments or aspects, may be operated by or on behalf of the transaction service provider.

[0089] As used herein, the term "acquirer" may refer to an entity that is licensed by a transaction service provider and approved by the transaction service provider to initiate a transaction (e.g., a payment transaction) using a portable financial device associated with the transaction service provider. As used herein, the term "acquirer system" may also refer to one or more computer systems, computer devices, etc. operated by or on behalf of an acquirer. Transactions may include payment transactions (e.g., purchases, original credit transactions (OCTs), account funding transactions (AFTs), etc.). In some non-limiting embodiments or aspects, an acquirer may be authorized by a transaction service provider to sign a contract with a merchant or service provider to initiate a transaction using a portable financial device of a transaction service provider. An acquirer may sign a contract with a payment service provider so that the payment service provider can provide initiation to a merchant. An acquirer may monitor the compliance of a payment service provider according to transaction service provider regulations. An acquirer may perform due diligence on a payment service provider and ensure that appropriate due diligence is performed before signing a contract with an initiated merchant. An acquirer may be responsible for all transaction service provider plans operated or initiated by an acquirer. The acquirer may be responsible for the actions of the acquirer payment service provider, merchants initiated by the acquirer payment service provider, etc. In some non-limiting embodiments or aspects, the acquirer may be a financial institution, such as a bank.

[0090] As used herein, the terms "electronic wallet," "electronic wallet mobile application," and "digital wallet" may refer to one or more electronic devices and / or one or more software applications configured to initiate and / or conduct transactions (e.g., payment transactions, electronic payment transactions, etc.). For example, an electronic wallet may include an application executed by a user device (e.g., a mobile device) for maintaining and providing transaction data to the user device and server-side software and / or databases. As used herein, the term "electronic wallet provider" may include an entity that provides and / or maintains an electronic wallet and / or electronic wallet mobile application for a user (e.g., a customer). Examples of electronic wallet providers include, but are not limited to, Google Android Apple and Samsung In some non-limiting examples, a financial institution (e.g., an issuer institution) may be an e-wallet provider. As used herein, the term "e-wallet provider system" may refer to one or more computer systems, computer devices, servers, server groups, etc. operated by or on behalf of an e-wallet provider.

[0091] As used herein, the term "portable financial device" may refer to a payment device, an electronic payment device, a payment card (e.g., a credit or debit card), a gift card, a smart card, smart media, a payroll card, a healthcare card, a wristband, a machine-readable medium containing account information, a key chain device or fob, an RFID transponder, a retailer discount or membership card, a cellular telephone, an electronic wallet mobile application, a personal digital assistant (PDA), a pager, a security card, a computer, an access card, a wireless terminal, a transponder, etc. In some non-limiting embodiments or aspects, a portable financial device may include volatile or non-volatile memory to store information (e.g., an account identifier, the name of an account holder, etc.).

[0092] As used herein, the term "payment gateway" may refer to an entity and / or a payment processing system operated by or on behalf of such an entity, which (e.g., a merchant service provider, a payment service provider, a payment service provider with a contract with an acquirer, a payment aggregator, etc.) provides payment services (e.g., transaction service provider payment services, payment processing services, etc.) to one or more merchants. The payment service may be associated with the use of a portable financial device managed by a transaction service provider. As used herein, the term "payment gateway system" may refer to one or more computer systems, computer devices, servers, server groups, etc. operated by or on behalf of a payment gateway, and / or the payment gateway itself. As used herein, the term "payment gateway mobile application" may refer to one or more electronic devices and / or one or more software applications configured to provide payment services for a transaction (e.g., a payment transaction, an electronic payment transaction, etc.).

[0093] As used herein, the terms "client" and "client device" may refer to one or more client-side devices or systems (e.g., at a remote location of a transaction service provider) for initiating or facilitating a transaction (e.g., a payment transaction). As an example, a "client device" may refer to one or more POS devices used by a merchant, one or more acquirer host computers used by an acquirer, one or more mobile devices used by a user, etc. In some non-limiting embodiments or aspects, a client device may be an electronic device configured to communicate with one or more networks and initiate or facilitate a transaction. For example, a client device may include one or more computers, portable computers, laptop computers, tablet computers, mobile devices, cellular phones, wearable devices (e.g., watches, glasses, lenses, clothing, etc.), PDAs, etc. In addition, a "client" may also refer to an entity (e.g., a merchant, an acquirer, etc.) that owns, utilizes, and / or operates a client device for initiating a transaction (e.g., for initiating a transaction with a transaction service provider).

[0094] As used herein, the term "computing device" may refer to one or more electronic devices configured to communicate directly or indirectly with or on one or more networks. The computing device may be a mobile device, a desktop computer, and / or any other similar device. In addition, the term "computer" may refer to any computing device including necessary components for receiving, processing, and outputting data and typically including a display, a processor, a memory, an input device, and a network interface. As used herein, the term "server" may refer to or include one or more processors or computers, storage devices, or similar computer arrangements that operate or facilitate the communication and processing in a network environment such as the Internet, but it should be understood that communication may be facilitated through one or more public or private network environments, and various other arrangements are also possible. In addition, multiple computers (e.g., servers) or other computerized devices (such as POS devices) that communicate directly or indirectly in a network environment may constitute a "system" such as a merchant's POS system.

[0095] As used herein, the term "processor" may refer to any type of processing unit, such as a single processor having one or more cores, one or more cores of one or more processors, multiple processors each having one or more cores, and / or other arrangements and combinations of processing units.

[0096] As used herein, the term "system" may refer to one or more computing devices or combinations of computing devices (e.g., processors, servers, client devices, software applications, components of these computing devices, etc.). References to "devices," "servers," "processors," and the like as used herein may refer to a previously stated device, server, or processor stated as performing a previous step or function, a different server or processor, and / or a combination of servers and / or processors. For example, as used in the specification and claims, a first server or first processor stated as performing a first step or first function may refer to the same or different server or the same or different processor stated as performing a second step or second function.

[0097] Non-limiting embodiments or aspects of the disclosed subject matter relate to systems, methods, and computer program products for normalizing embeddings, including but not limited to normalizing embeddings for cross-embedding alignment. For example, non-limiting embodiments or aspects of the disclosed subject matter provide a new preprocessing technique: spectral normalization. Spectral normalization can include decomposing an embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix, determining an average singular value of at least one embedding set, determining a corresponding alternative singular value for each corresponding singular value of the diagonal matrix based on a configurable (e.g., adjustable) hyperparameter, and replacing the embedding set based on the alternative diagonal matrix. In addition, non-limiting embodiments or aspects of the disclosed subject matter enable mean centering, spectral normalization, and length normalization to be applied iteratively based on a configurable (e.g., adjustable) hyperparameter. Such embodiments provide techniques and systems that provide improved performance (e.g., increased F1 scores) for cross-embedding alignment and downstream tasks (e.g., bilingual dictionary induction (BLI), cross-language document classification (CLDC), etc.). Additionally or alternatively, such embodiments provide techniques and systems for preprocessing an embedding set that improves spectral properties, including reduced condition numbers, increased numerical rank, and reduced joint condition numbers. Additionally or alternatively, such embodiments provide techniques and systems that allow for gentle adjustment of spectral properties of an embedding set (e.g., without abruptly removing singular values ​​and / or forcing metrics such as condition numbers to be infinite). Additionally or alternatively, such embodiments provide techniques and systems that can preprocess an embedding set that is agnostic to the alignment method subsequently used and can therefore be applied in conjunction with any alignment method. Additionally or alternatively, such embodiments provide techniques and systems for embedding sets that can be applied in a variety of contexts, including cross-lingual alignment, mappings between embeddings representing the same entity in two different time periods (e.g., because the embedding space will be different due to different data between the two time periods), merchant classification, fraud detection, restaurant recommendations, product recommendations, and the like.

[0098] For purposes of illustration, in the following description, although the presently disclosed subject matter is described with respect to methods, systems, and computer program products for normalizing word embeddings (e.g., for cross-language alignment), those skilled in the art will recognize that the disclosed subject matter is not limited to the illustrative embodiments or aspects. For example, the methods, systems, and computer program products described herein can be used with a variety of settings, such as normalizing embeddings in any setting suitable for using such embeddings, for example, mappings between embeddings representing the same entity in two different time periods (e.g., because the embedding space will be different due to different data between the two time periods), merchant classification, fraud detection, restaurant recommendations, product recommendations, and the like.

[0099] Reference now Figure 1A , Figure 1Ais a diagram of an exemplary system 100a for normalizing cross-embedding alignment according to some non-limiting embodiments or aspects. Figure 1A As shown, system 100a includes an embedding normalization / alignment system 102a, an embedding database 102b, and / or a requesting system 106a.

[0100] The embedded normalization / alignment system 102a may include one or more devices capable of receiving information from and / or transmitting information to the embedded database 102b and / or the request system 106a. For example, the embedded normalization / alignment system 102a may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the embedded normalization / alignment system 102a may communicate with a data storage device (e.g., the embedded database 102b, another data storage device separated from the embedded database 102b, any combination thereof, etc.), which may be local or remote to the embedded normalization / alignment system 102a. In some non-limiting embodiments or aspects, the embedded normalization / alignment system 102 may be capable of receiving information from the data storage device, storing information in the data storage device, transmitting information to the data storage device, or searching for information stored in the data storage device.

[0101] The embedded database 102b may include one or more devices capable of receiving information from the embedded normalization / alignment system 102a and / or the request system 106a and / or transmitting information to the embedded normalization / alignment system and / or the request system. For example, the embedded database 102b may include a computing device, such as a server, a server group and / or other similar devices. In some non-limiting embodiments or aspects, the embedded database 102b may include a data storage device. In some non-limiting embodiments or aspects, the embedded database 102b may be a part of the same system as the embedded normalization / alignment system 102a (for example, the embedded database 102b may be a part of the embedded normalization / alignment system 102a, a part of another system that also includes the embedded normalization / alignment system 102a, etc.). In some non-limiting embodiments or aspects, the embedded database 102b may be separated from the embedded normalization / alignment system 102a.

[0102] The request system 106a may include one or more devices capable of receiving information from and / or transmitting information to the embedded normalization / alignment system 102a and / or the embedded database 102b. For example, the request system 106a may include a computing device, such as a computer, a portable computer, a mobile device, a client device, a server, a server group, etc. In some non-limiting embodiments or aspects, the request system 106a may be part of the same system as the embedded normalization / alignment system 102a (e.g., the request system 106a may be part of the embedded normalization / alignment system 102a, part of another system that also includes the embedded normalization / alignment system 102a, etc.). In some non-limiting embodiments or aspects, the request system 106a may be separate from the embedded normalization / alignment system 102a. In some non-limiting embodiments or aspects, the request system 106a can be part of the same system as the embedded database 102b (e.g., the request system 106a can be part of the embedded normalization / alignment system 102a that also includes the embedded database 102b, part of another system that includes the request system 106a and the embedded database 102b, etc.). In some non-limiting embodiments or aspects, the request system 106a can be separate from the embedded normalization / alignment system 102a.

[0103] Figure 1A The number and arrangement of systems and / or devices shown are provided as examples. There may be additional systems and / or devices; fewer systems and / or devices; different systems and / or devices; and / or Figure 1A In addition, the different systems and / or devices shown can be implemented in a single system and / or device. Figure 1A Two or more systems or devices shown, or Figure 1A The single system or device shown may be implemented as multiple distributed systems or devices. Additionally or alternatively, a group of systems (e.g., one or more systems) or a group of devices (e.g., one or more devices) of the system 100a may perform one or more functions described as being performed by another group of systems or another group of devices of the system 100a.

[0104] Reference now Figure 1B , Figure 1B 1 is an exemplary environment 100b in which methods, systems, and / or computer program products as described herein may be implemented according to some non-limiting embodiments or aspects. Figure 1B As shown, environment 100 b includes transaction service provider system 102 , embedding normalization / alignment system 102 a , issuer system 104 , client device 106 , merchant system 108 , acquirer system 110 , and / or communication network 112 .

[0105] In some non-limiting embodiments or aspects, the embedded normalization / alignment system 102a can be used with the above reference Figure 1A In addition or alternatively, the embedded normalization / alignment system 102a may be capable of receiving information from and / or transmitting information to the transaction service provider system 102, the issuer system 104, the client device 106, the merchant system 108, and / or the acquirer system 110 (e.g., via the communication network 112). In some non-limiting embodiments or aspects, the embedded normalization / alignment system 102a may be part of the same system as the transaction service provider system 102 (e.g., the embedded normalization / alignment system 102a may be part of the transaction service provider system 102, part of another system that also includes the transaction service provider system 102, etc.). In some non-limiting embodiments or aspects, as described above with reference to Figure 1A As described, the embedded database 102b can be part of the same system as the embedded normalization / alignment system 102a (e.g., the embedded database 102b can be part of the embedded normalization / alignment system 102a, part of another system (such as the transaction service provider system 102) that also includes the embedded normalization / alignment system 102a, etc.). In some non-limiting embodiments or aspects, as described above with reference to Figure 1A As described, the request system 106a can be part of the same system as the embedded normalization / alignment system 102a (e.g., the embedded database 102b can be part of the embedded normalization / alignment system 102a, part of another system (such as the transaction service provider system 102) that also includes the embedded normalization / alignment system 102a, etc.). In some non-limiting embodiments or aspects, as described above with reference to Figure 1A As described, the requesting system 106a can be the same as, similar to, and / or a part of another system, another device, another group of systems, or another group of devices, separate from or including the embedded normalization / alignment system 102a, such as the issuer system 104 (e.g., one or more devices of the issuer system 104), the customer device 106, the merchant system 108 (e.g., one or more devices of the merchant system 108), the acquirer system 110 (e.g., one or more devices of the acquirer system 110), etc.

[0106] The transaction service provider system 102 may include one or more devices capable of receiving information from and / or transmitting information to the embedded normalization / alignment system 102a, the issuer system 104, the client device 106, the merchant system 108, and / or the acquirer system 110 via the communication network 112. For example, the transaction service provider system 102 may include a computing device, such as a server (e.g., a transaction processing server), a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the transaction service provider system 102 may be associated with a transaction service provider described herein. In some non-limiting embodiments or aspects, the transaction service provider system 102 may communicate with a data storage device, which may be local or remote to the transaction service provider system 102. In some non-limiting embodiments or aspects, the transaction service provider system 102 is capable of receiving information from the data storage device, storing information in the data storage device, transmitting information to the data storage device, or searching for information stored in the data storage device.

[0107] The issuer system 104 may include one or more devices capable of receiving information via the communication network 112 and / or transmitting information to the transaction service provider system 102, the embedded normalization / alignment system 102a, the client device 106, the merchant system 108, and / or the acquirer system 110. For example, the issuer system 104 may include a computing device, such as a server, a server group, and / or other similar devices. In some non-limiting embodiments or aspects, the issuer system 104 may be associated with an issuer organization described herein. For example, the issuer system 104 may be associated with an issuer organization that issues credit accounts, debit accounts, credit cards, debit cards, and / or the like to users associated with the client device 106.

[0108] The client devices 106 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 102, the embedded normalization / alignment system 102a, the issuer system 104, the merchant system 108, and / or the acquirer system 110 via the communication network 112. Additionally or alternatively, each client device 106 may include a device capable of receiving information from and / or transmitting information to other client devices 106 via the communication network 112, another network (e.g., a temporary network, a local network, a private network, a virtual private network, etc.), and / or any other suitable communication technology. For example, the client devices 106 may include client devices, etc. In some non-limiting embodiments or aspects, the client devices 106 may or may not be capable of communicating via a short-range wireless communication connection (e.g., an NFC communication connection, an RFID communication connection, Communication connection, communication connection, etc.) (e.g., from merchant system 108 or from another client device 106), and / or transmit information (e.g., to merchant system 108) via a short-range wireless communication connection.

[0109] The merchant system 108 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 102, the embedded normalization / alignment system 102a, the issuer system 104, the client device 106, and / or the acquirer system 110 via the communication network 112. The merchant system 108 may also include a communication connection (e.g., an NFC communication connection, an RFID communication connection, a NFC communication connection, a ... Communication connection, In some non-limiting embodiments or aspects, the merchant system 108 may include a computing device, such as a server, a server group, a client device, a client device group, and / or other similar devices. In some non-limiting embodiments or aspects, the merchant system 108 may be associated with the merchant described herein. In some non-limiting embodiments or aspects, the merchant system 108 may include one or more client devices. For example, the merchant system 108 may include a client device that allows the merchant to transmit information to the transaction service provider system 102. In some non-limiting embodiments or aspects, the merchant system 108 may include one or more devices that can be used by merchants to conduct payment transactions with users, such as computers, computer systems, and / or peripheral devices. For example, the merchant system 108 may include a POS device and / or a POS system.

[0110] The acquirer system 110 may include one or more devices capable of receiving information from and / or transmitting information to the transaction service provider system 102, the embedded normalization / alignment system 102a, the issuer system 104, the client device 106, and / or the merchant system 108 via the communication network 112. For example, the acquirer system 110 may include a computing device, a server, a server group, etc. In some non-limiting embodiments or aspects, the acquirer system 110 may be associated with an acquirer described herein.

[0111] The communication network 112 may include one or more wired and / or wireless networks. For example, the communication network 112 may include a cellular network (e.g., Long Term Evolution). networks, third generation (3G) networks, fourth generation (4G) networks, fifth generation (5G) networks, code division multiple access (CDMA) networks, etc.), public land mobile networks (PLMNs), local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), telephone networks (e.g., public switched telephone networks (PSTNs)), private networks (e.g., private networks associated with transaction service providers), ad hoc networks, intranets, the Internet, fiber-optic-based networks, cloud computing networks, etc., and / or combinations of these or other types of networks.

[0112] In some non-limiting embodiments or aspects, processing a transaction may include generating and / or transmitting at least one transaction message (e.g., an authorization request, an authorization response, any combination thereof, etc.). For example, a client device (e.g., a client device 106, a POS device of a merchant system 108, etc.) may initiate a transaction, for example, by generating an authorization request. Additionally or alternatively, a client device (e.g., a client device 106, at least one device of a merchant system 108, etc.) may transmit an authorization request. For example, the client device 106 may transmit an authorization request to a merchant system 108 and / or a payment gateway (e.g., a payment gateway of a transaction service provider system 102, a third-party payment gateway separated from the transaction service provider system 102, etc.). Additionally or alternatively, the merchant system 108 (e.g., its POS device) may transmit an authorization request to an acquirer system 110 and / or a payment gateway. In some non-limiting embodiments or aspects, the acquirer system 110 and / or the payment gateway may transmit an authorization request to the transaction service provider system 102 and / or the issuer system 104. Additionally or alternatively, the transaction service provider system 102 may transmit the authorization request to the issuer system 104. In some non-limiting embodiments or aspects, the issuer system 104 may determine an authorization decision (e.g., authorization, rejection, etc.) based on the authorization request. For example, the authorization request may cause the issuer system 104 to determine an authorization decision based on the authorization request. In some non-limiting embodiments or aspects, the issuer system 104 may generate an authorization response based on the authorization decision. Additionally or alternatively, the issuer system 104 may transmit the authorization response. For example, the issuer system 104 may transmit the authorization response to the transaction service provider system 102 and / or the payment gateway. Additionally or alternatively, the transaction service provider system 102 and / or the payment gateway may transmit the authorization response to the acquirer system 110, the merchant system 108, and / or the client device 106. Additionally or alternatively, the acquirer system 110 may transmit the authorization response to the merchant system 108 and / or the payment gateway. Additionally or alternatively, the payment gateway may transmit the authorization response to the merchant system 108 and / or the client device 106. Additionally or alternatively, the merchant system 108 can transmit the authorization response to the client device 106. In some non-limiting embodiments or aspects, the merchant system 108 can receive the authorization response (e.g., from the acquirer system 110 and / or the payment gateway). Additionally or alternatively, the merchant system 108 can complete the transaction based on the authorization response (e.g., provide, ship, and / or deliver goods and / or services associated with the transaction; fulfill an order associated with the transaction; any combination thereof; etc.).

[0113] For purposes of illustration, processing a transaction may include generating a transaction message (e.g., an authorization request, etc.) based on an account identifier of a customer (e.g., associated with a customer device 106, etc.) and / or transaction data associated with the transaction. For example, a merchant system 108 (e.g., a client device of the merchant system 108, a POS device of the merchant system 108, etc.) may initiate a transaction, for example, by generating an authorization request (e.g., in response to receiving an account identifier from a portable financial device, etc. of the customer). Additionally or alternatively, the merchant system 108 may transmit the authorization request to an acquirer system 110. Additionally or alternatively, the acquirer system 110 may transmit the authorization request to the transaction service provider system 102. Additionally or alternatively, the transaction service provider system 102 may transmit the authorization request to the issuer system 104. The issuer system 104 may determine an authorization decision (e.g., authorization, rejection, etc.) based on the authorization request, and / or the issuer system 104 may generate an authorization response based on the authorization decision and / or the authorization request. Additionally or alternatively, the issuer system 104 may transmit the authorization response to the transaction service provider system 102. Additionally or alternatively, the transaction service provider system 102 may communicate the authorization response to the acquirer system 110 , which may communicate the authorization response to the merchant system 108 .

[0114] For purposes of illustration, clearing and / or settlement of a transaction may include generating a message (e.g., a clearing message, a settlement message, etc.) based on an account identifier of a customer (e.g., associated with a customer device 106, etc.) and / or transaction data associated with the transaction. For example, the merchant system 108 may generate at least one clearing message (e.g., a plurality of clearing messages, a batch of clearing messages, etc.). Additionally or alternatively, the merchant system 108 may transmit the clearing message to the acquirer system 110. Additionally or alternatively, the acquirer system 110 may transmit the clearing message to the transaction service provider system 102. Additionally or alternatively, the transaction service provider system 102 may transmit the clearing message to the issuer system 104. Additionally or alternatively, the issuer system 104 may generate at least one settlement message based on the clearing message. Additionally or alternatively, the issuer system 104 may transmit the settlement message and / or funds to the transaction service provider system 102 (and / or a settlement bank system associated with the transaction service provider system 102). Additionally or alternatively, the transaction service provider system 102 (and / or the settlement bank system) may transmit the settlement message and / or funds to the acquirer system 110, which may transmit the settlement message and / or funds to the merchant system 108 (and / or an account associated with the merchant system 108).

[0115] Provided as an example Figure 1BThe number and arrangement of systems, devices, and / or networks shown are not intended to be construed as a whole. Additional systems, devices, and / or networks, fewer systems, devices, and / or networks, different systems, devices, and / or networks, and / or networks may exist in a similar manner. Figure 1B The systems, devices and / or networks shown in the figure may be arranged in different ways. In addition, the system and / or device may be implemented in a single system and / or device. Figure 1B Two or more systems or devices shown, or Figure 1B The single system or device shown may be implemented as multiple distributed systems or devices. Additionally or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of environment 100b may perform one or more functions described as being performed by another set of systems or another set of devices of environment 100b.

[0116] Reference now Figure 2 , Figure 2 is a diagram of exemplary components of an apparatus 200. The apparatus 200 may correspond to one or more apparatuses of the transaction service provider system 102, the embedded normalization / alignment system 102a, the embedded database 102b, one or more apparatuses of the issuer system 104, the client device 106, the request system 106a, one or more apparatuses of the merchant system 108, and / or one or more apparatuses of the acquirer system 110. In some non-limiting embodiments or aspects, the transaction service provider system 102, the embedded normalization / alignment system 102a, the embedded database 102b, the issuer system 104, the client device 106, the request system 106a, the merchant system 108, and / or the acquirer system 110 may include at least one apparatus 200 and / or at least one component of the apparatus 200. Figure 2 As shown, apparatus 200 may include a bus 202 , a processor 204 , a memory 206 , a storage component 208 , an input component 210 , an output component 212 , and a communication interface 214 .

[0117] The bus 202 may include components that permit communication between components of the device 200. In some non-limiting embodiments or aspects, the processor 204 may be implemented in hardware, software, firmware, and / or any combination thereof. For example, the processor 204 may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc.), a microprocessor, a digital signal processor (DSP), and / or any processing component that can be programmed to perform a function (e.g., a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), etc. The memory 206 may include a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, optical memory, etc.) that stores information and / or instructions for use by the processor 204.

[0118] The storage component 208 may store information and / or software associated with the operation and use of the device 200. For example, the storage component 208 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, a solid-state disk, etc.), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a tape, and / or another type of computer-readable medium, and a corresponding drive.

[0119] Input components 210 may include components that permit device 200 to receive information, for example, via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, a microphone, a camera, etc.). Additionally or alternatively, input components 210 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, an actuator, etc.). Output components 212 may include components that provide output information from device 200 (e.g., a display, a speaker, one or more light emitting diodes (LEDs), etc.).

[0120] The communication interface 214 may include a transceiver-type component (e.g., a transceiver, a separate receiver and transmitter, etc.) that enables the device 200 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 214 may permit the device 200 to receive information from another device and / or provide information to another device. For example, the communication interface 214 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, interface, interface, Interface, cellular network interface, etc.

[0121] The device 200 can perform one or more processes described herein. The device 200 can perform these processes based on the processor 204 executing software instructions stored by a computer-readable medium such as the memory 206 and / or the storage component 208. Computer-readable media (e.g., non-transitory computer-readable media) are defined herein as non-transitory memory devices. Non-transitory memory devices include memory space located within a single physical storage device or memory space spread across multiple physical storage devices.

[0122] The software instructions may be read into the memory 206 and / or storage component 208 from another computer-readable medium or from another device via the communication interface 214. When executed, the software instructions stored in the memory 206 and / or storage component 208 may cause the processor 204 to perform one or more processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, the embodiments or aspects described herein are not limited to any specific combination of hardware circuitry and software.

[0123] Provided as an example Figure 2 In some non-limiting embodiments or aspects, Figure 2 Compared to those shown in , device 200 may include additional components, fewer components, different components, or components arranged in a different manner. Additionally or alternatively, a set of components (e.g., one or more components) of device 200 may perform one or more functions described as being performed by another set of components of device 200.

[0124] Reference now Figure 3 , Figure 3 is a flow chart of an exemplary process 300 for embedding normalization for cross-embedding alignment according to some non-limiting embodiments or aspects. In some non-limiting embodiments or aspects, one or more steps of process 300 may be performed (e.g., in full, in part, and / or similarly) by embedding normalization / alignment system 102a and / or transaction service provider system 102 (e.g., one or more devices of transaction service provider system 102). In some non-limiting embodiments or aspects, one or more steps of process 300 may be performed (e.g., completely, partially, and / or similarly) by another system, another device, another group of systems, or another group of devices that is separate from or includes the embedded normalization / alignment system 102a and / or the transaction service provider system 102, such as an embedded database 102b, an issuer system 104 (e.g., one or more devices of the issuer system 104), a client device 106, a requesting system 106a (e.g., one or more devices of the requesting system 106a), a merchant system 108 (e.g., one or more devices of the merchant system 108), an acquirer system 110 (e.g., one or more devices of the acquirer system 110), a device 200, a computing device, a server, etc.

[0125] like Figure 3As shown, at step 302, process 300 may include receiving at least one embedding set. For example, embedding normalization / alignment system 102a (e.g., a server, a portion of transaction service provider system 102, a portion of a third-party system, etc.) may receive at least one embedding set. In some non-limiting embodiments or aspects, embedding normalization / alignment system 102a may receive at least one embedding set from at least one of embedding database 102b and / or requesting system 106a. For example, embedding database 102b may receive at least one embedding set from requesting system 106a, and / or embedding normalization / alignment system 102a may receive at least one embedding set from embedding database 102b. Additionally or alternatively, embedding normalization / alignment system 102a may receive at least one embedding set from requesting system 106a.

[0126] In some non-limiting embodiments or aspects, each embedding set may include a set of embedding vectors.

[0127] In some non-limiting embodiments or aspects, the at least one embedding set may include a first language embedding set and a second language embedding set. The first language embedding set may include a first set of word embedding vectors for a first language. Additionally or alternatively, the second language embedding set may include a second set of word embedding vectors for a second language.

[0128] In some non-limiting embodiments or aspects, the at least one embedding set may include a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period. In some non-limiting embodiments or aspects, the entity may include at least one of a merchant, a customer (e.g., a cardholder), an issuer, an acquirer, or a payment gateway.

[0129] like Figure 3 As shown, process 300 may include applying mean centering at step 304. For example, embedding normalization / alignment system 102a (e.g., a server, a portion of transaction service provider system 102, a portion of a third party system, etc.) may apply mean centering to at least one embedding set.

[0130] In some non-limiting embodiments or aspects, applying mean centering can include determining a mean based on all embedding vectors in the set of embedding vectors. Additionally or alternatively, the mean can be subtracted from each embedding vector in the set of embedding vectors.

[0131] In some non-limiting embodiments or aspects, the embedding normalization / alignment system 102a may apply geometric median normalization. For example, applying geometric median normalization may include determining a geometric median (x*) based on the embedding set (e.g., by the embedding normalization / alignment system 102a) and / or normalizing each embedding vector of the embedding set based on the geometric median (x*) (e.g., by the embedding normalization / alignment system 102a). For example, determining the geometric median may include inputting the embedding set into a Weiszfeld algorithm to determine a geometric mean. Additionally or alternatively, normalizing each embedding vector may include replacing each respective embedding vector with a respective modified embedding vector determined based on subtracting the geometric median (x*) from the respective embedding vector to provide a respective difference and dividing the difference by a magnitude of the difference (e.g., a vector magnitude). For purposes of illustration, applying geometric median normalization may include applying the following algorithm, where A is the embedding set, a i is the i-th embedding vector of the embedding set A, x* is the geometric median, and Weiszfeld() is the Weizfeld algorithm:

[0132] Algorithm 1

[0133]

[0134] For purposes of illustration and not limitation, the Weiszfeld algorithm may include applying the following algorithm, where a i is the embedding set A (e.g., a1 to a n ), x0 is the starting point, x k is the value of x at the kth iteration, T() is Equation 1, and x k+1 is determined based on Equation 2:

[0135] Algorithm 2

[0136]

[0137] Equation 1

[0138]

[0139] Equation 2

[0140]

[0141] In some non-limiting embodiments or aspects, applying geometric median normalization can be used in addition to or instead of applying mean centering.

[0142] like Figure 3As shown, process 300 can include applying spectral normalization at step 306. For example, embedding normalization / alignment system 102a (e.g., a server, a part of transaction service provider system 102, a part of a third party system, etc.) can apply spectral normalization to at least one embedding set.

[0143] In some non-limiting embodiments or aspects, applying spectral normalization to at least one embedding set may include decomposing the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix. For example, decomposing the at least one embedding set may include performing singular value decomposition (SVD) on the at least one embedding set.

[0144] In some non-limiting embodiments or aspects, an average singular value of the at least one embedding set may be determined. For example, determining the average singular value may include determining a square root of an average squared singular value.

[0145] In some non-limiting implementations or aspects, for each corresponding singular value of the diagonal matrix, it may be determined whether the corresponding singular value is greater than a configurable multiple of the average singular value.

[0146] In some non-limiting embodiments or aspects, if the corresponding singular value is greater than the configurable multiple of the average singular value, the corresponding alternative singular value can be determined based on the quotient of the corresponding singular value divided by the configurable multiple of the average singular value. Additionally or alternatively, if the corresponding singular value is not greater than the configurable multiple of the average singular value, the corresponding alternative singular value can be determined to be a configurable value (e.g., 1, a predetermined integer, a predetermined value, etc.). In some non-limiting embodiments or aspects, the alternative diagonal matrix may include the corresponding alternative singular values ​​for each corresponding singular value of the diagonal matrix. In some non-limiting embodiments or aspects, the at least one embedded set may be replaced by the product of the at least one embedded set, the right singular vector, and the inverse matrix of the alternative diagonal matrix.

[0147] In some non-limiting embodiments or aspects, for purposes of illustration, applying spectral normalization may include applying the following algorithm, where A is the embedding set, svd() is the singular value decomposition function, U is the left singular vector, V is the right singular vector, Σ is a diagonal matrix, T is the transpose operator, η is the mean singular value, D is the (alternative) diagonal matrix, d is the dimension of the embedding vector, ||A|| F is the Frobenius norm of the embedding set A, and β is a parameter (e.g., a hyperparameter, optional parameter, etc.) that determines a configurable multiple of the mean singular value:

[0148] Algorithm 3

[0149]

[0150] For example, if the corresponding singular value (e.g., Σ ii ) is greater than a configurable multiple of the average singular value (e.g., βη), then the corresponding singular value may be divided by a configurable multiple of the average singular value (e.g., Σ ii / (βη)) to determine the corresponding alternative singular value (e.g., D ii ). Additionally or alternatively, if the corresponding singular value (e.g., Σ ii ) is not greater than a configurable multiple of the average singular value (e.g., βη), then the corresponding alternative singular value (e.g., D ii ) is a configurable value (e.g., 1, a predetermined integer, a predetermined value, etc.).

[0151] like Figure 3 As shown, process 300 may include applying length normalization at step 308. For example, embedding normalization / alignment system 102a (e.g., a server, a portion of transaction service provider system 102, a portion of a third party system, etc.) may apply length normalization to at least one embedding set.

[0152] In some non-limiting embodiments or aspects, applying length normalization may include adjusting each embedding vector in the set of embedding vectors to have a 2-norm (e.g., Euclidean norm) of 1.

[0153] In some non-limiting embodiments or aspects, Figure 3 As shown, steps 304, 306, and 308 may be repeated for a configurable number of iterations. For example, the embedding normalization / alignment system 102a (e.g., a server, a part of the transaction service provider system 102, a part of a third-party system, etc.) may iteratively repeat applying mean centering, applying spectral normalization, and applying length normalization for a configurable number of iterations on at least one embedding set.

[0154] In some non-limiting embodiments or aspects, for purposes of illustration, iteratively repeating may include applying the following algorithm, where m is a configurable number of iterations, A is an embedding set, Center is mean centering, SpecNorm is spectral normalization, and Unit Length Normalization is length normalization, as described herein:

[0155] Algorithm 4

[0156]

[0157] In some non-limiting embodiments or aspects, parameters can be adjusted. For example, a parameter β for determining a configurable multiple of the average singular value and / or a parameter m for a configurable number of iterations can be adjusted (e.g., by embedding the normalization / alignment system 102a) to achieve at least one of avoiding overfitting, improving performance, any combination thereof, etc. For illustrative purposes, Table 1 shows the average precision (MAP) achieved using different values ​​of parameter β (e.g., 1, 2, 3, 4, and 5) and different values ​​of parameter m (e.g., 1, 2, 3, 4, and 5) for Procrustes alignment based on ten exemplary language pairs (e.g., English to another language or another language to English):

[0158]

[0159]

[0160] Table 1

[0161] For illustration purposes, Table 2 shows the average Spearman rank coefficient scores for the monolingual word similarity task of the disclosed technique without normalization (e.g., None) and with different values ​​of parameter β and parameter m:

[0162] none β＝2,m＝2 β＝2,m＝3 β＝2,m＝4 β＝2,m＝5 0.651 0.67077 0.67101 0.67108 0.67111

[0163] Table 2

[0164] like Figure 3 As shown, process 300 may include aligning the embedding sets at step 310. For example, at least one embedding set may include a first embedding set and a second embedding set, and an embedding normalization / alignment system 102a (e.g., a server, a portion of transaction service provider system 102, a portion of a third party system, etc.) may align the first embedding set with the second embedding set.

[0165] In some non-limiting embodiments or aspects, aligning the embedding set may include applying at least one cross-lingual word embedding (CLWE) alignment model. For example, the CLWE alignment model may include a Procrustes model, a BootstrapProcrustes (PROC-B) model, a Multilingual Unsupervised and Supervised Embedding (MUSE) model, a Canonical Correlation Analysis (CCA) model, a Discriminant Latent Variable (DLV) model, a ranking-based optimization model, a Cross-Domain Similarity Local Scaling (CSLS) model, a Relaxed Cross-Domain Similarity Local Scaling (RCSLS) model, a VECMAP model, a supervised alignment model, an unsupervised alignment model, a semi-supervised alignment model, any combination thereof, and the like.

[0166] In some non-limiting embodiments or aspects, aligning the embedding sets may include applying at least one CLWE alignment model, even if the embedding sets do not represent a language. For example, at least one embedding set may include a first embedding set representing entities in a first embedding space associated with a first time period (e.g., merchant, customer / cardholder, issuer, acquirer, payment gateway, etc.), and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period. In some non-limiting embodiments or aspects, such non-language-based embedding sets may be aligned (e.g., by embedding normalization / alignment system 102a) using one or more CLWE alignment models (e.g., one or more of the exemplary CLWE alignment models listed above), for example, by treating each embedding set as a language and treating each embedding vector of each embedding set as a word of the corresponding language.

[0167] Reference now FIG. 4A to FIG. 4C , FIG. 4A to FIG. 4C is a diagram showing some non-limiting embodiments or aspects of Figure 3 Bar graphs 400a, 400b, 400c of the performance of an exemplary implementation of a process.

[0168] like Figure 4A As shown, the vertical axis can represent the condition number, and the horizontal axis can include the categories of the following four exemplary languages: English (EN), German (DE), Hindi (HI), and Japanese (JA). For each exemplary language, the condition number is represented by a corresponding bar for each of the following preprocessing techniques: no normalization 401 (e.g., none), iterative mean centering and spectral normalization and length normalization 402 (e.g., I-C+SN+L, which can be a shorthand for an iterative combination of mean centering (C), spectral normalization (SN), and length normalization (L), as described herein), PCA removal 403 (e.g., PR), mean centering and length normalization 404 (e.g., C+L, which can be a single round / no iteration), iterative mean centering and length normalization 405 (e.g., I-C+L, which can be multiple rounds (e.g., 5 rounds) of iterations), and geometric median 406 (e.g., GeoMedian). Notably, for all four exemplary languages, the condition number for iterative mean centering and spectral normalization and length normalization 402 (e.g., I-C+SN+L) is smaller than each of the other techniques, demonstrating improved performance.

[0169] like Figure 4BAs shown, the vertical axis can represent the numerical rank, and the horizontal axis can include the categories of the following four exemplary languages: English (EN), German (DE), Hindi (HI), and Japanese (JA). For each exemplary language, the numerical rank is represented by a corresponding bar for each of the following preprocessing techniques: no normalization 411 (e.g., None), iterative mean centering and spectral normalization and length normalization 412 (e.g., I-C+SN+L, which can be a shorthand for an iterative combination of mean centering (C), spectral normalization (SN), and length normalization (L), as described herein), PCA removal 413 (e.g., PR), mean centering and length normalization 414 (e.g., C+L, which can be a single round / no iteration), iterative mean centering and length normalization 415 (e.g., I-C+L, which can be multiple rounds (e.g., 5 rounds) of iterations), and geometric median 416 (e.g., GeoMedian). Notably, for all four exemplary languages, the numerical rankings for iterative mean centering and spectral normalization and length normalization 402 (e.g., I-C+SN+L) are greater than each of the other techniques, demonstrating improved performance.

[0170] like Figure 4C As shown, the vertical axis can represent the joint condition number, and the horizontal axis can include categories for the following five exemplary language pairs (e.g., for translation from a first language to a second language): English to Bulgarian (EN-BG), English to German (EN-DE), English to Finnish (EN-FI), English to Hindi (EN-HI), and English to Korean (EN-KO). For each exemplary language pair, the condition number is represented by a corresponding bar for each of the following preprocessing techniques: no normalization 431 (e.g., None), and iterative mean centering and spectral normalization and length normalization 432 (e.g., I-C+SN+L). Notably, the joint condition number for iterative mean centering and spectral normalization and length normalization 402 (e.g., I-C+SN+L) is reduced compared to no normalization, thereby demonstrating improved performance.

[0171] Reference now Figure 5A and Figure 5B , Figure 5A and Figure 5B is a diagram showing some non-limiting embodiments or aspects of Figure 3 500a, 500b are line graphs of performance of an exemplary implementation of a process.

[0172] like Figure 5A and Figure 5B As shown in each of the above, the vertical axis can represent the singular value, and the horizontal axis can represent the number of singular values. It is worth noting that Figure 5BThe scale ratio of the vertical axis of the middle graph 500b is Figure 5A The vertical axis of the graph 500a is narrow, and Figure 5B The maximum value of the vertical axis of the graph 500b is less than Figure 5A The maximum value of the vertical axis of the middle graph 500a.

[0173] like Figure 5A As shown, for each of the following exemplary languages, there are singular value lines with respect to the number of singular values ​​without using a normalization technique (e.g., None): Bulgarian (BG) 501, German (DE) 502, English (EN) 503, Finnish (FI) 504, Hindi (HI) 505, and Korean (KO) 506. For each of these lines, the singular values ​​decay sharply as the number of singular values ​​increases. Therefore, aligning these languages ​​without using a normalization technique may result in a forced alignment based on the top singular values ​​due to the clustering of words, regardless of whether the words in these clusters are actually aligned.

[0174] like Figure 5B As shown, for each of the following exemplary languages, after applying iterative mean centering and spectral normalization and length normalization (e.g., I-C+SN+L), there are singular value lines with respect to the number of singular values: Bulgarian (BG) 511, German (DE) 512, English (EN) 513, Finnish (FI) 514, Hindi (HI) 515, and Korean (KO) 516. For each of these lines, as the number of singular values ​​increases, the singular values ​​are relatively uniform. Therefore, an alignment model (e.g., a CLWE alignment model) will have more freedom to align words that actually match without the burden of clustering described above without using normalization techniques.

[0175] For illustrative purposes, Table 3 shows the MAP achieved using three different CLWE alignment models (CCA, PROC, and PROC-B) for bilingual dictionary induction (BLI) using different preprocessing techniques (no normalization (None), PCA removal (PR), geometric median (GeoMedian), mean centering and length normalization (C+L), iterative mean centering and length normalization (I-C+L, 5 iterations), mean centering and spectral normalization and length normalization (C+SN+L), and iterative mean centering and spectral normalization and length normalization (I-C+SN+L, 5 iterations)) based on eighteen exemplary language pairs (e.g., English to each of the following and from each of the following to English: Bulgarian (BG), Catalan (CA), Czech (CS), German (DE), Spanish (ES), French (FR), Korean (KO), Thai (TH), and Chinese (ZH)):

[0176]

[0177] Table 3

[0178] For illustrative purposes, Table 4 shows the MAP achieved using five different CLWE alignment models (CCA, PROC, PROC-B, DLV, and RCSLS) for BLI using different preprocessing techniques (no normalization (None), and iterative mean centering and spectral normalization and length normalization (ICSNL)) on 28 language pairs for dictionary sizes of 1000 (1K), 3000 (3K), and 5000 (5K):

[0179]

[0180] Table 4

[0181] Although the disclosed subject matter has been described in detail for purposes of illustration based on what are presently considered to be the most practical and preferred embodiments or aspects, it should be understood that such details are only for that purpose and that the disclosed subject matter is not limited to the disclosed embodiments or aspects, but rather is intended to cover modifications and equivalent arrangements within the spirit and scope of the appended claims. For example, it should be understood that the presently disclosed subject matter contemplates that one or more features of any embodiment or aspect may be combined with one or more features of any other embodiment or aspect as much as possible.

Claims

1. A method for normalizing cross-embedding alignment embeddings, the method comprising: receiving, with at least one processor, at least one embedding set, each embedding set in the at least one embedding set comprising a set of embedding vectors; applying, with at least one processor, mean centering to the at least one embedding set; Applying, with at least one processor, spectral normalization to the at least one embedding set, wherein applying spectral normalization to the at least one embedding set comprises: decomposing, with at least one processor, the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining, with at least one processor, an average singular value of the at least one embedding set; for each corresponding singular value of the diagonal matrix, determining, with at least one processor, whether the corresponding singular value is greater than a configurable multiple of the average singular value; for each corresponding singular value of the diagonal matrix, determining, with at least one processor, a corresponding replacement singular value based on a quotient of the corresponding singular value divided by the configurable multiple of the average singular value if the corresponding singular value is greater than the configurable multiple of the average singular value, and determining, with at least one processor, the corresponding replacement singular value to be 1 if the corresponding singular value is not greater than the configurable multiple of the average singular value, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing, with at least one processor, the at least one embedding set with a product of the at least one embedding set, the right singular vectors, and an inverse of the replacement diagonal matrix; and applying, with at least one processor, length normalization to the at least one embedding set; The at least one embedding set comprises a first language embedding set and a second language embedding set, the first language embedding set comprising a first set of word embedding vectors for a first language, and the second language embedding set comprising a second set of word embedding vectors for a second language.

2. The method of claim 1, wherein applying mean centering comprises: determining, with at least one processor, a mean based on all embedded vectors in the set of embedded vectors; as well as The mean is subtracted from each embedding vector in the set of embedding vectors with at least one processor.

3. The method of claim 1, wherein decomposing the at least one embedding set comprises performing a singular value decomposition on the at least one embedding set.

4. The method of claim 1, wherein determining the average singular value comprises determining a square root of an average squared singular value.

5. The method of claim 1, wherein applying length normalization comprises: Each embedding vector in the set of embedding vectors is adjusted with at least one processor to have a Euclidean norm of 1.

6. The method of claim 1, further comprising: Applying mean centering, applying spectral normalization, and applying length normalization iteratively repeats with at least one processor to the at least one embedding set for a configurable number of iterations.

7. The method of claim 1, wherein the at least one embedding set comprises a first embedding set and a second embedding set, the method further comprising: The first embedding set is aligned with the second embedding set using at least one processor.

8. The method of claim 1, wherein the at least one embedding set comprises a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period.

9. The method of claim 8, wherein the entity comprises at least one of a merchant, a customer, an issuer, an acquirer, or a payment gateway.

10. A system for normalizing cross-embedding alignment embeddings, the system comprising: at least one processor; as well as At least one non-transitory computer-readable medium comprising one or more instructions that, when executed by the at least one processor, direct the at least one processor to: receiving at least one embedding set, each embedding set in the at least one embedding set comprising a set of embedding vectors; applying mean centering to the at least one embedding set; Applying spectral normalization to the at least one embedding set, wherein applying spectral normalization to the at least one embedding set comprises: decomposing the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining an average singular value of the at least one embedding set; for each corresponding singular value of the diagonal matrix, determining whether the corresponding singular value is greater than a configurable multiple of the average singular value; for each corresponding singular value of the diagonal matrix, determining a corresponding replacement singular value based on a quotient of the corresponding singular value divided by the configurable multiple of the average singular value if the corresponding singular value is greater than the configurable multiple of the average singular value, and determining the corresponding replacement singular value to be 1 if the corresponding singular value is not greater than the configurable multiple of the average singular value, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing the at least one embedding set with the product of the at least one embedding set, the right singular vectors, and the inverse of the replacement diagonal matrix; and applying length normalization to the at least one embedding set; The at least one embedding set comprises a first language embedding set and a second language embedding set, the first language embedding set comprising a first set of word embedding vectors for a first language, and the second language embedding set comprising a second set of word embedding vectors for a second language.

11. The system of claim 10, wherein applying mean centering comprises: determining a mean based on all embedding vectors in the set of embedding vectors; as well as The mean is subtracted from each embedding vector in the set of embedding vectors.

12. The system of claim 10, wherein decomposing the at least one embedding set comprises performing a singular value decomposition on the at least one embedding set.

13. The system of claim 10, wherein determining the average singular value comprises determining a square root of an average squared singular value.

14. The system of claim 10, wherein applying length normalization comprises: Each embedding vector in the set of embedding vectors is adjusted to have a Euclidean norm of 1.

15. The system of claim 10, wherein the instructions, when executed by the at least one processor, further direct the at least one processor to: Applying mean centering, applying spectral normalization, and applying length normalization are iteratively repeated on the at least one embedding set for a configurable number of iterations.

16. The system of claim 10, wherein the at least one embedding set comprises a first embedding set and a second embedding set, wherein the instructions, when executed by the at least one processor, further direct the at least one processor to: The first set of embeddings is aligned with the second set of embeddings.

17. The system of claim 10, wherein the at least one embedding set comprises a first embedding set representing entities in a first embedding space associated with a first time period, and a second embedding set representing entities in a second embedding space associated with a second time period different from the first time period, and The entity includes at least one of a merchant, a customer, an issuer, an acquirer, or a payment gateway.

18. A computer program product for normalizing cross-embedding alignment, the computer program product comprising at least one non-transitory computer readable medium comprising one or more instructions that, when executed by at least one processor, cause the at least one processor to: receiving at least one embedding set, each embedding set in the at least one embedding set comprising a set of embedding vectors; applying mean centering to the at least one embedding set; Applying spectral normalization to the at least one embedding set, wherein applying spectral normalization to the at least one embedding set comprises: decomposing the at least one embedding set to provide left singular vectors, right singular vectors, and a diagonal matrix; determining an average singular value of the at least one embedding set; for each corresponding singular value of the diagonal matrix, determining whether the corresponding singular value is greater than a configurable multiple of the average singular value; for each corresponding singular value of the diagonal matrix, determining a corresponding replacement singular value based on a quotient of the corresponding singular value divided by the configurable multiple of the average singular value if the corresponding singular value is greater than the configurable multiple of the average singular value, and determining the corresponding replacement singular value to be 1 if the corresponding singular value is not greater than the configurable multiple of the average singular value, wherein a replacement diagonal matrix includes the corresponding replacement singular value for each corresponding singular value of the diagonal matrix; and replacing the at least one embedding set with the product of the at least one embedding set, the right singular vectors, and the inverse of the replacement diagonal matrix; and applying length normalization to the at least one embedding set; The at least one embedding set comprises a first language embedding set and a second language embedding set, the first language embedding set comprising a first set of word embedding vectors for a first language, and the second language embedding set comprising a second set of word embedding vectors for a second language.

Images