Method and system for identifying a matching with a product

Abstract

The present disclosure provides methods and systems for identifying a match to a product. In some examples, there is provided a method for identifying a match with a product, comprising: extracting, by a processor, one or more attributes related to the product from data indicated in a request for identifying the product, each attribute of the one or more attributes describing the product; adjusting, by the processor, the one or more attributes to a same format as a format of one or more entries in a database based on historical data relating to the product, the database including the one or more entries for each of the plurality of products; and identifying a match to the product from the plurality of products, the identifying based on comparing, by the processor, the one or more attributes to each corresponding entry of the plurality of products.

Description

Technical Field

[0001] The present disclosure relates broadly, but not exclusively, to methods and systems for identifying matches with products. Background Art

[0002] Currently, on platforms that offer a variety of products for sale, the listed products are typically each identified with an item ID that is unique to the merchant or platform offering the product, but these products cannot be identified with a universal identifier (such as, for example, a product stock keeping unit (SKU) number that can be used across merchants and stores). This therefore limits the platform's ability to work with fast moving consumer goods (FMCG) companies and prevents them from advertising on the platform.

[0003] Ideally, merchants add Global Trade Item Numbers (GTINs) to their product items when they onboard and categorize FMCG products into their menus. However, in reality, most merchants do not add GTINs because it is time-consuming and does not provide them with a direct benefit. As a result, due to this gap, FMCGs are unable to track product availability and sales performance, so they are unable to drive merchant demand through methods such as discounting specific products across merchants, evaluating the return on investment / return on advertising spend (ROI / ROAS) of campaigns, and justifying further investment. Further, manually adding GTINs for all FMCG in the platform would take an extremely long time to complete and is impractical. As a result, it is necessary to develop a system to help add GTINs to as many FMCG brand product items as possible to achieve GTIN-level tracking.

[0004] Therefore, there is a need to provide methods and systems that are directed to overcoming or at least minimizing the above challenges. Summary of the invention

[0005] According to a first aspect of the present disclosure, a method for identifying a match with a product is provided, the method comprising: extracting, by a processor, one or more attributes related to a product from data indicated in a request for identifying a product, each of the one or more attributes describing the product, wherein the data comprises an image of the product, and extracting the one or more attributes comprises identifying the one or more attributes from the image based on a visual question answering (VQA) paradigm; adjusting, by the processor, the one or more attributes to a format identical to a format of one or more entries in a database based on historical data related to the product, the database comprising one or more entries for each of a plurality of products; and identifying a match with the product from the plurality of products, identifying the match based on the processor's analysis of the one or more attributes with the plurality of products. The processor comprises: a processor that compares each corresponding entry of a product with a rule-based comparison of at least one of the one or more attributes with each corresponding entry in the database; selects one or more products from a plurality of products in a database based on a rule-based comparison of at least one of the one or more attributes with each corresponding entry in the database; identifies a match with the product from the selected one or more products, the identification being based on the processor comparing the one or more attributes with each corresponding entry in the selected one or more products; calculates a score for each of the selected one or more products based on the comparison; identifies a match with the product when the match score is the highest score among the calculated scores and is greater than a threshold; verifies whether each product in the selected one or more products other than the identified match with a score greater than the threshold also matches the product; and retrains the processor based on the verification.

[0006] According to a second aspect of the present disclosure, a system for identifying a match with a product is provided, the system comprising: at least one processor; and at least one memory, the at least one memory comprising computer program code; the at least one memory and the computer program code being configured to utilize the at least one processor so that the system at least: extracts, by the processor, one or more attributes related to the product from data indicated in a request for identifying the product, each of the one or more attributes describing the product, wherein the data comprises an image of the product, and extracting the one or more attributes comprises identifying the one or more attributes from the image based on a visual question answering (VQA) paradigm; adjusts, by the processor, the one or more attributes to a format identical to a format of one or more entries in a database based on historical data related to the product, the database comprising one or more entries for each of a plurality of products; and Identify a match with the product from a plurality of products, the identification based on a processor comparing one or more attributes with each corresponding entry of the plurality of products; select one or more products from a plurality of products in a database based on a rule-based comparison of at least one of the one or more attributes with each corresponding entry in the database; identify a match with the product from the selected one or more products, the identification based on a processor comparing one or more attributes with each corresponding entry of the selected one or more products; calculate a score for each of the selected one or more products based on the comparison; identify a match with the product when the match score is the highest score among the calculated scores and is greater than a threshold; verify whether each of the selected one or more products, except for the identified match having a score greater than the threshold, also matches the product; and retrain the processor based on the verification. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The embodiments and implementations are provided by way of example only and will be better understood and apparent to one of ordinary skill in the art from the following written description in conjunction with the accompanying drawings, in which:

[0008] Figure 1 A system for identifying a match with a product according to various embodiments of the present disclosure is illustrated.

[0009] Figure 2 is a schematic diagram of a product matching server according to various embodiments of the present disclosure.

[0010] Figure 3 is an overview of a process for identifying a match with a product, according to various implementations.

[0011] Figure 4 Depicted is an example diagram of a visual question answering (VQA) process in accordance with various implementations.

[0012] Figure 5An example framework for identifying matches with products is illustrated in accordance with various implementations.

[0013] Figure 6 An example diagram of mapping products to a GTIN reference list is illustrated in accordance with various implementations.

[0014] Figure 7 An example flow diagram of a method for identifying a match with a product is illustrated in accordance with various implementations.

[0015] Fig. 8A and Figure 8B A schematic block diagram of a general purpose computer system on which the Figure 1 Transaction processing server.

[0016] Figure 8C is a schematic block diagram of a general-purpose computer system on which the Figure 2 Product matching server.

[0017] Fig.8D is a schematic block diagram of a general-purpose computer system on which the Figure 1 A combined transaction processing and product matching server.

[0018] Fig. 9 Shown is a method for implementing Figure 1 An example of a computing device of a transaction processing server is shown in .

[0019] Fig.10 Shown is a method for implementing Figure 1 An example of a computing device that is a product match server is shown in FIG.

[0020] Fig.11 Shown is a method for implementing Figure 1 An example of a computing device that combines a transaction processing and product matching server is shown in FIG.

[0021] The skilled person will appreciate that the elements in the drawings are illustrated for simplicity and clarity and are not necessarily drawn to scale. For example, the size of some elements in the diagrams, block diagrams or flow charts may be enlarged relative to other elements to help improve understanding of the present embodiment. DETAILED DESCRIPTION

[0022] Embodiments will be described by way of example only with reference to the accompanying drawings. Like reference numerals and characters in the drawings refer to like elements or equivalents.

[0023] Some portions of the following description will be presented explicitly or implicitly in the form of algorithms and functional or symbolic representations of operations on data in a computer memory. These algorithmic descriptions and functional or symbolic representations are the means used by those skilled in the art of data processing to most effectively communicate the substance of their work to others skilled in the art. An algorithm, as used herein, is generally considered to be a series of self-consistent steps that can achieve a desired result. These steps are steps that require physical manipulation of physical quantities, such as electrical, magnetic or optical signals that can be stored, transmitted, combined, compared and otherwise manipulated.

[0024] Unless otherwise specifically stated, and as will become apparent from the following, it will be understood that throughout this specification, discussions utilizing terms such as "extract," "adjust," "identify," "select," "calculate," "validate," "retrain," "indicate," "map," "compare," and the like refer to actions and processes of a computer system or similar electronic device that manipulate and transform data represented by physical quantities within a computer system into other data represented by the same physical quantities within a computer system or other information storage, transmission, or display device.

[0025] In addition, this specification also implicitly discloses a computer program, and it is obvious to those skilled in the art that the various steps of the method described herein can be implemented by computer code. The computer program is not intended to be limited to any specific programming language and its implementation. It should be understood that various programming languages ​​and their encodings can be used to implement the teachings of the present disclosure contained herein. In addition, the computer program is not intended to be limited to any specific control flow. There are many other variations of computer programs, which can use different control flows without departing from the scope of the specification.

[0026] In addition, one or more of the steps of the computer program may be performed in parallel rather than sequentially. Such a computer program may be stored on any computer readable medium. The computer readable medium may include a storage device such as a disk or optical disk, a memory chip or other storage device suitable for being connected to a computer interface. The computer readable medium may also include a hardwired medium such as illustrated in an Internet system or a wireless medium such as illustrated in a GSM mobile phone system. When the computer program is loaded and executed on such a computer, the computer program effectively generates a device that implements the preferred method steps.

[0027] A platform refers to a business model that creates value by facilitating exchanges between two or more interdependent groups, such as exchanges between requesters (of products or services) and providers (of products or services). For example, a platform may provide services provided by providers to requesters, such as rides, deliveries, online shopping, insurance, and other similar services. Requesters can typically access the platform via a website, application, or other similar method.

[0028] A Global Trade Item Number (GTIN) is a globally unique identifier and is defined as a way to identify a trade item as a product or service that is priced, ordered or invoiced at any point in the supply chain. It is widely used in the fast moving consumer goods (FMCG) industry (e.g. Coca Cola TM ) to uniquely identify all its trade items. In order to expand the growth of the platform's partnership with FMCG, a GTIN-level product tracking system across the platform is critical for most FMCG business use cases, such as participation in advertising campaigns, market sampling, introductory training for services provided by the platform (such as delivery of (related products)), and performance reporting for product sales. GTINs can be compiled in a database, for example, in the form of a list or other similar format including one or more GTINs, each GTIN corresponding to a different product, and each GTIN also includes one or more entries related to the respective corresponding products. Each of the one or more entries can be an attribute describing the respective corresponding product, such as brand, color, size, capacity, flavor, quantity, or other similar characteristics that describe or define the product. The database can then be used for reference to identify a match with the product, and the GTIN corresponding to the match can be mapped to the relevant product. It should be understood that the product referred to herein is not necessarily limited to FMCG, and generally refers to any item that a provider can provide to a requester for sale.

[0029] The attributes of a product refer to product characteristics that can be used to distinguish one product from another. For example, the attributes of a product can be brand, color, size, capacity, flavor, quantity, or other similar features that describe or define the product. Such attributes of a product can be extracted from information related to the product (e.g., text information and image information) and used to identify matches with the product. This extraction of attributes involving different information modes in the form of text and images can be referred to as multimodal attribute extraction. This is particularly useful when the information related to the product is unstructured and multimodal (e.g., including text and images). Common forms of text information may include product names, descriptions, nutritional information, and other similar metadata. The extracted attributes can also be adjusted to a format suitable for further analysis (such as for comparison with a database including one or more GTINs mentioned above). The adjustment of the appropriate format can be based on historical data related to the relevant product. For example, domain knowledge injection can be used to fill the gap between the extracted attributes and one or more entries in the GTIN database, wherein the extracted attributes are enriched with historical data related to the product (such as domain knowledge related to the product). Domain knowledge may include synonyms of product attributes in various languages, unit systems, or other similar formats. Historical data may also include labeled data and unlabeled data related to the product. Labeled data refers to any data related to the product that has been annotated with certain characteristics or properties or features for supervised training of machine learning models (including neural network models) to make accurate predictions. On the other hand, unlabeled data refers to such raw data related to the product that has not been labeled or annotated with additional characteristics. Therefore, the main process for adjusting the extracted attributes to a suitable format (for example, to the same format as one or more entries in the database) may include language conversion, unit conversion, quantity and synonym standardization, and other similar processing, so that the extracted attributes can be reliably and accurately compared with each corresponding entry in the database.

[0030] The purpose discussed in this disclosure is to identify the GTIN of a product based on information related to the product (e.g., information received from a merchant to add the product to the platform so that platform users can view and purchase the product) and a GTIN reference database. Figure 6 As shown in the diagram 600 of FIG. 1 , item information 602 typically includes image information 604 and text information 606 (e.g., including item name, description, quantity, capacity, weight, and other similar information) and other metadata depending on merchant input. The attributes of the relevant item extracted from the item information 602 can be compared with the GTIN reference list 608. The GTIN reference list 608 can be based on information from sources such as GS1 TMIn addition to the GTIN number 610, each row of the GTIN reference list 608 also includes entries for product information such as brand 612, description 614, capacity 616, and quantity 618. It should be understood that the entries may also include other specifications or attributes describing the item or product.

[0031] There are several challenges in identifying a match to a product and deciding whether an item should be mapped to a GTIN in the GTIN reference list 608. Product attributes that describe or define a product, such as brand, color, size, capacity, flavor, quantity, or other similar characteristics, are features that can be used to distinguish one product from another. However, attribute-based matching is not easy because information about products is often provided in an unstructured and multimodal form, such as by merchants who wish to add products to the platform. Merchants often write all item information in a free-style in a single item name string (e.g., "Coke TM Sugar Free 12x320ml”). In other cases, some attributes, such as capacity, flavor, or container type, are not included in the item name and can only be found from the image uploaded as part of the data related to the product. Even after the attributes are extracted, there may still be a knowledge gap between the attributes extracted from the data and the GTIN reference list 608. Since platforms can often operate in many different countries, merchants may use multiple languages, while the GTIN reference list 608 may often be in English. To make things more complicated and confusing, merchants also like to use synonyms for brands or products, such as “Coke TM "For Coca-cola TM In addition, the units of measurement may also differ between the merchant and the GTIN reference list 608. For example, some local merchants prefer to use "fl.oz" instead of "ml" to indicate volume, and "lbs" instead of "kg" to indicate weight.

[0032] Matching items to GTINs is inherently a dynamic process that requires human assistance and intervention cycles. New brands and new products with unique attributes enter the market every day. The GTIN reference list 608 needs to be constantly updated with new GTINs, and the matching rules must also adapt to new attribute types. Further, the matching system also needs to learn from feedback and empirical rules provided and set by, for example, the content operations of the relevant platform. Content operations refers to a set of processes, people and technologies required to strategically plan, create, manage and analyze all content types for all channels of an enterprise such as a platform. For example, although a large number of products may only use 'Coke TM ' as a title without additional information, but content operations can be achieved by assigning this title to theTM Original 320ml" to simplify the process and provide better mapping coverage in practice.

[0033] A user may be any suitable type of entity, which may include: a person, a customer who desires to purchase a product or service via a transaction processing server, a seller or merchant who desires to sell a product or service via a transaction processing server, a motorcycle driver or pillion rider in the case where a user desires to book or provide a motorcycle ride via a transaction processing server, a car driver or passenger in the case where a user desires to book or provide a ride via a transaction processing server, and other similar entities. Users registered in a transaction processing server or a commodity matching server will be referred to as registered users. Users not registered in a transaction processing server or a commodity matching server will be referred to as unregistered users. The term "user" will be used to uniformly refer to registered users and unregistered users. Users may be interchangeably referred to as requesters (e.g., people who request products or services) or providers (e.g., people who provide requested products or services to requesters).

[0034] A product matching server is a server that hosts a software application used to perform product matching. Figure 3 The implementation shown in the schematic diagram 300 of FIG. 1 is to identify a match with a product.

[0035] A transaction processing server is a server that hosts a software application for processing payment transactions (e.g., purchases of goods or services by users). The transaction processing server communicates with any other servers involved in processing payment transactions related to the purchase of goods or services (e.g., a product matching server). For example, where product matching is a service provided by the platform, the transaction processing server may communicate with the product matching server to facilitate payment for the product matching service after a product match is identified (e.g., a match based on product information provided by a user requesting a product match) and / or after the corresponding GTIN of the identified product is provided to the user requesting a product match. In another example, where product matching is part of a product onboarding process, the transaction processing server may or may not be required to communicate with the product matching server to facilitate the onboarding process. The transaction processing server may use a variety of different protocols and processes to process payment and / or travel coordination requests.

[0036] Transactions that may be conducted via the transaction processing server include product or service purchases, credit purchases, debit transactions, fund transfers, account withdrawals, etc. The transaction processing server may be configured to process transactions via cash substitutes, which may include payment cards, letters of credit, checks, payment accounts, etc.

[0037] The transaction processing server is typically managed by a service provider, which may be an entity (e.g., a company or organization), and operates to process transaction requests and / or travel coordination requests, for example, pairing a provider of a travel coordination request with a requester of the travel coordination request. The transaction processing server may include one or more computing devices for processing transaction requests and / or travel coordination requests.

[0038] A transaction account is an account of a user registered on a transaction processing server. A user can be a customer, a merchant offering products for sale on the platform and / or joining the platform, a service provider (e.g., a driver), or any third party who wants to use a transaction processing server (e.g., a courier). In some cases, a transaction account does not require the use of a facial recognition server. A transaction account includes detailed information about the user (e.g., name, address, vehicle, facial image, etc.). The transaction processing server manages the user's transaction account and the user's interactions with other external servers.

[0039] Figure 1 A block diagram of a system 100 for identifying a match with a product is illustrated. Further, the system 100 supports payment transactions for goods or services, and / or requests for rides between a requester and a provider.

[0040] The system 100 includes a requester device 102 , a provider device 104 , an acquirer server 106 , a transaction processing server 108 , an issuer server 110 , a product matching server 140 , and a reference database 150 .

[0041] The requesting device 102 communicates with the providing device 104 via a connection 112. The connection 112 may be wireless (e.g., via NFC communication, Bluetooth, etc.) or implemented over a network (e.g., the Internet). The requesting device 102 also communicates with the product matching server 140 via a connection 121. The connection 121 may be implemented over a network (e.g., the Internet). The requesting device 102 may also be connected to a cloud that facilitates the system 100 to identify matches with products. For example, the requesting device 102 may send a signal or data directly to the cloud via a wireless connection (e.g., via NFC communication, Bluetooth, etc.) or over a network (e.g., the Internet).

[0042] The provider device 104 typically communicates with the requester device 102 via the transaction processing server 108, as described above. The provider device 104 in turn communicates with the acquirer server 106 via connection 114. The provider device 104 also communicates with the product matching server 140 via connection 123. Connections 114 and 123 may be implemented via a network (e.g., the Internet). The provider device 104 may also be connected to a cloud that facilitates the system 100 to identify matches with products. For example, the provider device 104 may send signals or data directly to the cloud via a wireless connection (e.g., via NFC communication, Bluetooth, etc.) or over a network (e.g., the Internet).

[0043] The acquirer server 106, in turn, communicates with the transaction processing server 108 via a connection 116. The transaction processing server 108, in turn, communicates with the issuer server 110 via a connection 118. The connections 116 and 118 may be networks (eg, the Internet).

[0044] The transaction processing server 108 further communicates with the product matching server 140 via a connection 120. The connection 120 may be implemented through a network (e.g., a local area network, a wide area network, the Internet, etc.). In one arrangement, the transaction processing server 108 and the product matching server 140 are combined together, and the connection 120 may be an interconnecting bus.

[0045] Product matching server 140, in turn, communicates with reference database 150 via corresponding connection 122. Connection 122 may be a network (e.g., the Internet). Product matching server 140 may also be connected to a cloud that facilitates system 100 to identify matches with products. For example, product matching server 140 may send signals or data directly to the cloud via a wireless connection (e.g., via NFC communication, Bluetooth, etc.) or over a network (e.g., the Internet).

[0046] The reference database 150 includes a compilation of entries for a plurality of products, wherein each product includes one or more entries describing the associated product. The reference database 150 may be combined with the product matching server 140. In an example, the reference database 150 may be a database managed by an external entity, and the product matching server 140 is a server that extracts one or more attributes related to a product based on data related to the product indicated in a request to identify the product (e.g., a request provided by a user or provided during an onboarding process for the product), each of the one or more attributes describing the product. The product matching server 140 may then adjust the one or more attributes based on the historical data related to the product to the same format as the format of one or more entries in the reference database 150, which includes one or more entries for each of the plurality of products. Further, the product matching server 140 may then identify a match to the product from the plurality of products, the identification being based on comparing the one or more attributes to each corresponding entry of the plurality of products.

[0047] Alternatively, a module may store a compilation of information for a plurality of products instead of referencing database 150, wherein the module may be integrated as part of product matching server 140 or external to the product matching server.

[0048] In an illustrative embodiment, each of the devices 102, 104 and servers 106, 108, 110, 140, and 150 provides an interface to enable communication with other connected devices 102, 104 and / or servers 106, 108, 110, 140, and 150. Such communication is facilitated by an application program interface ("API"). Such an API may be part of a user interface, which may include a graphical user interface (GUI), a web-based interface, a programming interface (such as, an application program interface (API)) and / or a remote procedure call (RPC) set corresponding to interface elements), a messaging interface where the interface elements correspond to messages of a communication protocol, and / or a suitable combination thereof. For example, at least one of the requester device 102 and the provider device 104 may send data related to a product in response to a query shown on a GUI running on a corresponding API.

[0049] The term 'server' as used herein may refer to a single computing device or multiple interconnected computing devices operating together to perform a specific function. That is, a server may be contained within a single hardware unit or distributed among several or many different hardware units.

[0050] The product matching server 140 is associated with an entity (e.g., a company or organization or manager of a service). In one arrangement, the product matching server 140 is owned and operated by the entity that operates the transaction processing server 108. In such an arrangement, the product matching server 140 may be implemented as a part of the transaction processing server 108 (e.g., a computer program module, a computing device, etc.).

[0051] The transaction processing server 108 may also be configured to manage the registration of users. A registered user has a transaction account (see above discussion) that includes the user's details. The registration step is called joining. A user may use the requester device 102 or the provider device 104 to perform joining to the transaction processing server 108.

[0052] In order to access the functionality of the transaction processing server 108, it may not be necessary to have a transaction account on the transaction processing server 108. However, there are functions available for registered users. These additional functions will be discussed below.

[0053] The user's onboarding process is performed by the user through one of the requester device 102 or the provider device 104. In one arrangement, the user downloads an application (which includes an API that interacts with the transaction processing server 108) to the requester device 102 or the provider device 104. In another arrangement, the user accesses a website (which includes an API that interacts with the transaction processing server 108) on the requester device 102 or the provider device 104. The user is then able to interact with the product matching server 140. The user can be a requester or a provider associated with the requester device 102 or the provider device 104, respectively.

[0054] The registered details include, for example, user name, user address, emergency contact, blood type or other healthcare information, family contacts, permissions to retrieve data and information from the requester device 102 and / or provider device 104 for product identification purposes, such as permission to take a picture of the product using the camera of the requester device 102 and / or provider device 104, where the picture can be used as data for product identification purposes by the product matching server 140. Alternatively, another mobile device can be selected to retrieve data instead of the requester device 102 and / or provider device 104. Once joined, the user will have a transaction account where all the details are stored.

[0055] The requester device 102 is associated with a customer (or requester) who is a party to a transaction occurring between the requester device 102 and the provider device 104 or between the requester device 102 and the product matching server 140. The requester device 102 may be a computing device such as a desktop computer, an interactive voice response (IVR) system, a smart phone, a laptop computer, a personal digital assistant (PDA) computer, a mobile computer, a tablet computer, etc.

[0056] The requester device 102 includes the transaction credentials (e.g., payment account) of the requester to make the requester device 102 a party to the payment transaction. If the requester has a transaction account, the transaction account may also be included (i.e., stored) in the requester device 102. For example, a mobile device (which is the requester device 102) may have a transaction account of a customer stored in the mobile device.

[0057] In one exemplary arrangement, the requester device 102 is a computing device in a watch or similar wearable device and is equipped with a wireless communication interface (e.g., an NFC interface). The requester device 102 can then electronically communicate with the provider device 104 regarding the transaction request. The customer uses the watch or similar wearable device to make a request regarding the transaction request by pressing a button on the watch or wearable device.

[0058] The provider device 104 is associated with a provider that is also a party to the transaction request occurring between the requester device 102 and the provider device 104. The provider device 104 may be a computing device such as a desktop computer, an interactive voice response (IVR) system, a smart phone, a laptop computer, a personal digital assistant (PDA) computer, a mobile computer, a tablet computer, etc.

[0059] Hereinafter, the term "provider" refers to a service provider and any third party associated with providing a product or service for purchase, or a travel or ride or delivery service via the provider device 104. Thus, a provider's transaction account refers to both the provider's transaction account and the transaction account of a third party associated with the provider (e.g., a travel coordinator or merchant).

[0060] If the provider has a transaction account, the transaction account may also be included (ie, stored) in the provider device 104. For example, a mobile device (which is the provider device 104) may store the provider's transaction account in the mobile device.

[0061] In one exemplary arrangement, the provider device 104 is a computing device in a watch or similar wearable device and is equipped with a wireless communication interface (e.g., an NFC interface). The provider device 104 can then electronically communicate with the requester to make a request regarding the transaction request by pressing a button on the watch or wearable device.

[0062] The acquirer server 106 is associated with an acquirer, which may be an entity (e.g., a company or organization) that issues (e.g., establishes, manages, performs management of) a merchant's payment account (e.g., a financial bank account). Examples of acquirers include banks and / or other financial institutions. As discussed above, the acquirer server 106 may include one or more computing devices that are used to establish communication with another server (e.g., a transaction processing server 108) by exchanging messages with the other servers and / or passing information to the other servers. The acquirer server 106 forwards the payment transaction associated with the transaction request to the transaction processing server 108.

[0063] The transaction processing server 108 is configured to handle processes related to transaction accounts by, for example, forwarding data and information associated with the transaction to other servers in the system 100, such as the product matching server 140. In an example, instead of the requester device 102 or the provider device 104, the transaction processing server 108 may transmit data related to the product to the product matching server 140 to identify a match with the product before the transaction processing server 108 can process the transaction (e.g., a payment transaction for identification of the requested product).

[0064] The issuer server 110 is associated with an issuer and may include one or more computing devices for conducting payment transactions. The issuer may be an entity (e.g., a company or organization) that issues (e.g., establishes, manages, performs management of) transaction credentials or payment accounts (e.g., financial bank accounts) associated with the owner of the requester device 102. As discussed above, the issuer server 110 may include one or more computing devices for establishing communication with another server (e.g., transaction processing server 108) by exchanging messages with the other server and / or passing information to the other server.

[0065] The reference database 150 is a database or server associated with an entity (e.g., a company or organization) that manages (e.g., creates, performs management) data related to a plurality of products. In one arrangement, the data includes one or more entries for each of the plurality of products, each entry representing an attribute describing the product. The data may also include an identifier, such as a GTIN for each of the plurality of products. The product matching server 140 uses the data to identify matches with the products and then maps the products with the identified matching identifiers. The data may further include historical data related to the products, which the product matching server 140 may use to adjust the extracted one or more attributes related to the products to the same format as the format of one or more entries in the reference database 150.

[0066] Advantageously, the system 100 can efficiently and accurately identify matches with products and then assign corresponding identifiers (such as GTIN) to the products so that each product sold through the platform can be monitored, for example, in terms of sales performance under discounts, different seasons, months, specific holidays, and other similar scenarios, so that merchants can evaluate and adjust their marketing strategies. This identification can also be upgraded and expanded to easily identify multiple products without spending a lot of man-hours manually adding identifiers, such as GTIN, to each product.

[0067] Figure 2 A schematic diagram of a product matching server 140 according to various embodiments is illustrated. The product matching server 140 may include a data module 260 configured to receive data and information from a requester device 102, a provider device 104, a transaction processing server 108, the cloud, and other information sources to facilitate the product matching server 140 in identifying matches with products. For example, the data module 260 may be configured to receive data related to a product from the requester device 102, the provider device 104, the transaction processing server 108, and / or other information sources. The data module 260 may also be configured to receive historical data related to a product from the transaction processing server 108, the reference image database 150, the Internet, and / or other information sources. The data module 260 may be further configured to send information related to the identified match for the product, such as an identifier, item name, brand name, and / or other similar information obtained after the product matching process (e.g., information obtained based on one or more entries from the reference database 150 associated with the identified match), to the requester device 102, the provider device 104, the transaction processing server 108, or other destination requiring the information.

[0068] The product matching server 140 may include an extraction module 262 configured to extract one or more attributes related to the product from data indicated in the request for identifying a product, each of the one or more attributes describing the product. To address the challenges of unstructured and multimodal data that may be provided for identifying matches with products, multimodal attribute extraction may be employed to advantageously enable the extraction module 262 to extract one or more attributes related to the product from image information as well as common text information such as item names, descriptions, and other metadata.

[0069] The product matching server 140 may also include an injection module 264, which is configured to adjust the extracted one or more attributes to the same format as one or more entries in a database (e.g., a reference database 150) based on historical data related to the product, and the database includes one or more entries for each of the multiple products. For example, one or more attributes may be adjusted to at least one of the same unit of measurement, quantity, language, and item name as one or more entries in the reference database 150. In one implementation, domain knowledge injection may be used to fill the gap between the extracted attributes and one or more entries in the reference database 150, wherein the extracted attributes are enriched with historical data related to the product (such as domain knowledge related to the product, as well as labeled data and unlabeled data). The domain knowledge may include synonyms of product attributes in different languages, unit systems, or other similar formats.

[0070] The product matching server 140 may also include a matching module 266 configured to identify a match for the product from the plurality of products based on comparing the one or more attributes to each corresponding entry of the plurality of products in the reference database 150. Identifying a match for the product from the plurality of products may further include selecting one or more products from the plurality of products in the reference database 150 based on a rule-based comparison of at least one of the one or more attributes to each corresponding entry in the list, and identifying a match for the product from the selected one or more products based on comparing, by the processor, the one or more attributes to each corresponding entry of the selected one or more products.

[0071] In one implementation, the rule-based comparison may involve identifying one or more key attributes from the one or more attributes, and then selecting one or more products from a plurality of products in the reference database 150 based on comparing the one or more key attributes to each corresponding entry in the reference database 150 .

[0072] Comparing the one or more attributes to each corresponding entry of the selected one or more products may further include calculating a score for each of the one or more products based on the comparison, and identifying a match with the product when the score of the match is the highest score among the calculated scores and is greater than a threshold. Selecting one or more products from a plurality of products in the reference database 150 based on the rule-based comparison may further include identifying one or more key attributes from the one or more attributes, and selecting one or more products from a plurality of products in the reference database 150 based on the comparison of the one or more key attributes to each corresponding entry in the list.

[0073] In one implementation, based on the extracted attributes and injected domain knowledge of a given product, a rule-based filtering method is used to screen one or more matching candidates from the reference database 150 (e.g., one or more products selected based on comparison of one or more attributes with each corresponding entry in the database 150). For each pairing of a product with a matching candidate, the matching module 266 uses a machine learning model to use pairs of heterogeneous attributes as input to calculate a matching score as output. Among all pairings between the product and the matching candidate, if the highest score is greater than a given threshold, a match is identified based on the highest score, and the GTIN corresponding to the match is mapped or assigned to the product. Otherwise, the potential matching pair is considered a confusion case (e.g., a pair with a matching score above a given threshold but not the highest score) and will be sent to the platform's content operations for verification. Content operations' verification of confusion cases may include various methods, such as manual inspection by content operations staff, and / or through algorithms, machine learning models, and other similar methods.

[0074] The product matching server 140 may also include a retraining module 268 configured to retrain the matching module 266 based on verification of whether each product in the selected one or more products (excluding the identified matches) with a score greater than a threshold also matches the product. In one implementation, a regular cadence (e.g., daily, weekly, or monthly, depending on the application) may be set to retrain the matching module 266 based on historical mappings from the content operation of the platform and newly verified mappings (e.g., associations between products and their identified or verified matches in the reference database 150). This enables the matching module 266 to stay updated with the latest feedback and learn updated empirical rules set by the content operation. Advantageously, the verified mappings may also help enrich the original records in the reference database 150 by adding additional attributes (such as a typical image of the product or a local colloquial name of the product), which are adjusted as new entries in the reference database 150.

[0075] Each of the data module 260, the extraction module 262, the injection module 264, the matching module 266, and the retraining module 268 may further communicate with a processing module (not shown) of the product matching server 140, for example, to coordinate corresponding tasks and functions in the product matching process. The data module 260 may be further configured to communicate with and store data and information for each of the processing module, the extraction module 262, the injection module 264, the matching module 266, and the retraining module 268. Alternatively, all tasks and functions required to identify matches with products may be performed by a single processor of the product matching server 140.

[0076] Figure 3An overview of a process 300 for identifying a match with a product is depicted in accordance with various embodiments. In a first stage, a multimodal extraction is implemented (e.g., by an extraction module 262) for extracting one or more attributes of a product from data indicated in a request for identifying a product. The request for identifying a product may be a direct request sent from a requestor device 102 or a provider device 104, or may be part of a process by which a merchant's product is added to the platform so that the product may subsequently be viewed and purchased by users of the platform. The data from which the attributes are extracted may be in various formats, such as text and image formats.

[0077] The data obtained from merchants for attribute extraction is usually incomplete, noisy, and prone to errors. In addition, depending on the product type, there may be thousands of different types of attributes (e.g., sugar content, flavor, color, quantity, capacity, and other similar attributes). To address these challenges, multimodal attribute extraction methods based on the visual question answering (VQA) paradigm can be utilized, such as Figure 4 As shown in the process 400. For example, given Figure 6 The image 604 and text 606 shown are used as context. We can extract the information of 'capacity' by asking "What is the capacity?" as question 402 to the VQA model 406 in combination with the context 404, and the returned answer 416 will be the extracted attribute value, such as "320Ml". There are three types of embeddings, namely question embedding 408, text embedding 410 and image embedding 412, and the VQA model 406 can perform its attribute extraction based on these three types of embeddings. The question embedding 408 can be directly converted from the attribute type string of the question "What is the capacity?", which is 'capacity' in this example. For the context of the item, the text embedding 410 can be composed of the item name, description and text, which can be obtained from the image and text data related to the product by using optical character recognition (OCR) technology. The image embedding 412 can be generated from the image block, which is obtained by processing the image data related to the product using object and landmark detection technology. These three embeddings can be used as input to obtain the answer 416 of the question 402 by processing the model (such as the hierarchical neural network model 414).

[0078] The multimodal attribute extraction process 302 using both images and text is different from existing methods that rely on Tf-idf or word embedding methods on text or focus only on using English for attribute extraction. A multilingual transformer can be implemented in the multimodal attribute extraction process 302 to interpret text information, and a multilingual module can be developed and integrated to interpret multilingual OCR from images. For example, the extraction module 262 can be fine-tuned to adjust to regional languages ​​such as Malay, Thai, and Vietnamese, making it possible to extract attributes from information in these languages.

[0079] Prior to comparison with entries in the reference database 150, the extracted attributes (example being answers 416 from the VQA model 406) may need to be interpreted and normalized to ensure that the information in both the extracted attributes and the corresponding entries in the reference database 150 is normalized to the same format. Therefore, the extracted attributes undergo a domain knowledge injection process 304 (e.g., implemented by the injection module 264), in which one or more attributes are adjusted to the same format as the format of one or more entries in the reference database 150 based on historical data related to the product. How each extracted attribute is adjusted will depend on the type of the attribute in question. Some extracted attributes may be unit converted based on domain knowledge rules to convert the values ​​and units of these attributes to be consistent with the form of the corresponding entries in the reference database 150. For example, where the reference database 150 includes entries in milliliters for volume, grams for weight, and centimeters for length, extracted attributes in fluid ounces may be converted to milliliters for volume, extracted attributes in pounds may be converted to grams for weight, and extracted attributes in inches may be converted to centimeters for length. Some of the extracted attributes may be quantity normalized, where a set of rules are formulated for the domain knowledge injection process 304 to extract and calculate the quantity of items from common phrases used in product descriptions that may be present in data related to the product. For example, a rule may be set for the domain knowledge injection process 304 that the phrase "buy 1 get 1 free" will be equivalent to a quantity of 2, and therefore the quantity attribute associated with the phrase is converted to 2 in the domain knowledge injection process 304. Further, some of the extracted attributes may be synonym normalized. For example, the platform's content operations may maintain a synonym database of brand and product attribute synonyms for different languages. The algorithm utilized by the domain knowledge injection process 304 may reference the synonym database to normalize the extracted brand names and attributes to be consistent with the corresponding entries in the reference database 150. Incorporating and injecting historical data related to the product (such as domain knowledge) to fill the gap between the extracted attributes and the entries in the reference database 150 advantageously enables the system to accurately interpret and compare the extracted attributes across different sources or regions.

[0080] After the domain knowledge injection process 304 is completed, the extracted and standardized attributes are subjected to an adaptive matching process 306 (e.g., implemented by the matching module 368), wherein a match with the product is identified from a plurality of products in the reference database 150, and the identification is based on a comparison of one or more attributes with each corresponding entry of the plurality of products. The identification can be based on a two-stage hybrid framework to advantageously reduce computational complexity. In the first stage, a candidate list (e.g., potential matches with related products) can be selected from a plurality of products in the reference database 150. For example, one or more products can be selected from a plurality of products in the reference database 150 based on a rule-based comparison of at least one of the one or more attributes with each corresponding entry in the reference database 150. The rule-based comparison can be implemented such that one or more key attributes can be identified from the one or more attributes, and then one or more products can be selected from a plurality of products in the reference database 150 based on a comparison of the one or more key attributes with each corresponding entry in the reference database 150. The key attributes to be identified from the one or more attributes can be set by the content operation of the relevant platform, or determined by the matching module 368 based on, for example, historical data from past comparisons. In the second stage, a computationally expensive neural network model may be implemented to further process the shortlisted candidates with high accuracy by comparing one or more extracted attributes to each corresponding entry of the shortlisted candidates to identify matches with relevant products. A score may be calculated for each of the shortlisted candidates based on the comparison, and a match with a product may be identified when the match score is the highest score among the calculated scores and is greater than a threshold. The threshold may be set by the content operation of the relevant platform, or determined by the matching module 368 based on historical data from past comparisons, for example.

[0081] exist Figure 5 An exemplary implementation of the two-stage hybrid framework is shown in FIG500. Referring to Table 502, the relevant products to be identified and matched can be represented as q, and the attributes of the relevant products extracted after knowledge injection can be represented as A 1,2,…,n , where n is the number of attributes extracted. Similarly, referring to Table 504, multiple products on the reference database 150 can be represented as c 1,2,…,m , whose corresponding attribute is A 1,2,…,n , where m is the row number of table 504. Rule-based matching will help to identify products based on related products q and multiple products c. 1,2,…,m The k pairs of candidates are filtered out by matching key attributes (such as brand, size, quantity, and other attributes) between them. potential matches, as shown in Table 506. This screening process advantageously helps to reduce the computational load of the next stage. For each pair, product q and multiple shortlisted candidates Together with their attributes, they will be fed into the multi-mode converter based neural network model 508 to calculate their final matching scores (s 1,2,…,k ), as shown in Table 510. Based on the matching score, when the matching score is the highest score among the calculated scores and is greater than the threshold, for example, as in Figure 3 As shown in process 308, the matching module 368 can select the shortlisted candidates. The matching module 368 may also proceed to map the product q with the identified matching identifier (such as a GTIN).

[0082] For the remaining shortlisted candidates, for example, with scores greater than a threshold but not the highest score, these potential pairs may be forwarded to the content operations of the relevant platform for further verification, e.g. Figure 3 As shown in the process 310 of . In the process 310, each product in the potential pair (e.g., the selected one or more products other than the identified match with a score greater than a threshold) is verified by the content operation whether it also matches the product. The verification of the potential pair by the content operation may include various methods, such as manual inspection by the staff of the content operation, and / or via algorithms, machine learning models and other similar methods. Based on the verification, the matching module 266 is retrained. This can be referred to as an adaptive data flywheel, which advantageously enables the matching module 266 to automatically adapt to changes in, for example, the reference database 150 and based on feedback from the content operation of the relevant platform. This can be performed regularly (e.g., daily, weekly or monthly, depending on the application), and enables the reference database 150 to be enriched with additional information of the verified matching pairs from the content operation while retraining the matching module 266. During the retraining of the matching model, a difficult negative sample pair mining scheme can be used to sample those uncertain or incorrectly identified results based on the feedback of the content operation, and thus the matching module 266 can learn such feedback during the retraining and improve its accuracy in future identifications.

[0083] Figure 7 An exemplary flow chart of a method for identifying a match with a product according to various embodiments is illustrated. In step 702, one or more attributes related to the product are extracted from data indicated in a request for identifying a product, each of the one or more attributes describing the product. In step 704, the one or more attributes are adjusted to a format that is the same as a format of one or more entries in a database based on historical data related to the product, the database including one or more entries for each of a plurality of products. In step 706, a match with the product is identified from the plurality of products, the identification being based on comparing, by a processor, the one or more attributes to each corresponding entry for the plurality of products.

[0084] Fig. 8A and Figure 8B A schematic block diagram of a general purpose computer system on which the Figure 1 Transaction processing server.

[0085] like Fig. 8A As shown, computer system 1300 includes computer module 1301. Computer module 1301 can communicate with communication network 1320 via connection 1321 using external modulator-demodulator (modem) transceiver device 1316. Communication network 1320 can be a wide area network (WAN), such as the Internet, a cellular telecommunications network, or a dedicated WAN. In the case where connection 1321 is a telephone line, modem 1316 can be a traditional "dial-up" modem. Alternatively, in the case where connection 1321 is a high-capacity (e.g., cable) connection, modem 1316 can be a broadband modem. Wireless modems can also be used for wireless connections to communication network 1320.

[0086] The computer module 1301 typically includes at least one processor unit 1305, and a memory unit 1306. For example, the memory unit 1306 may have a semiconductor random access memory (RAM) and a semiconductor read-only memory (ROM). The computer module 1301 also includes an interface 1308 for an external modem 1316. In some implementations, the modem 1316 may be incorporated into the computer module 1301, for example, into the interface 1308. The computer module 1301 also has a local network interface 1311 that allows the computer system 1300 to be coupled to a local area communication network 1322, known as a local area network (LAN), via a connection 1323. As shown in FIG. Fig. 8A As shown, the local communication network 1322 may also be coupled to the wide area network 1320 via a connection 1324, which typically includes a so-called "firewall" device or a device with similar functionality. The local network interface 1311 may include an Ethernet circuit card, wireless arrangement or IEEE 802.11 wireless arrangement; however, many other types of interfaces may be practiced for interface 1311.

[0087] I / O interface 1308 may provide either or both serial and parallel connections, the former typically implemented according to the Universal Serial Bus (USB) standard and having a corresponding USB connector (not shown). Storage device 1309 is provided, which typically includes a hard disk drive (HDD) 1310. Other storage devices such as floppy disk drives and tape drives (not shown) may also be used. An optical disk drive 1312 is typically provided as a non-volatile data source. Portable storage devices, such as optical disks, USB-RAM, portable external hard disk drives, and floppy disks may be used as suitable data sources for system 1300.

[0088] The components 1305 to 1312 of the computer module 1301 typically communicate via an interconnect bus 1304 and communicate in a conventional mode of operation of the computer system 1300 known to those skilled in the art. For example, the processor 1305 is coupled to the system bus 1304 using connection 1318. Similarly, the memory 1306 and the optical drive 1312 are coupled to the system bus 1304 via connection 1319. Examples of computers that can implement the described arrangements include IBM-PCs and compatibles, Sun Sparcstations, Apple or similar computer systems.

[0089] Facilitated by the transaction processing server 108 Figure 7 The steps of method 700 in the method 700 may be implemented using computer system 1300. For example, the steps of method 700 may be implemented as one or more software applications 1333 that may be executed within computer system 1300. Specifically, the steps of method 700 facilitated by transaction processing server 108 are executed by instructions 1331 in software 1333 executed within computer system 1300 (see Figure 8B ) is implemented. The software instructions 1331 may be formed into one or more code modules, each of which is used to perform one or more specific tasks. The software may also be divided into two separate parts, where a first part and corresponding code modules facilitate the steps of method 700, and a second part and corresponding code modules manage the user interface between the first part and the user.

[0090] The software may be stored in a computer readable medium, including, for example, the storage devices described below. The software is loaded into the computer system 1300 from the computer readable medium and then executed by the computer system 1300. A computer readable medium having such software or a computer program recorded on a computer readable medium is a computer program product. The use of the computer program product in the computer system 1300 preferably produces an advantageous arrangement for the transaction processing server 108.

[0091] The software 1333 is typically stored in the HDD 1310 or the memory 1306. The software is loaded into the computer system 1300 from a computer readable medium and executed by the computer system 1300. Thus, for example, the software 1333 may be stored on an optically readable disk storage medium (e.g., a CD-ROM) 1325 read by the optical drive 1312. A computer readable medium having such software or a computer program recorded thereon is a computer program product. Use of the computer program product in the computer system 1300 preferably produces an apparatus for the transaction processing server 108.

[0092] In some examples, the application 1333 may be supplied to the user by being encoded on one or more CD-ROMs 1325 and read via the corresponding drive 1312, or alternatively may be read by the user from the network 1320 or 1322. Further, the software may also be loaded into the computer system 1300 from other computer-readable media. Computer-readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and / or data to the computer system 1300 for execution and / or processing. Examples of such storage media include floppy disks, magnetic tapes, optical disks, hard disk drives, ROMs or integrated circuits, USB memories, magneto-optical disks, or computer-readable cards such as PCMCIA cards, whether these devices are located inside or outside the computer module 1301. Examples of temporary or non-tangible computer-readable transmission media that may also be involved in providing software, application programs, instructions and / or data to the computer module 1301 include radio or infrared transmission channels and network connections to another computer or networked device, as well as the Internet or intranet including email transmissions and information recorded on websites, etc.

[0093] The second portion of the executable application 1333 and the corresponding code modules described above can implement one or more graphical user interfaces (GUIs) rendered or otherwise represented on a display. Through manipulation, typically a keyboard and a mouse, users and applications of the computer system 1300 can manipulate the interface in a functionally adaptable manner to provide control commands and / or input to applications associated with the GUI. Other forms of functionally adaptable user interfaces can also be implemented, such as audio interfaces that utilize voice prompts output via a speaker and user voice commands input via a microphone.

[0094] Figure 8B is a detailed schematic block diagram of the processor 1305 and the "memory" 1334. The memory 1334 represents the memory that can be used by Fig. 8A A logical aggregation of all memory modules (including HDD 1309 and semiconductor memory 1306) accessed by the computer module 1301 in.

[0095] When computer module 1301 is initially powered on, a power-on self-test (POST) program 1350 is executed. POST program 1350 is typically stored in Fig. 8A 1349 of semiconductor memory 1306. Hardware devices such as ROM 1349 that store software are sometimes referred to as firmware. POST program 1350 checks the hardware within computer module 1301 to ensure that it is functioning properly, and typically checks that processor 1305, memory 1334 (1309, 1306), and basic input / output system software (BIOS) module 1351 (also typically stored in ROM 1349) are operating correctly. Once POST program 1350 runs successfully, BIOS 1351 activates Fig. 8A Activation of the hard disk drive 1310 causes the boot loader 1352 resident on the hard disk drive 1310 to be executed via the processor 1305. This loads the operating system 1353 into the RAM memory 1306, on which the operating system 1353 begins to operate. The operating system 1353 is a system-level application that can be executed by the processor 1305 to implement various high-level functions, including processor management, memory management, device management, storage management, software application interfaces, and general user interfaces.

[0096] The operating system 1353 manages the memory 1334 (1309, 1306) to ensure that each process or application running on the computer module 1301 has enough memory to execute and does not conflict with the memory allocated to another process. Fig. 8A The different types of memory available in the computer system 1300 must be used appropriately so that each process can run efficiently. Therefore, the aggregate memory 1334 is not intended to illustrate how specific segments of memory are allocated (unless otherwise specified), but rather to provide a general view of the memory accessible by the computer system 1300 and how it is used.

[0097] like Figure 8B As shown, the processor 1305 includes multiple functional modules, including a control unit 1339, an arithmetic logic unit (ALU) 1340, and a local or internal memory 1348, sometimes referred to as a cache memory. The cache memory 1348 typically includes a plurality of storage registers 1344-1346 in a register section. One or more internal buses 1341 functionally interconnect these functional modules. The processor 1305 also typically has one or more interfaces 1342 for communicating with external devices via the system bus 1304 using a connection 1318. The memory 1334 is coupled to the bus 1304 using a connection 1319.

[0098] The application program 1333 includes a series of instructions 1331, which may include conditional branching and looping instructions. The program 1333 may also include data 1332 used when executing the program 1333. The instructions 1331 and the data 1332 are stored in memory locations 1328, 1329, 1330 and 1335, 1336, 1337, respectively. Depending on the relative size of the instructions 1331 and the memory locations 1328-1330, a particular instruction may be stored in a single memory location, as depicted by the instruction shown in memory location 1330. Alternatively, the instruction may be split into multiple parts, each of which is stored in a separate memory location, as depicted by the instruction segments shown in memory locations 1328 and 1329.

[0099] Typically, the processor 1305 is given a set of instructions to execute within it. The processor 1305 waits for a subsequent input, to which the processor 1305 reacts by executing another set of instructions. Each input may be provided by one or more of a number of sources, including data generated by one or more input devices 1302, 1303, data received from an external source via one of the networks 1320, 1302, data retrieved from one of the storage devices 1306, 1309, or data retrieved from a storage medium 1325 inserted into a corresponding reader 1312, all of which are stored in a separate computer. Fig. 8A Execution of a set of instructions may, in some cases, result in the output of data. Execution may also involve storing data or variables to memory 1334.

[0100] The disclosed transaction processing server 108 arrangement uses input variables 1354, which are stored in corresponding memory locations 1355, 1356, 1357 in the memory 1334. The transaction processing server 108 arrangement produces output variables 1361, which are stored in corresponding memory locations 1362, 1363, 1364 in the memory 1334. Intermediate variables 1358 may be stored in memory locations 1359, 1360, 1366, and 1367.

[0101] refer to Figure 8B The processor 1305, registers 1344, 1345, 1346, arithmetic logic unit (ALU) 1340, and control unit 1339 work together to perform a series of micro-operations required to "fetch, decode, and execute" cycles for each instruction in the instruction set that makes up program 1333.

[0102] Each fetch, decode, and execute cycle consists of:

[0103] A fetch operation, fetching or reading instruction 1331 from memory locations 1328, 1329, 1330;

[0104] A decode operation, wherein the control unit 1339 determines which instruction has been fetched; and

[0105] An execution operation is performed, wherein the control unit 1339 and / or the ALU 1340 executes instructions.

[0106] Thereafter, a further fetch, decode and execute cycle for the next instruction may be performed.Similarly, a store cycle may be performed by which the control unit 1339 stores or writes a value to the memory location 1332 .

[0107] Each step or sub-process in the process performed by the transaction processing server 108 is associated with one or more segments of the program 1333 and is performed jointly by the register parts 1344, 1345, 1347, ALU 1340 and control unit 1339 in the processor 1305 to obtain, decode and execute each instruction in the instruction set of the said segment of the program 1333.

[0108] It should be understood that the architectural context of the computer system 1300 (i.e., the transaction processing server 108) is presented by way of example only. Thus, in some arrangements, one or more features of the server 1300 may be omitted. Furthermore, in some arrangements, one or more features of the server 1300 may be combined together. Additionally, in some arrangements, one or more features of the server 1300 may be separated into one or more component parts.

[0109] Fig. 9 An alternative implementation of a transaction processing server 108 (i.e., a computer system 1300) is shown. In an alternative implementation, the transaction processing 108 can be generally described as a physical device including at least one processor 802 and at least one memory 804 including computer program code. The at least one memory 804 and the computer program code are configured to, together with the at least one processor 802, cause the transaction processing server 108 to facilitate the operations described in the method 700. The transaction processing server 108 may also include a transaction processing module 806. The memory 804 stores computer program code compiled by the processor 802 to cause the transaction processing module 806 to perform corresponding functions.

[0110] refer to Figure 1, the transaction processing module 806 performs the function of communicating with the requester device 102 and the provider device 104; and the acquirer server 106 and the issuer server 110, respectively, to receive and transmit transactions, travel request messages, requests for identifying matches with products, requests to add products to the relevant platform, or other similar messages. Further, the transaction processing module 806 can provide data and information related to the product to the product matching server 140, such as to identify matches with products before the transaction (payment transaction for requesting identification of the product) can be processed by the transaction processing server 108, or before the product can be added to the platform.

[0111] Figure 8C A general purpose computer system 1400 is depicted on which the described product matching server 140 can be practiced. The computer system 1400 includes a computer module 1401. The computer module 1401 can communicate with a communication network 1420 via a connection 1421 using an external modulator-demodulator (modem) transceiver device 1416. The communication network 1420 can be a wide area network (WAN) such as the Internet, a cellular telecommunications network, or a dedicated WAN. When the connection 1421 is a telephone line, the modem 1416 can be a traditional "dial-up" modem. Alternatively, when the connection 1421 is a high capacity (e.g., cable) connection, the modem 1416 can be a broadband modem. A wireless modem can also be used to connect wirelessly to the communication network 1420.

[0112] The computer module 1401 typically includes at least one processor unit 1405, and a memory unit 1406. For example, the memory unit 1406 may have a semiconductor random access memory (RAM) and a semiconductor read-only memory (ROM). The computer module 1401 also includes an interface 1408 for an external modem 1416. In some implementations, the modem 1416 may be incorporated into the computer module 1401, for example, into the interface 1408. The computer module 1401 also has a local network interface 1411 that allows the computer system 1400 to be coupled to a local area communication network 1422, known as a local area network (LAN), via a connection 1423. As shown in FIG. Figure 8C As shown, the local communication network 1422 may also be coupled to the wide area network 1420 via a connection 1424, which typically includes a so-called "firewall" device or a device with similar functions. The local network interface 1411 may include an Ethernet circuit card, wireless arrangement or IEEE 802.11 wireless arrangement; however, many other types of interfaces may be practiced for interface 1411.

[0113] I / O interface 1408 may provide either or both of serial and parallel connections, the former typically implemented according to the Universal Serial Bus (USB) standard and having a corresponding USB connector (not shown). Storage device 1409 is provided, which typically includes a hard disk drive (HDD) 1410. Other storage devices, such as floppy disk drives and tape drives (not shown) may also be used. An optical disk drive 1412 is typically provided as a non-volatile data source. Portable storage devices, such as optical disks, USB-RAM, portable external hard disk drives, and floppy disks, may be used as suitable data sources for system 1400.

[0114] The components 1405 to 1412 of the computer module 1401 typically communicate via an interconnect bus 1404 and communicate in a conventional mode of operation of the computer system 1400 known to those skilled in the art. For example, the processor 1405 is coupled to the system bus 1404 using connection 1418. Similarly, the memory 1406 and the optical drive 1412 are coupled to the system bus 1404 via connection 1419. Examples of computers that can implement the described arrangement include IBM-PC and compatible machines, Sun Sparcstations, Apple or similar computer systems.

[0115] Method 700 (performed by product matching server 140) may be implemented using computer system 1400. These processes may be implemented as one or more software applications 1433 executable within computer system 1400. Specifically, subprocesses 400, 500, and 600 are executed by instructions in software 1433 executed within computer system 1400 (see Figure 8B The software instructions may be formed into one or more code modules, each of which is used to perform one or more specific tasks. The software may also be divided into two separate parts, wherein a first part and corresponding code modules perform the method, and a second part and corresponding code modules manage a user interface between the first part and a user.

[0116] The software may be stored in a computer readable medium, such as a storage device described below. The software is loaded into the computer system 1400 from the computer readable medium and then executed by the computer system 1400. A computer readable medium having such software or a computer program recorded on a computer readable medium is a computer program product. The use of a computer program product in the computer system 1400 preferably implements an advantageous arrangement for the product matching server 140.

[0117] The software 1433 is typically stored in the HDD 1410 or the memory 1406. The software is loaded into the computer system 1400 from a computer readable medium and executed by the computer system 1400. Thus, for example, the software 1433 may be stored on an optically readable disk storage medium (e.g., a CD-ROM) 1425 read by the optical drive 1412. A computer readable medium having such software or a computer program recorded thereon is a computer program product. The use of the computer program product in the computer system 1400 preferably implements an apparatus for the product matching server 140.

[0118] In some examples, the application 1433 may be supplied to the user by being encoded on one or more CD-ROMs 1425 and read via corresponding drives 1412, or alternatively may be read by the user from a network 1420 or 1422. Further, the software may also be loaded into the computer system 1400 from other computer-readable media. Computer-readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and / or data to the computer system 1400 for execution and / or processing. Examples of such storage media include floppy disks, magnetic tapes, optical disks, hard disk drives, ROMs or integrated circuits, USB memories, magneto-optical disks, or computer-readable cards such as PCMCIA cards, whether these devices are located inside or outside the computer module 1401. Examples of temporary or non-tangible computer-readable transmission media that may also be involved in providing software, application programs, instructions and / or data to the computer module 1401 include radio or infrared transmission channels and network connections to another computer or network device, as well as the Internet or intranet including email transmissions and information recorded on websites, etc.

[0119] The second portion of the executable application 1433 and the corresponding code modules described above can implement one or more graphical user interfaces (GUIs) rendered or otherwise represented on a display. Through the manipulation of a conventional keyboard and mouse, users and applications of the computer system 1400 can manipulate the interface in a functionally adaptable manner to provide control commands and / or input to applications associated with the GUI. Other forms of functionally adaptable user interfaces can also be implemented, such as audio interfaces that utilize voice prompts output via a speaker and user voice commands input via a microphone.

[0120] It should be understood that the architectural context of computer system 1400 (i.e., product matching server 140) is presented by way of example only. Thus, in some arrangements, one or more features of computer system 1400 may be omitted. Also, in some arrangements, one or more features of computer system 1400 may be combined together. Additionally, in some arrangements, one or more features of computer system 1400 may be split into one or more component parts.

[0121] Fig.10 An alternative implementation of the product matching server 140 (i.e., computer system 1400) is shown. In an alternative implementation, the product matching server 140 can be generally described as a physical device that includes at least one processor 902 and at least one memory 904 including computer program code. The at least one memory 904 and the computer program code are configured to, together with the at least one processor 902, cause the product matching server 140 to perform the operations described in the method 700. The product matching server 140 may also include a data module 906, an extraction module 908, an injection module 910, a matching module 912, and a retraining module 914. The memory 904 stores computer program code compiled by the processor 902 to cause each of the modules 906 to 914 to perform its respective functions.

[0122] refer to Figures 1 to 7 The extraction module 908 performs the function of extracting one or more attributes related to the product from the data indicated in the request for identifying the product, each of the one or more attributes describing the product.

[0123] refer to Figures 1 to 7 The injection module 910 performs a function of adjusting one or more attributes to the same format as one or more entries in a database (e.g., reference database 150) based on historical data related to the product, the database including one or more entries for each of a plurality of products. The one or more attributes may be adjusted to at least one of the same unit of measurement, quantity, language, and item name as the one or more entries in the database.

[0124] See also Figures 1 to 7, the matching module 912 identifies a match with the product from a plurality of products, the identification being based on a comparison of one or more attributes with each corresponding entry of the plurality of products. Identifying a match with the product from a plurality of products may further include: selecting one or more products from a plurality of products in a database based on a rule-based comparison of at least one of the one or more attributes with each corresponding entry in a database; and identifying a match with the product from the selected one or more products, the identification being based on a comparison of the one or more attributes with each corresponding entry of the selected one or more products. The comparison of the one or more attributes with each corresponding entry of the selected one or more products may further include calculating a score for each of the selected one or more products based on the comparison; and identifying a match with the product when the score of the match is the highest score among the calculated scores and is greater than a threshold. Further, selecting one or more products from a plurality of products in a database based on the rule-based comparison may further include identifying one or more key attributes from the one or more attributes; and selecting one or more products from a plurality of products in a database based on a comparison of the one or more key attributes with each corresponding entry in the database. The matching module 912 may also perform a function of mapping products with the identified matching identifiers.

[0125] Reference Figures 1 to 7 , the retraining module 914 performs the functionality of the retraining matching module 912 based on verifying whether each of the selected one or more products other than the identified match having a score greater than a threshold is also a match for the product.

[0126] Reference Figures 1 to 7 , the data module 906 performs the functions of receiving data and information from the requester device 102, the provider device 104, the transaction processing server 108, the reference database 150, the cloud, the Internet, and other information sources to facilitate the method 700. For example, the data module 906 can be configured to receive data related to the product from the requester device 102, the provider device 104, the transaction processing server 108, and / or other information sources. The data module 260 can also be configured to receive historical data related to the product from the transaction processing server 108, the reference image database 150, the Internet, and / or other information sources. The data module 260 can be further configured to send information related to the identified match of the product, such as an identifier, an item name, a brand name, and / or other similar information obtained after the product matching process (e.g., information obtained based on one or more entries from the reference database 150 associated with the identified match) to the requester device 102, the provider device 104, the transaction processing server 108, or other destinations that require the information.

[0127] Fig.8DA general purpose computer system 1500 is depicted on which the described combined transaction processing server 108 and product matching server 140 can be practiced. The computer system 1500 includes a computer module 1501. The computer module 1501 can communicate with a communication network 1520 via a connection 1521 using an external modulator-demodulator (modem) transceiver device 1516. The communication network 1520 can be a wide area network (WAN) such as the Internet, a cellular telecommunications network, or a dedicated WAN. In the case where the connection 1521 is a telephone line, the modem 1516 can be a traditional "dial-up" modem. Alternatively, in the case where the connection 1521 is a high capacity (e.g., cable) connection, the modem 1516 can be a broadband modem. A wireless modem can also be used for wireless connection to the communication network 1520.

[0128] The computer module 1501 typically includes at least one processor unit 1505 and a memory unit 1506. For example, the memory unit 1506 may have a semiconductor random access memory (RAM) and a semiconductor read-only memory (ROM). The computer module 1501 also includes an interface 1508 for an external modem 1516. In some implementations, the modem 1516 may be incorporated into the computer module 1501, for example, into the interface 1508. The computer module 1501 also has a local network interface 1511, which allows the computer system 1500 to be coupled to a local area communication network 1522, known as a local area network (LAN), via a connection 1523. As shown in FIG. Fig.8D As shown in FIG. 1 , the local communication network 1522 may also be coupled to the wide area network 1520 via a connection 1524, which typically includes a so-called "firewall" device or a device with similar functionality. The local network interface 1511 may include an Ethernet circuit card, wireless arrangement or IEEE 802.11 wireless arrangement; however, various other types of interfaces may also be practiced for interface 1511.

[0129] I / O interface 1508 may provide either or both serial and parallel connections, the former typically implemented in accordance with the Universal Serial Bus (USB) standard and having a corresponding USB connector (not shown). Storage device 1509 is provided, which typically includes a hard disk drive (HDD) 1510. Other storage devices such as floppy disk drives and tape drives (not shown) may also be used. An optical disk drive 1512 is typically provided as a non-volatile data source. Portable storage devices, such as optical disks, USB-RAM, portable external hard disk drives, and floppy disks, may be used as suitable data sources for system 1500.

[0130] The components 1505 to 1512 of the computer module 1501 typically communicate via an interconnect bus 1504 and communicate in a conventional mode of operation of the computer system 1500 known to those skilled in the art. For example, the processor 1505 is coupled to the system bus 1504 using connection 1518. Similarly, the memory 1506 and the optical drive 1512 are coupled to the system bus 1504 via connection 1519. Examples of computers on which the described arrangements may be implemented include IBM-PCs and compatibles, Sun Sparcstations, Apple or similar computer systems.

[0131] The steps of method 700 performed by product matching server 140 and facilitated by transaction processing server 108 may be implemented using computer system 1500. For example, the steps of method 700 performed by product matching server 140 may be implemented as one or more software applications 1533 executable within computer system 1500. Specifically, the steps of method 700 are executed by instructions in software 1533 executed within computer system 1500 (see Figure 8B The software instructions may be formed into one or more code modules, each of which is used to perform one or more specific tasks. The software may also be divided into two separate parts, wherein the first part and the corresponding code modules perform the steps of method 700, and the second part and the corresponding code modules manage the user interface between the first part and the user.

[0132] The software may be stored in a computer readable medium, including, for example, a storage device described below. The software is loaded from the computer readable medium into the computer system 1500 and then executed by the computer system 1500. A computer readable medium having such software or a computer program recorded on a computer readable medium is a computer program product. The use of the computer program product in the computer system 1500 preferably implements an advantageous means for combining a transaction processing and product matching server.

[0133] The software 1533 is typically stored in the HDD 1510 or the memory 1506. The software is loaded into the computer system 1500 from a computer readable medium and executed by the computer system 1500. Thus, for example, the software 1533 may be stored on an optically readable disk storage medium (e.g., a CD-ROM) 1525 read by the optical drive 1512. A computer readable medium having such software or a computer program recorded thereon is a computer program product. The use of the computer program product in the computer system 1500 preferably implements an apparatus for a combined transaction processing and product matching server.

[0134] In some examples, the application 1533 may be supplied to the user by being encoded on one or more CD-ROMs 1525 and read via corresponding drives 1512, or alternatively may be read by the user from a network 1520 or 1522. Further, the software may also be loaded into the computer system 1500 from other computer-readable media. Computer-readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and / or data to the computer system 1500 for execution and / or processing. Examples of such storage media include floppy disks, magnetic tapes, optical disks, hard drives, ROMs or integrated circuits, USB memories, magneto-optical disks, or computer-readable cards such as PCMCIA cards, whether such devices are located internal or external to the computer module 1501. Examples of temporary or non-tangible computer-readable transmission media that may also be involved in providing software, application programs, instructions and / or data to the computer module 1501 include radio or infrared transmission channels and network connections to another computer or networked device, as well as the Internet or intranet including email transmissions and information recorded on websites, etc.

[0135] The second portion of the executable application 1533 and the corresponding code modules described above can implement one or more graphical user interfaces (GUIs) rendered or otherwise represented on a display. Through the manipulation of a conventional keyboard and mouse, users and applications of the computer system 1500 can manipulate the interface in a functionally adaptable manner to provide control commands and / or input to applications associated with the GUI. Other forms of functionally adaptable user interfaces can also be implemented, such as audio interfaces that utilize voice prompts output via a speaker and user voice commands input via a microphone.

[0136] It should be understood that the architectural context of computer system 1500 (i.e., combined transaction processing and product matching server 1500) is presented by way of example only. Thus, in some arrangements, one or more features of server 1500 may be omitted. Moreover, in some arrangements, one or more features of server 1500 may be combined together. Additionally, in some arrangements, one or more features of server 1500 may be separated into one or more component parts.

[0137] Fig.11An alternative implementation of a combined transaction processing and product matching server (i.e., computer system 1500) is shown. In an alternative implementation, the combined transaction processing and product matching server may generally be described as a physical device including at least one processor 1002 and at least one memory 904 including computer program code. The at least one memory 1004 and the computer program code are configured to, together with the at least one processor 1002, cause the combined transaction processing and product matching server to perform the operations described in the steps of method 700. The combined transaction processing and product matching server may also include a transaction processing module 806, a data module 906, an extraction module 908, an injection module 910, a matching module 912, and a retraining module 914. The memory 1004 stores computer program code compiled by the processor 1002 to cause each of the modules 806 to 912 to perform its respective functions. The transaction processing module 806 performs operations related to the processing of the transaction processing and product matching server. Fig. 9 The data module 906, the extraction module 908, the injection module 910, the matching module 912 and the retraining module 914 perform the same functions as those described for the same transaction processing module in FIG. Fig.10 The same functions are described in the same corresponding modules.

[0138] Those skilled in the art will appreciate that, without departing from the broad scope of this specification, many changes and / or modifications may be made to the present disclosure as shown in the specific embodiments. Therefore, the present embodiments are to be considered in all aspects as illustrative and not restrictive.

Claims

1. A method for identifying a match with a product, include: extracting, by a processor, one or more attributes related to the product from data indicated in the request for identifying the product, each of the one or more attributes describing the product, wherein the data includes an image of the product, and extracting the one or more attributes includes identifying the one or more attributes from the image based on a visual question answering (VQA) paradigm; adjusting, by the processor, the one or more attributes to a format identical to a format of one or more entries in a database based on historical data related to the product, the database including one or more entries for each product in a plurality of products; identifying a match to the product from the plurality of products, the identifying based on comparing, by the processor, the one or more attributes to each corresponding entry of the plurality of products; selecting one or more products from the plurality of products in the database based on a rule-based comparison of at least one of the one or more attributes with each corresponding entry in the database; identifying a match to the product from the selected one or more products, the identifying based on comparing, by the processor, the one or more attributes to each corresponding entry of the selected one or more products; calculating a score for each of the selected one or more products based on the comparison; When the score of the match is the highest score among the calculated scores and is greater than a threshold, identifying the match with the product; verifying that each of the selected one or more products other than the identified match having a score greater than the threshold also matches the product; and The processor is retrained based on the verification.

2. The method according to claim 1, in, The rule-based comparison to select one or more products from the plurality of products in the database further comprises: identifying one or more key attributes from the one or more attributes; and The one or more products are selected from the plurality of products in the database based on a comparison of the one or more key attributes to each corresponding entry in the database.

3. The method according to claim 1, further comprising: include: The one or more attributes are adjusted, by the processor, to be at least one of a same unit of measurement, quantity, language, and item name as one or more entries in the database.

4. The method according to claim 1, in, The database further indicates an identifier for each product of the plurality of products, the method further comprising mapping the products with the identified matching identifiers.

5. A system for identifying a match with a product, include: at least one processor; as well as at least one memory including computer program code; The at least one memory and the computer program code are configured to: with the at least one processor, cause the system to at least: extracting, by a processor, one or more attributes related to the product from data indicated in the request for identifying the product, each of the one or more attributes describing the product, wherein the data includes an image of the product, and extracting the one or more attributes includes identifying the one or more attributes from the image based on a visual question answering (VQA) paradigm; adjusting, by the processor, the one or more attributes to a format identical to a format of one or more entries in a database based on historical data related to the product, the database including one or more entries for each product in a plurality of products; identifying a match to the product from the plurality of products, the identifying based on comparing, by the processor, the one or more attributes to each corresponding entry of the plurality of products; selecting one or more products from the plurality of products in the database based on a rule-based comparison of at least one of the one or more attributes with each corresponding entry in the database; identifying a match to the product from the selected one or more products, the identifying based on comparing, by the processor, the one or more attributes to each corresponding entry of the selected one or more products; calculating a score for each of the selected one or more products based on the comparison; When the score of the match is the highest score among the calculated scores and is greater than a threshold, identifying the match with the product; verifying that each of the selected one or more products other than the identified match having a score greater than the threshold also matches the product; and The processor is retrained based on the verification.

6. The system according to claim 5, in, The rule-based comparison to select one or more products from the plurality of products in the database further comprises: identifying one or more key attributes from the one or more attributes; and The one or more products are selected from the plurality of products in the database based on a comparison of the one or more key attributes to each corresponding entry in the database.

7. The system according to claim 5, further configured to: The one or more attributes are adjusted, by the processor, to be at least one of a same unit of measurement, quantity, language, and item name as one or more entries in the database.

8. The system according to claim 5, in, The database further indicates an identifier of each product of the plurality of products, and wherein the system is further configured to map the products with the identified matching identifiers.

Images