Hierarchical data relations for machine learning based query responses

Abstract

The present disclosure is directed to a solution for ML-based generating of hierarchical data relations. A data processing system (DPS) can identify, based on an image of an object of a first entity input into a first ML model trained on a plurality of images of objects, a first subset of a plurality of attributes of the object. DPS can identify, based on a description of the object of the first entity input into a second ML model trained on a plurality of descriptions of the plurality of objects, a second subset of the plurality of attributes of the object. DPS can identify, from second entity, a third subset of the plurality of attributes of the object and determine a weight of a first attribute of the plurality of attributes. DPS can provide an indication of the object responsive to a query determined by a third model to correspond to the object based on the weight of the first attribute.

Description

HIERARCHICAL DATA RELATIONS FOR MACHINE LEARNING BASED QUERY RESPONSESCROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority under 35 U.S. C. § 119 to U.S. Provisional Patent Application No. 63 / 513,939, filed July 17, 2023, which is hereby incorporated by reference herein in its entirety and for all purposes.TECHNICAL FIELD

[0002] The present disclosure relates generally to machine learning (ML) based hierarchical data relations, including, but not limited to, providing outputs based on hierarchical classification of data using ML.BACKGROUND

[0003] A computing device can process data relating to objects using images and textual descriptions. The computing device can utilize machine learning (ML) to derive an output.SUMMARY

[0004] Technical solutions in this disclosure utilize a multidimensional contextual relationship hierarchy of product data using natural language processing (NLP) models to optimize the accuracy and efficiency of search results for client device product queries. Utilizing machine learning (ML) to reliably and accurately respond to queries on specific products that can be described using varying terminology, can be a challenge. Variations in product descriptions of the same products across different websites can lead to performance issues (e.g., hallucinations) of ML models trained on such metadata-tagged product data, increasing computational and system energy inefficiencies. These technical solutions overcome such challenges by enhancing product-specific classification and contextualization derived from diverse data sources within an embedding knowledge space using product classifications and their associated attributes. By matching query embeddings withclassification attributes, these solutions allow for precise product identification, mitigating ML model performance issues and improving computational and system efficiencies.

[0005] An aspect of the technical solutions is directed to a data processing system. The data processing system can include one or more processors coupled with memory. The one or more processors can be configured to identify a first embedding generated based on a first description of a product input into one or more machine learning models. The one or more processors can be configured to identify a second embedding generated based on a second description of the product input into the one or more machine learning models. The one or more processors can be configured to generate, using the one or more machine learning models, a classification of the product and an attribute of the classification based on a relation between the first embedding and the second embedding. The one or more processors can be configured to receive, from a client device, a query comprising a content. The one or more processors can be configured to identify the product based on a match between an embedding of the content and an embedding of the attribute of the classification. The one or more processors can be configured to provide, based on the match, a response to the query comprising information about the product.

[0006] An aspect of the technical solutions is directed to a method. The method can include one or more processors coupled with memory identifying a first embedding generated based on a first description of a product input into one or more machine learning models. The method can include the one or more processors identifying a second embedding generated based on a second description of the product input into the one or more machine learning models. The method can include the one or more processors generating, using the one or more machine learning models, a classification of the product and an attribute of the classification based on a relation between the first embedding and the second embedding. The method can include the one or more processors receiving, from a client device, a query comprising a content. The method can include the one or more processors identifying the product based on a match between an embedding of the content and an embedding of the attribute of the classification. The method can include the one or more processors providing, based on the match, a response to the query comprising information about the product.

[0007] In one aspect, the technical solutions are directed to a non-transitory computer- readable medium comprising instructions. The instructions, when executed by one or more processors, can cause the one or more processors to identify a first embedding generated based on a first description of a product input into one or more machine learning models. The instructions, when executed by one or more processors, can cause the one or more processors to identify a second embedding generated based on a second description of the product input into the one or more machine learning models. The instructions, when executed by one or more processors, can cause the one or more processors to generate, using the one or more machine learning models, a classification of the product and an attribute of the classification based on a relation between the first embedding and the second embedding. The instructions, when executed by one or more processors, can cause the one or more processors to receive, from a client device, a query comprising a content. The instructions, when executed by one or more processors, can cause the one or more processors to identify the product based on a match between an embedding of the content and an embedding of the attribute of the classification. The instructions, when executed by one or more processors, can cause the one or more processors to provide, based on the match, a response to the query comprising information about the product.

[0008] In one aspect, the technical solutions are directed to a system. The system can include a data processing system comprising one or more processors coupled with memory. The data processing system can identify, based on an image of an object of a first entity input into a first machine learning model trained on a plurality of images of a plurality of objects, a first subset of a plurality of attributes of the object. The data processing system can identify, based on a description of the object of the first entity input into a second machine learning model trained on a plurality of descriptions of the plurality of objects, a second subset of the plurality of attributes of the object. The data processing system can identify, from a website of a second entity, a third subset of the plurality of attributes of the object. The data processing system can determine a weight of a first attribute of the plurality of attributes of the object. The data processing system can provide an indication of the object responsive to a query determined by a third model to correspond to the object based on the weight of the firstattribute.

[0009] In one aspect, the technical solutions are directed to a method. The method can include a data processing system comprising one or more processors coupled with memory identifying, based on an image of an object of a first entity input into a first machine learning model trained on a plurality of images of a plurality of objects, a first subset of a plurality of attributes of the object. The method can include the data processing system identifying based on a description of the object of the first entity input into a second machine learning model trained on a plurality of descriptions of the plurality of objects, a second subset of the plurality of attributes of the object. The method can include the data processing system identifying, by the data processing system, from a website of a second entity, a third subset of the plurality of attributes of the object. The method can include the data processing system determining a weight of a first attribute of the plurality of attributes of the object. The method can include the data processing system providing an indication of the object responsive to a query determined by a third model to correspond to the object based on the weight of the first attribute.

[0010] In one aspect, the technical solutions are directed to a non-transitory computer- readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to identify, based on an image of an object of a first entity input into a first machine learning model trained on a plurality of images of a plurality of objects, a first subset of a plurality of attributes of the object. The instructions, when executed can cause the one or more processors to identify, based on a description of the object of the first entity input into a second machine learning model trained on a plurality of descriptions of the plurality of objects, a second subset of the plurality of attributes of the object. The instructions, when executed can cause the one or more processors to identify, from a website of a second entity, a third subset of the plurality of attributes of the object. The instructions, when executed can cause the one or more processors to determine a weight of a first attribute of the plurality of attributes of the object and provide an indication of the object responsive to a query determined by a third model to correspond to the object based on the weight of the first attribute.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Aspects of the present disclosure are described in the detailed description which follows, in reference to the noted plurality of drawings by way of non-limiting examples of exemplary embodiments of the present disclosure.

[0012] FIG. 1 is an example block diagram of a system for providing and using hierarchical data relations for machine learning based query responses.

[0013] FIG. 2 an example architecture of a computing system implemented in embodiments of the present disclosure.

[0014] FIGS. 3 is an example of an image illustrating machine learning in image recognition for feature and attribute extraction, in accordance with embodiments of the present solution.

[0015] FIG. 4 is an example of an illustration of product description comparison, in accordance with embodiments of the present solution.

[0016] FIG. 5 is an example of a knowledge graph including embeddings of classifications and attributes for various products to identify in response to client device queries.

[0017] FIG. 6 is an example a process flow for ML-based generating of hierarchical data relations, in accordance with embodiments of the present solution., in accordance with aspects of the present disclosure.

[0018] FIG. 7 is an example a process flow for providing and using hierarchical data relations for machine learning based query responses.DETAILED DESCRIPTION

[0019] Technical solutions in this disclosure utilize a multidimensional contextual relationship hierarchy of product data to enhance the efficiency and accuracy of search results for user queries using natural language processing (NLP) models. Overcoming the challenge of providing accurate responses to queries about specific products involves navigating complexities of diverse terminology used across various product categories, contexts and sites. Training machine learning (ML) models using metadata tags associated with product descriptions based on such diverse terminology can introduce inaccuracies,leading to ML performance issues, such as hallucinations. These issues can prolong communication exchanges and also decrease search query fulfillment efficiency, increasing the use of computational resources and impacting system energy efficiency.

[0020] The technical solutions of this disclosure can overcome these challenges by providing a product-specific classification and contextualization of data within an embedding knowledge space in order to improve the effectiveness and accuracy of the ML models in identifying results for the product search queries. Using the product-specific multidimensional taxonomy generated from data spanning multiple sources, the technical solutions can improve and supplement the contextual relationships between product entities and their attributes. By integrating image and textual data alongside the product-specific taxonomy, these solutions can improve attribute contexts such as classification specifications, resulting in more precise search outcomes. To form the hierarchical data relations for embedding processes of product-specific data classifications, the solutions can utilize encoder and decoder functionalities along with transformer-based ML models for vectorization and creation of nested categories and attributes specific to each product. In doing so, the technical solutions can provide feature extraction and attribute specification by matching query embeddings with classification attributes using similarity search functions, thereby accurately identifying products for corresponding search queries, while reducing the chance of model hallucinations, saving computational resources and increasing the energy efficiency of the system.

[0021] The solutions can improve accuracy of ML-based search query responses using classifications with weighted relations between various attributes of classes and sub-classes to reduce the reliance on inconsistencies in the product dataset metadata tags. As metadata tagging and product descriptions across online stores is normally independent and manually implemented across different sites, variations in classifications and attribute descriptions are common. This makes it challenging to use such inconsistently tagged data to provide efficient and on-point responses to query searches as results may vary based on variations in descriptions. The systems and methods of the present disclosure can allow for objects (e.g., classifications) for search queries to be contextually improved by supplementing thedescriptions of the attributes (e.g., specifications of the classifications) of the objects using textual or image data of the object from multiple (e.g., third party) sources (e.g., websites).

[0022] The technical solutions can allow entities to leverage data (e.g., texts or images) from various product catalogues and descriptions or documentations of the products from various third-party sources to create a relationship schema combining semantic and syntactic information. This schema can provide a more efficient retrieval of product information using weighted relationships between various attributes of the classification objects allowing imprecise and unsophisticated product descriptions from the user queries to be matched with specific technical attributes of the products matching the queries.

[0023] The technical solutions can overcome the challenges associated with time-consuming and compute and resource intensive storage and retrieval of the data for large, diverse, and poorly structured product datasets. By utilizing product-specific multi-relational weighted classification taxonomy and hierarchical data mapping, the technical solutions can overcome the issues caused by the heterogeneity across product details and syntactic discrepancies in product descriptions. For instance, product data from various online stores can have variations in data formats and syntactic discrepancies leading to similar but differently phrased product attributes. The technical solutions can utilize transformer-based graph neural network (GNN) language models to create knowledge graphs (KGs) in the embedding space of the nested categories and attributes allowing the solutions to pinpoint exact product searched for by the queries based on embedding comparisons of the query and the product classifications and attributes. The knowledge graphs can allow the solutions to provide more compute and energy efficient query responses using pre-processed hierarchical data relations of the product classification taxonomies.

[0024] These solutions can include an ingestor service for analyzing data from online store web pages, product feeds, and linked documents (e.g., support or how to manuals) to combine them into a consistent list of specifications. Techniques for attribute extraction and relationship generation (e.g., computer vision via deep learning neural networks and encoder / decoder techniques) can be used, along with NLPs with LLMs to extract attributes and perform validation using similarity searches against other sources of product data. In thecase of image recognition and feature extraction, neural network algorithms of these solutions can perform pattern and text recognition and form attributes to be extracted, such as: a type of an object (e.g., a product), a style of an object, a category of an object, a color of an object, a type of fit of an object, a brand of the object, a pattern of the object, etc.

[0025] Transformer-based LLMs can be used to vectorize textual product data and compare data in product feeds or catalogues with those of online product detail pages (PDPs) and other data sources. Using these comparisons, relations can be determined to validate and attain an increased level of confidence in the classifications and the attributes. The solutions can utilize transformer-based LLMs optimized for semantic search that can chunk and vectorize the product data so that semantic similarities can be determined (e.g.., using a cosine similarity), thereby determining matching results for those product taxonomies whose classification attributes result in cosine similarity values that are closest to the value of 1.

[0026] FIG. 1 is an example block diagram of a system 100 for providing and using hierarchical data relations for machine learning based query responses, is illustrated.Example system 100 can include a data processing system (DPS) 102 that can be a standalone system (e.g., on an independent server or a cloud), or can be provided on a system of a first entity 104 (e.g., an enterprise serving responses to client device queries). Data processing system 102 can communicate, via a network 108, with a second entity 104 (e.g., server or a website of another enterprise) and a client device 106 that a client (e.g., user) can utilize to generate queries 170 on one or more products to purchase.

[0027] DPS 102 can include one or more of model trainers 110, image models 112, natural language processing (NLP) models 114 and vectorizing models 116. DPS 102 can include one or more ingestor functions 120, repositories 130, data relations generators (DRGs) , response functions 160 and websites 164 that can be provided for client device interactions. Data relations generator (DRG) 150 can include or generate taxonomy 124 for various products. Taxonomies 124 can include classifications 142 for products and attributes 148 of the classifications that can be associated with various weights 152. Repository 132 can include or store one or more of training datasets 130, data structures 158, taxonomies 124 and knowledge graphs 156. Vectorizing models 116 can include or generate embeddings 122(e.g., for atributes 148 or various aspects of products data 140) and utilize one or more similarity functions 118 for comparing different embeddings 122. Ingestor function 120 can include or ingest products data 140, including images 144 or descriptions 146 of various products from the local or remote websites 164 in order to generate the hierarchical data relations for the response function 160 to provide the responses 162, responsive to the queries 170 from the client devices.

[0028] Network 108 can include any type and form of network for communication between different network devices, such as the Internet comprising one or more wide area network (WANs), local area networks (LANs), cellular networks, wired or wireless networks and devices communicating using communication protocols, such as TCP / IP and others. DPS 102 can communicate with a second entity 104 (e.g., or any number of third-party entities) that can include websites 164 providing, representing or offering products data 140 and their corresponding images 144 and descriptions 146.

[0029] System 100 can include any combination of hardware and software, including a data processing system 102, for generating hierarchical data relations (e.g., taxonomy 124) based on varying descriptions 146 or images 144 from the products data 140 gathered from one or more entities 104 and their associated websites 164. System 100 can include a data processing system 102, which can include any computing device providing or executing functionalities described herein. System 100, including a data processing system 102, can include, or be implemented on, or via, a computing system 200 as described in FIG. 2. System 100, and the data processing system 102, can include or have its various operations or functionalities implemented using one or more processors 210 that can be coupled with one or more memories 215. For instance, a data processing system 102 can be implemented using instructions, commands, data and computer code that can be stored in main memory 215 or storage device 225. For instance, one or more processors 210 can access the instructions (e.g., commands, computer code or data) that can cause the one or more processors 210 to implement any functionality or feature (e.g., component or technique) implemented by the data processing system 102.

[0030] Product data 140 can include any type and form of information or data about a product that can be offered for sale via a website 164. Product data 140 can include data on any product, such as an electronics device or a system (e.g., smartphones, laptops, televisions, cameras, toys), clothing products (e.g., shirts, pants, dresses or shoes), home appliances (e.g., kitchen appliances, home entertainment systems and household gadgets), health and beauty products (e.g., skincare products, cosmetics, hair care products and personal care items), furniture and home decor products (e.g., indoor and outdoor furniture, decorative items), books and media (e.g., physical or electronic books, music, movies), sports and fitness products (e.g., sports equipment, fitness gear), automotive products (e.g., vehicles, vehicle parts), or groceries and food (e.g., food items, beverages, or similar). Product data 140 can include information about any aspects or characteristics of a product, such as product specifications, features, performance, design, dimensions, functionalities, color, shape, identifying information (e.g., name, serial number or a stock keeping unit or SKU), authorship data, brand name, make and model, or any other information. Product data 140 can include, or be expressed as, or based on, any one or more media (e.g., images 144) or textual descriptions 146.

[0031] Taxonomy 124 can include any arrangement, structure, or any structured framework, for organizing or categorizing product data 140. Taxonomy 124 can include a structured arrangement to organize and categorize data hierarchically, allowing for efficient data retrieval and analysis. Taxonomy 124 can include classifications, including categories grouping products based on shared characteristics, and attributes, which can include details or features (e.g., attributes 148) for each given classification 142 (e.g., category or subcategory of product features). Taxonomy 124 can include a relational structure personalized or configured for each individual product, such that taxonomy 124 using a product data 140 of a first product can include a set of categories and sub- categories (e.g., hierarchy of data) that is different from the set of categories and sub-categories of product data 140 of another product. Taxonomy 124 can include classifications 142 that are linked to their respective attributes 148, based on common features or characteristics identified by the attributes 148, thereby facilitating precise identification and contextualization of products. Taxonomy 124can be utilized for machine learning applications, such as by providing a framework for organizing embeddings 122 and contextual relationships between the embeddings 122 within a knowledge graph 156 (e.g., based on the hierarchical organization of classifications 142 and attributes 148 of the taxonomy 124).

[0032] Images 144 and descriptions 146 can be used to describe or illustrate products data 140. Product images 144 can include any images (e.g., still images or motion images, such as videos) of the product data 140 on any type and form of product. Images 144 include any media or images of the product or its use that can be provided via websites 164 that can be used to offer products for sale via e-commerce websites 164 of any number of entities 104 (e.g., enterprises, organizations, companies or corporations).

[0033] Descriptions 146 can include any textual descriptions of products or any text-based product data 140. Descriptions 146 of a product can include, for example, summary product descriptions, technical specifications, dimensions, identification information of a product (e.g., product make, model or serial number), description of various product attributes 148 (e.g., performance characteristics, size, shape, color, or material used) or any other textual content that can be used to provide data of a product that can be sold via websites 164.

[0034] Classifications 142 can include to any hierarchical categories used to organize and identify the product based on its attributes and characteristics. Classifications 142 can include broad categories or subcategories of any type and form of products, such as electronics, clothing, or home goods, which can group products based on their general type. Classifications 142 include specifications or sub-categories, such as those distinguishing between men's and women's clothing or categorizing electronics into subgroups television sets, smartphones, laptops or accessories. Classifications 142 can include or be associated with attributes 148 that can indicate grouping within the given classification. For instance, classification 142 can group products into specific categories based on shared characteristics, while an attribute 148 of a classification 142 can provide details, descriptions or specifications for, or unique to, the products within the given category. For instance, for classifications 142 such as a brand of a product, the attribute 148 can identify themanufacturer of the product. For instance, for a classification 142, such as a color, the attribute 148 can identify the color (e.g., green, yellow or blue).

[0035] Classifications 142 can include any representation or classification of a product (e.g., an image 144 or product description 146 helping categorize or group the product with other products based on their characteristics, attributes, or properties. Classification 142 can include a distinct class or category of a product. Classification 142 can be determined using classification systems that can be hierarchical, with broader categories subdivided into more specific subcategories. Such classifications can be based on factors such as product type, industry, function, material composition, or intended use. By assigning products to specific classes, DPS 102 can streamline operations and improve search and retrieval processes.

[0036] Attributes 148 (e.g., specifications) of the classification 142 can include any information or details about classification 142, including any features, attributes or properties of a particular category or sub-category in the hierarchical taxonomy 124. Attributes 148 can be organized in a structured format. For instance, attributes 148 can include quantitative measurements, qualitative descriptions, or other relevant details that define the object's features, functions, or requirements. Attributes 148 can include dimensions, weights, materials, performance metrics, feature descriptions, operating conditions, safety considerations, and more. Attributes 148 can allow for effective specification of a classification 142 to provide a more accurate communication, comparison, and evaluation of classifications 142.

[0037] Image models 112 can include any ML models for processing images 144. Image models 112 can include any ML models or a computational framework, such as a model or a framework based on deep learning neural networks (e.g., convolutional neural networks (CNNs)). Image models 112 can include the functionality to analyze and interpret visual data by automatically learning hierarchical representations and patterns from images. Image model 112 can include the functionality to provide, identify or generate image-based classifications 142, object (e.g., product) detection, attributes 148 or their corresponding embeddings 122.

[0038] An image model 112 can be trained using a large dataset 130 of images 144 to generate taxonomy 124, including one or more classifications 142 and the associated attributes 148, based on one or more images 144 input into the image model 112. For example, an image model 112 can identify one or more attributes 148 (e.g., specifications) for classifications 142 of a product, based on the product data 140. Image model 112 can utilize images 144 of a product based on product data 140 ingested from a website 164 of an entity (e.g., a first entity 104) to infer classifications 142 (e.g., a vehicle type, a clothing type or an electronics product) as well as any associated attributes 148 (e.g., brand or make and model of the product).

[0039] For instance, the image model 112 can act as a model that identifies a first subset of a plurality of attributes 148 based on a particular image 144 of an object (e.g., a classification 142). For instance, an image 144 can be input into the image model 112 that can be trained on any number of images 144 of any number of objects (e.g., classifications 142), which can correspond to any number of products data 140 for any number of products. The image 144 can be used to identify a first subset of a plurality of attributes 148 of the product.

[0040] NLP models 114 can include any ML models for processing descriptions 146. NLP models 114 can include any ML model (e.g., a computational framework), that can be based deep learning techniques. NLP models 114 can include the functionality to process and understand human language, enabling or performing tasks such as text classification, sentiment analysis, language translation, and question answering, by leveraging patterns, semantics, and context within textual data.

[0041] An NLP models 114 can be trained based on a large dataset 130 of descriptions 146 to identify any number of attributes 148 of a classification 142 when on one or more descriptions 146 (e.g., string of text data or characters) input into the NLP model 114. NLP model 114 can include any NLP model, such as an LLM trained on textual data corresponding to a product, such as data specifications, product description summaries, transcribed service calls for the products, product catalogues or any other data. For example, an NLP model 114 can identify a second subset of the plurality of attributes 148 of the classification 142 based on a description 146 of the product on a website 164 of a first entity104 or a second entity 104. The product descriptions can be input into the NLP model 114 (e.g., a second ML model trained on a plurality of descriptions 146 of the plurality of objects).

[0042] DPS 102 can utilize an ingestor function 120 to access and capture data (e.g., images 144 or descriptions 146 of products data 140) from one or more websites 164 from various entities 104. For instance, data processing system 102 implemented on a server device of a first entity 104 serving a website 164 of the first entity 104 can utilize an ingestor function 120 to ingest or capture products data 140 both from the local website 164 of the first entity and the remote website 164 of the second entity 104. Entities 104 can include any websites (e.g., server or cloud based) online services providing product data 140 for sale of products. Ingestor function 120 can capture and store products data 140 (e.g., images 144 or descriptions 146) from the website of the first entity 104 or the second entity 104 into the repository 132. For example, an ingestor function 120 can identify, from a website 164 of a second entity 104, one or more subsets of attributes 148 of a classification of a product.

[0043] Data Relations Generator (DRG) 150 can include any combination of hardware and software for generating relations between classifications 142 and attributes 148 in a taxonomy 124. DRG 150 can include the functionality for determining relations or establishing connections between data attributes in a taxonomy 124, such as by assigning weights 152 to attributes 148 based on syntactic and / or semantic similarity. By identifying attributes 148 from images 144 (e.g., via image models 112) and descriptions 146 (e.g., via NLP models 114) from both the first entity 104 (e.g., client entity) and a second entity 104 (e.g., third party), DRG 150 can utilize ML models (e.g., 112, 114 or 116) to determine relations between classifications 142 and attributes 148. For instance, DRG 150 can utilize ML models (e.g., 112, 114 or 116) to identify instances of attributes 148 or related descriptions of attributes 148 that may be repeated in more than one instance in one or more websites 164. By identifying the number of instances in which attributes 148 for a given classification 142 or a given product appear, DRG 150 can assign weights 152 to each such attribute 148. As a result, some attributes 148 can have greater weight 152 than others and therefore may be relied on more than other attributes (e.g., that are less weighted).

[0044] DRG 150 can work together with a response function 160 analyze the NLP tokens from the original query 170 received from a client device 106. DRG 150 and / or response function 160 can identify attributes 148 from the query 170 that are most relevant to the search. These attributes 148 can be given weights 152 based on their similarity to the query 170. For instance, DRG 150 can consider both syntactic and semantic aspects to determine the weights 152 to the attributes 148.

[0045] By leveraging a non-recursive normalized structure, the DRG 150 can organize the data into a hierarchical taxonomy corresponding to different attributes. The DRG 150 can enable data access through query JOINs, which can be less exhaustive compared to recursive joins (self-joins). This can improve query performance and allow for easier schema changes when adding new hierarchies to the taxonomy. Instead of extensive modifications, introducing a new hierarchy can often be achieved by adding a new value to the column that defines the hierarchy type.

[0046] DPS 102 can utilize a DRG 150 to determine a weight of a first attribute 148 of the plurality of attributes 148 of the classification 142. For example, DRG 150 can assign a weight to the first attribute 148 based on a determination that the first attribute was identified based on both images 144 data and descriptions 146 data. The first attribute 148 can be assigned a weight, such as for example a value of 0.9, for a range of weights between 0.0 and 1.0. The first attribute 148 can be assigned a weight according to similarity search (e.g., Euclidean distance or cosine similarity) performed by a similarity function 118.

[0047] DRG 150 can utilize ML models (e.g., any combination of 112, 114 or 116) to generate one or more classifications 142 of a product (e.g., based on product data 140), as well as any one or more attributes 148 of such one or more classifications 142. DRG 150 can generate classifications 142 or attributes 148, based on a relation between one or more embeddings 122, such as a first embedding 122 of a product gathered from a first products data 140 form a first website 164 and a second embedding 122 of the same product gathered from a second products data 140 from a second website 164. For instance, the two products data 140 can utilize different verbiage, phrases or terminology to describe the same product in different ways. The DRG 150 can generate the classifications 142 and the attributes 148based on vector representation (e.g., embedding 122) of a context generated from a plurality of embeddings 122 from the tokens or words of the different products data 140 from the different websites 164, in order to include the attributes 148 and classifications 142 from any combination of different descriptions of the product.

[0048] The DRG 150 can identify a product described by a query 170 based on a match between an embedding 122 of the content of the query 170 (e.g., embedding 122 determined by a vectorizing model 116 on a portion of the text of the query 170) and an embedding of the attribute 148 of the classification 142 of the product being identified. For instance, by comparing vector representations of the portion of the query 170 with the vector representation (e.g., embedding 122) of the attribute 148, the DRG 150 can identify (e.g., using a similarity function 118) that the query 170 is discussing this particular product.

[0049] The DRG 150 can determine, based on a match between an embedding of the content of the incoming query 170 and an embedding of the attribute 148 of the classification 142 of a product, that the content of the query 170 corresponds to the classification 142 of the product. The DRG 150 can identify the product, based on such a determination or based on such a match.

[0050] The DRG 150 can generate the response 162 for the query 170 comprising the information (e.g., products data 140) that can correspond to the classification 142 and the attribute 148. The information can include, for example, products data 140 (e.g., any combination of a description of a product, a link to a product, a specification of a product or an image 144 of the product), which can be provided as a part of the response 162 (e.g., web page or a content page) responsive to the query 170.

[0051] The DRG 150 can use the one or more ML models (e.g., 112, 114 or 116) to generate, a data structure 158. The data structure 158 can include a taxonomy 124 of the product comprising any number of classifications 142 and attributes 148. For instance, the taxonomy 124 can include the classification 142 associated with the attribute 148 and a second classification 142 associated with a second attribute 148. The DRG 150 can map the query 170 (e.g., by comparing one or more embeddings 122 of the query 170 with one or more embeddings 122 of one or more attributes 148 of the taxonomy 124 of the product) to thetaxonomy 124 of the product based on the match (e.g., between the embedding 122 of the attribute 148 and the embedding of the query 170) and a also based on a second match between a second embedding 122 of the content of the query with an embedding 122 of the second attribute 148 of the same taxonomy 124 of the product.

[0052] The DRG 150 can generate a knowledge graph 156 that can include, or be composed of, any number of embeddings 122 of a plurality of attributes 148 associated with a plurality of classifications 142 of a plurality of products. The DRG 150 can identify a data structure 158 of the product based on a distance (e.g., a match in similarity) in the knowledge graph 156 between the embedding 122 of the content of the query 170 and the embedding 122 of the attribute 148 satisfying a threshold for similarity. The threshold for similarity can include, for example, a value of 0.9 or 0.95 for cosine similarity between two embeddings 122 (e.g., indicating that the two vector representations or embeddings are similar to each other by at least or more than a threshold value). The data structure 158 can include the information about the product.

[0053] The DRG 150 can utilize the response function 160, or operate together with the response function 160, to generate the response 162. The response function 160 or the DRG 150 can generate the response 162 based on the identified data structure 158. The generated response 162 can include the information about the product, such as any product data 140 (e.g., product description, product specifications, product links, product sales materials, product marketing materials, product price, product link to page to complete a purchase of the product, or any other product data 140).

[0054] The DRG 150 can store the embedding 122 of the attribute 148 into knowledge graph 156 of embeddings 122. The knowledge graph 156 can identify a plurality of embeddings 122 of a plurality attributes 148 corresponding to a plurality of classifications 142 of a plurality of products (e.g., objects queried by the query 170). The plurality of products can include the product queried by the query 170 and can include the embedding 122 of the attribute 148 of the plurality of embeddings 122 of the plurality of attributes 148. The DRG 150 can identify, from the plurality of products, the product based on the match (e.g., between the embedding 122 of the attribute 148 of the product and an embedding 122 of acontent of the query 170). The DRG 150 can identify the product, from the plurality of product, based on the attribute 148 associated with the classification 142 of the product in the knowledge graph 156.

[0055] The DRG 150 can generate the embedding 122 of the attribute 148 based on the first embedding 122 and the second embedding 122. The DRG 150 can associate the embedding 122 of the attribute 148 with the classification 142 of a plurality of classifications 142 of a taxonomy 124 of the product. The DRG 150 can generate, using the one or more ML models 112, 114 or 116, a data structure 158 of the product. The data structure 158 can include the classification 142 and the information (e.g., product data 140) about the product. The product data 140 can include a link to a web page about the product, such as a link to a page to purchase the product via a website 164. The DRG 150 utilize or work with a response function 160 to generate, based on the match (e.g., between the embedding of the attribute 148 and the embedding of the query), the response 162 comprising the link to the page about the product.

[0056] The DRG 150 can utilize one or more ML models (e.g., 112, 114 or 116) to generate a second attribute 148 for the same classification 142 of the product (e.g., as the first attribute 148), based on the third embedding 122. The third embedding 122 can correspond to a third portion of the product data 140. The DRG 150 can identify the product based on a second match between a second embedding 122 of the content of the query 170 and an embedding of the second attribute 148 (e.g., generated based on the third embedding 122). The DRG 150 can utilize the response function 160 to provide the response 162 to the query 170, based on this second match.

[0057] Vectorizing model 116 can include any ML model for generating or determining vector representations or embeddings 122 of any portion of products data 140 (e.g., images 144 or descriptions 146). Vectorizing model 116 can utilize one or more similarity function 118 (e.g., Euclidean distance or cosine similarity function applied to embeddings 122 of attributes 148) to determine relations between the attributes 148. Vectorizing model 116 can utilize embeddings 122 and similarity functions 118 to determine, apply or utilize weights152 and the attributes 148 to provide responses 162 to queries 170 (e.g., on behalf of, or responsive to instruction or call from, response function 160).

[0058] Vectorizing model 116 can utilize embeddings 122 of attributes 148 or classifications 142 of any products data 140 of any products to generate a knowledge graph 156. Knowledge graph 156 can include any collection of embeddings 122 (e.g., corresponding to any attributes 148 or classifications 142) that can be used as a vector space or an embedding space to make determinations in terms of relations or contextual or semantic similarity between content of the queries 170 and the embeddings 122.

[0059] Vectorizing model 116 can include a computational framework generating embeddings 122 or transforming weighted attributes 148 derived from product images 144 and descriptions 146 into numerical representations (e.g., vectors). Vectorizing model 116 can work together with DRG 150 to assign weights 152 to attributes 148 based on their relevance (e.g., instances of their appearance in the image or description data), and then combine them into a high-dimensional vector representation. Embeddings can be used to encode the semantic relationships and similarities between the attributes 148 within a multidimensional vector space. By combining weighted attributes (e.g., weights 152 and their attributes 148) with embeddings, the vectorizing model 116 can create a dense and continuous representation that covers both the importance of each attribute 148 (e.g., using weights 152) and their underlying semantic meaning. This can allow for efficient computation, comparison, and retrieval of products based on their attributes 148 from images 144 and descriptions 146 of the product data 140.

[0060] ML models 112, 114 or 116 can include any machine learning architecture of functionality including any attention-mechanism or transformer-based mechanisms. ML models 112, 114 or 116 can include, for example, graph neural network (GNN) functionality, including transformer-based GNN models trained on a dataset of information on a plurality of products (e.g., products data 140) from the training datasets 130 stored in the repository 130.

[0061] Vectorization model 116 can include the functionality to generate or identify embeddings 122. Vectorization model 116 can be utilized by DRG 150 to identify a firstembedding 122 generated based on a first description of a product (e.g., product data 140 from a first website 164 of a first entity 104) input into one or more machine learning models. The vectorization model 116 can identify a second embedding 122 generated based on a second description of the product (e.g., product data 140 from a second website 164 of a second entity 104) input into the one or more machine learning models. The two websites 164 can include different descriptions of the same product, including different phrases or different formatting of the product data 140.

[0062] The vectorization model 116 can be used by the DRG 150 to determine a weight 152 for an attribute 148 based on a number of instances of occurrence of a first word associated with the first embedding 122 and a number of instances of occurrence of a second word associated with the second embedding 122. The first and the second words can be similar in meaning or context (e.g., high value output from a cosine similarity function 118). The vectorizing model 116 can identify a third embedding 122 of the product based on an image 144 of the product input into a ML model (e.g., 112, 114 or 116) of the one or more ML models that can be trained (e.g., by model trainer 110) on a plurality of images 144 of a plurality of products (e.g., based on the products data 140 for various products).

[0063] Response function 160 of the DPS 102 can including any combination of hardware and software for providing a response 162 to a query 170 from one or more client devices 106. Client device 106 can include any device that can be utilized by a client (e.g., a user), such as a computer, a tablet, or a smartphone and that can be used to generate queries 170 via a network 108.

[0064] Query 170 can be any query (e.g., a search query), including a string of text or an image corresponding to, or describing, a product. The query 170 may include language that is similar to, or relates to, any classifications 142 or attributes 148. The queries 170 can include textual descriptions of features or characteristics of a product that the data processing system 102 can utilize identify the product. The queries 170 can include images 144 or descriptions 146 that can relate to the products or categories of products the user is seeking. Queries 170 can be determined to correspond to the attributes 148 or classification 142 by GRD 150 using, for example, an image model 112, an NLP model 114 or a vectorizing model 116 (e.g.,a third model) to generate or determine a relation between a first attribute 148 and the query 170, based on, for example the weight 152 of the first attribute 148. Vectorizing model 116 can be used to determine that an incoming query 170 from a client device 106 corresponds to a particular product, using a plurality of weights 152 corresponding to a plurality of attributes 148 of one or more classifications 142 of the object or product.

[0065] Response function 160 can utilize any ML models 112, 114 or 116 to generate responses 162 (e.g., a page, a link, a product description, a message, a webpage or a file) responsive to the determination that the query 170 corresponds to the object. Response function 160 can organize or store taxonomies 124, including any classifications 142 and their corresponding attributes 148 or weights 152 using data structures 158. Data structures 158 can include content corresponding to the product, such as, any product data 140 that can be provided via a response 162, responsive to the query 170.

[0066] Response function 160 can be configured (e.g., implemented via instructions, computer code and data stored in a memory 215 that is accessed and executed by a processor 210) to receive from a client device 106, a query 170 comprising a content. The content can include any description, summary or characterization of a product, or include any description or mention of product data 140 (e.g., characteristic of a product, performance level, color, size, shape or any other attribute 148 or classification 142 corresponding to the product). Response function 160 can be configured to generate and provide (e.g., prepare for transmission or send for transmission via a transceiver of a computing system 200) a response 162. The response 162 can be a response to the query 170 and can include information about the product, such as any product data 140 (e.g., description or specification of a product, images of the product or a link to a web page of a website 164 to purchase). The response 162 can be provided or generated based on, or responsive to, a match between one or more embeddings 122 of one or more portions (e.g., words or phrases) of the content of the query 170 and one or more embeddings 122 of one or more attributes 148 of one or more classifications 142 corresponding to the product.

[0067] DPS 102 can be configures to receive the query 170 from user of a website 164 of the first entity 104. The user can send the query 170 from a client device 106. DPS 102 canutilize the DRG 150 to determine the weight 152 of the first attribute 148 based on the third subset of the attributes 148 gathered or generated from a search of a third-party website 164 at a remote second entity 104. Response function 160 can provide the response 162 within a webpage corresponding to the product data 140 on the website 164 of the first entity 104. The first entity can include an entity comprising the DPS 102.

[0068] DPS 102 can use, for example DRG 150 and / or ingestor function 120, to generate the plurality of attributes 148 of the classification 142 using the first subset (e.g., corresponding to the image 144), the second subset (e.g., corresponding to descriptions 146 from the local first entity 104 descriptions 146, such as catalogues or web page on the product data 140) and the third subset (e.g., corresponding to images 144 or descriptions 146 of the same product from a third party website 164 on a second entity 104). DRG 150 can determine the weight 152 of the first attribute 148 according to a number of instances of the first attribute 148 identified in the first subset, the second subset and the third subset of the plurality of attributes 148.

[0069] DRG 150 can determine, for each respective attribute 148 of the plurality of attributes 148 of the classification 142, a respective weight 152. Each respective weight 152 for each respective attribute 148 can be determined according to a number of instances of occurrence of the each respective attribute 148 in the first subset of the plurality of attributes 148 (e.g., corresponding to image 144 of the first entity 104), the second subset of the plurality of attributes 148 (e.g., corresponding to descriptions 146 of the first entity) and the third subset of the plurality of attributes 148 (e.g., corresponding to images 144 or descriptions 146 from a remote second entity 104). DPS 102 can utilize a model trainer 110 to train the vectorizing model 116 to generate, for each such respective attribute 148 of the plurality of attributes, a respective embedding (e.g., a vector) according to the respective weight 152 of each such respective attribute 148. The model trainer can include any functionality for training ML models (e.g., 112, 114 or 116) using training datasets 130 stored in the repository 132. Training datasets 130 can include any collection of products data 140 or any other information or data for training the functionalities or operational characteristics of any ML models 112, 114 or 116.

[0070] DPS 102 can utilize an ingestor function 120 and / or DRG 150 to identify the first attribute within the third subset of the plurality of attributes. The first attribute 148 may be not identified in the first subset of attributes 148 (e.g., from the image 144 data) or the second subset of attributes (e.g., from descriptions 146 of the website on the first entity 104). DPS 102 can determine, based on the first attribute, that the query corresponds to the object.

[0071] Ingestor function 120 of the DPS 102 can identify the classification 142 (e.g., on a websites 164 of a remote second entity 104) based on a unique identifier. The unique identifier can be a unique identifier of the product data 140 or the classification 142 on the website 164 of the second entity 104. The second entity 104 can be a third-party e-commerce website offering the same product data 140 to which the classification 142 and the attributes 148 correspond.

[0072] Ingestor function 120 of the DPS 102 can identify the second subset of the plurality of attributes 148 based on a search of a website 164 of the first entity 104. The first entity can include an e-commerce website of a client including, accessing or using the DPS 102. Ingestor function 120 of the DPS 102 can identify the third subset of the plurality of attributes 148 based on a search of the website of the second entity.

[0073] Ingestor function 120 of the DPS 102 can identify the third subset from the website 164 of the second entity using any type of an identifier. Identifier can be an identifier of the product data 140, uniquely identifying the product data 140 out of a plurality of product data 140. Identifier can include, for example, a serial identifier of the product data 140, a model identifier of product data 140, a stock keeping unit (SKU) of product data 140, a universal product code (UPC) of product data 140, a European Article Number (EAN) of product data 140, an international standard book number (ISBN) of product data 140, an ASIN (amazon standard identification number) of product data 140, a manufacturer part number (MPN) of product data 140, and a uniform resource locator (URL) of product data 140.

[0074] Ingestor function 120 can gather, collect, record or otherwise ingest any information or data (e.g., product data 140) from any sources, such as websites 164 from a plurality of different entities 104 on the network 108. For example, ingestor function 120 can identify a first description 146 of a particular product via (e.g., using the content of) a first webpage ofa first website 164 of a first entity 104. The ingestor function 120 can identify a second description 146 of the same product using a second web page of the product from a second website 164 of a second entity 104. The two descriptions can include two different sets of products data 140 on the same product, including different phrasing, format, language or characterizations of the product data 140 for the same product. Ingestor function 120 can compile or combine such diverse products data 140 (e.g., different descriptions 146 or images 144) for the same product into a data structure 158. The DRG 150 can utilize this compiled product data 140 to identify and generate classifications 142 and their associated attributes 148, based on the diverse product data 140 compiled from such multiple sources.

[0075] Ingestor function 120 can identify classifications 142 (e.g., hierarchical classes and subclasses descriptive of the products) and their corresponding attributes 148. Attributes 148 of the plurality of attributes 148 can be comprised or compiled from the first subset (e.g., attributes 148 of image data of the local, first entity 104), the second subset (e.g., attributes 148 of textual data of the local, first entity 104) and / or the third subset of attributes 148 (e.g., attributes 148 of image or textual data of remote, third-party entities 104). Attributes 148 can include any one or more of: a type of a silhouette of a classification 142 or product, a color of the classification 142 or product, an occasion associated with the classification 142 or product (e.g., wedding dress), a style of the classification 142 or product, a material of classification 142 or product (e.g., silk shirt), a category of the classification 142 or product, an embellishment of the classification 142 or product, a length of the classification 142 or product, a width of the classification 142 or product, a type of a fit of the classification 142 or product, a pattern of the classification 142 or product, a length or a width of a portion of the classification 142 or product, a type of a feature or a part of the classification or product, a type or generation of technology of the classification 142 or product (e.g., Bluetooth or Wi-Fi compatible), a brand of the product, a manufacturer of the product, a model name of the product, a shape of the classification 142 or product, a weight of the classification or product, a name of a product line of the classification 142 or product, an output of the classification or product, a memory specification of the product (e.g., memory size of a communication device), a performance parameter of the classification 142 or product, a processing capabilityof the classification 142 or product, a power specification of the classification 142 or product or an energy consumption specification of the classification 142 or product. Attribute 148 can include any feature, characteristic or description of the classification 142 or of the product.

[0076] FIG. 2 illustrates a block diagram of an example computing system 200, also referred to as the computer system 200, which can be used to implement elements of the systems and methods described and illustrated herein. Computing system 200 can include or be used to implement any computation or functionality described herein, using for example, commands, instructions or data described herein. Computing system 200 can be included in and run any device (e.g., a server of a first or second entities 104, a client device 106 or any other devices communicating over a network 108). Computing system 200 can be used for operating, executing or providing a model trainer 110, image model 112, NLP model 114, vectorizing model 116, ingestor function 120, DRG 150, repository 132, response function 160 or website 164 and implement any functionality of these components described herein. For instance, any aspects or components of the data processing system 102 can be implemented using instructions, computer code or data stored in memories 215 or storage devices 225 and accessed and processed by one or more processors 210, such that processed instructions cause the processors 210 to implement various operations and actions of any components or features of the data processing system 102.

[0077] Computing system 200 can include at least one bus data bus 205 or other communication component for communicating information. Computing system 200 can include at least one processor 210 or processing circuit coupled to the data bus 205 for executing instructions or processing data or information. Computing system 200 can include one or more processors 210 or processing circuits coupled to the data bus 205 for exchanging or processing data or information along with other computing systems 200. Computing system 200 can include one or more main memories 215, such as a random-access memory (RAM), dynamic RAM (DRAM) or other dynamic storage device, which can be coupled to the data bus 205 for storing information and instructions to be executed by the processor(s) 210. Main memory 215 can be used for storing information (e.g., data, computer code, commands or instructions) during execution of instructions by the processor(s) 210.

[0078] Computing system 200 can include one or more read only memories (ROMs) 220 or other static storage device 225 coupled to the data bus 205 for storing static information and instructions for the processor(s) 210. Storage devices 225 can include any storage device, such as a solid-state device, magnetic disk or optical disk, which can be coupled to the data bus 205 to persistently store information and instructions.

[0079] Computing system 200 may be coupled via the data bus 205 to one or more output devices 235, such as speakers or displays (e.g., liquid crystal display or active-matrix display) for displaying or providing information to a user. Input devices 230, such as keyboards, touch screens or voice interfaces, can be coupled to the data bus 205 for communicating information and commands to the processor(s) 210. Input device 230 can include, for example, a touch screen display (e.g., output device 235). Input device 230 can include a cursor control, such as a mouse, a trackball, or cursor direction keys, for communicating direction information and command selections to the processor(s) 210 for controlling cursor movement on a display.

[0080] Computer system 200 can also include one or more interfaces 240 coupled via data buses 205. Interfaces 240 can include any physical or virtual components enabling communication between the computer system 200 and any external networks (e.g., the Internet of the network 108). Interface 245 can include a network interface providing transfer of data between the processor(s) 210, memories 215 and any external networks (e.g., network 108) to communicate with other devices on the network 108 (e.g., client devices 106, third party entities 104 or DPS 102 provided on a remote server or a cloud).

[0081] The processes, systems and methods described herein can be implemented by the computing system 200 in response to the processor 210 executing an arrangement of instructions contained in main memory 215. Such instructions can be read into main memory 215 from another computer-readable medium, such as the storage device 225. Execution of the arrangement of instructions contained in main memory 215 causes the computing system 200 to perform the illustrative processes described herein. One or more processors 210 in a multi-processing arrangement may also be employed to execute the instructions contained in main memory 215. Hard-wired circuitry can be used in place of orin combination with software instructions together with the systems and methods described herein. Systems and methods described herein are not limited to any specific combination of hardware circuitry and software.

[0082] Although an example computing system has been described in FIG. 2, the subject matter including the operations described in this specification can be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.

[0083] FIGS. 3 is an example 300 of product data 140 extracted (e.g., via ML-based image recognition) to identify classifications 142 (e.g., classes 502 and any number of levels of subclasses 504 of various product features) and their corresponding attributes 148. Example 300 can include an image 144 of a product data 140, such as an image of a woman’s dress that can be offered for sale by a first entity 104, via a website 164. The product (e.g., the dress) can be one of numerous products made available to users operating client devices 106.

[0084] In example 300, the image 144 can be used by ingestor function 120 and DRG 150 (e.g., using an image model 112) to identify features or classifications 142, such as a silhouette, color, material pattern or type of a cut of the product (e.g., the dress). Each of the classifications 142 (e.g., classes 502 or sub-classes 504) can have one or more attributes 148 (e.g., specifications of the class 502 or sub-class 504), describing, for example, a classification 142 of “silhouette” with an attribute 148 of “wrap dress”, a classification 142 of “occasion” with an attribute 148 of a “casual party”, or a classification 142 of “style” with an attribute 148 of “feminine / romantic.” The product data 140 can include a classification 142 of a “sleeve length” specified by an attribute 148 of “short sleeve”, a classification 142 “material” with an attribute 148 of “polyester”, a classification “category” with an attribute 148 or sub-class of “dress.” Product data 140 can include classifications 142 “embellishments” specified by “lace trim” attribute 148. Classifications 142 identified in the image 144 can include a “neck” which can be specified by a “V neck” attribute 148, a “dress length” specified by “mini” attribute 148, a “fit” specified by “loose” attribute 148 and a “pattern” specified by “floral” attribute 148. An image model 112 can use an image 144,such as the one in example 300, as an input to generate or provide classifications 142 and for each one of them any number of their corresponding attributes 148, such as those illustrated in FIG. 3. Classifications 142 and attributes 148 can vary based on the type of the product or product data 140. The DRG 150 and the ingestor function 120 can utilize the classifications 142 (e.g., classes or sub-classes) along with their corresponding attributes 148 to populate the data structure 158 and generate any associated embeddings 122 (e.g., for the attributes 148 or classifications 142) associated with the product.

[0085] FIG. 4 is an example 400 presents a comparison of product descriptions 146 from different entities 104 that the data processing system 102 can use to create a combined hierarchical product-specific classifications and attributes. Example 400 can include a first description 146 of a first product from a first product data 140 gathered from a first website 164 of a first entity 104 and a second description 146 of the same product from a second product data 140 gathered from a second website 164 of a second entity 104. Example 400 can include usage of one or more NLP models 114 to process textual data corresponding to the product data 140 (e.g., text or descriptions of products data 140 from catalogues, webpage descriptions of the product, to gather any metadata corresponding to the product and any other text-based information about the product).

[0086] Using the textual data of the descriptions 146, transformer based LLMs (e.g., NLP models 114) can be used to vectorize data (e.g., create vectors or embeddings 122 corresponding to the chunks of content of the descriptions 146). The DRG 150 can then utilize the vectorizing model 116 or NLP model 114 to compare the embeddings 122 of the vectorized data in product feed or catalogue against online PDP and other product data sources to validate and ascertain confidence on the specification and classification of attributes beyond a particular threshold level.

[0087] The DRG 150 can utilize threshold levels to test or ascertain the levels of similarity between the embeddings 122. The threshold level can be a level of 0.9, 0.95 or 0.99 in the determination of similarity using, for example, a cosine similarity algorithm (e.g., similarity function 118). The data can get chunked and vectorized so that semantic similarity can be calculated by the similarity function 118 (e.g., using a cosine similarity or Euclidean distanceapproach). In a cosine similarity approach, a value close to 1 (e.g., 0.99, 0.95 or 0.90) can indicate a high semantic similarity between two compared embeddings 122, while a value close to 0 (e.g., 0.01, 0.05, 0.1) can indicate a low semantic similarity. In a Euclidean distance approach, a value close to 1 (e.g., 0.99, 0.95 or 0.90) can indicate a low semantic similarity while a value close to 0 (e.g., 0.01, 0.05, 0.1) can indicate a high semantic similarity. Relations between different chunks of descriptions 146 (e.g., instances of attributes 148 from products data 140) pertaining to the same classification 142 (e.g., same class 502 or sub-class 504) can be used to determine or generate an embedding 122 for the given attribute.

[0088] Ingestor function 120 can include capture or grab the data (e.g., images or texts) from store pages, product feeds, or linked documents, such as support or how to manuals. Ingestor function 120 can capture or grab the data from the local entity (e.g., DPS 102 on a first entity 104) or a remote entity (e.g., a computing device on a second entity 104). Upon retrieving the product information or data, ingestor function 120 and / or DRG 150 can combine the information or data into a consistent list of specifications that can be used for processing.DPS 102 can include algorithms that remove duplicate information as several specifications can be differently described despite describing the same product.

[0089] Example 400 of FIG. 4 relates to different descriptions 146 from different entities 104 regarding the same product data 140, such as a specific printer that can be provided for sale on an e-commerce website. The first and second entities 104 can each describe the product using slightly different descriptions 146 (e.g., different phrases, focusing on different feature descriptions, different nomenclature and so on). For each specification of each of the entities 104, DPS 102 can identify the descriptions 146 from the other source (e.g., other entity 104) that represents the same or a similar meaning, to the extent such a description or feature is present.

[0090] Matching product specifications (e.g., matching of classifications 142 and attributes 148) can be implemented using a sentence similarity approach or a semantic search task. DPS 102 can perform product specification matching operations similarly to the way a detection task can be performed (e.g., comparing the vectors of features in a similarity searchand taking actions based on the similarity threshold being satisfied or met). For example, DPS 102 can convert each textual data of the product data 140 (e.g., specification) into an attribute 148 based on a similarity function determination value (e.g., 0.95 similarity threshold being satisfied) using a plurality of distinct words, phrases or sentences that can be compared. The attribute 148 can include, for example, multi-dimensional vector representations. The attribute 148 can include multiple vector representations for multiple varying statements, nomenclature or terms used for description of the same classification 142 (e.g., class or sub-class).

[0091] In example 400, a description 146 of a classification of “Printing Technology Inkjet” that originates from a first entity 104 can be compared or combined (e.g., in a single embedding 122 for the classification 142) with a description “Brand HP”. For example, a description 146 of a classification 142 of “technology” can be compared or combined with “printing technology”. As such, each of the classifications 142 and embeddings 122 can have multi-dimensional or multiple vector representations of a plurality of statements or variations of descriptions of the aspect, characteristic or feature of the product.

[0092] For instance, DPS 102 can utilize an NLP model 114 that includes a sentencetransformers model (e.g., multi-qa-MiniLM-L6-cos-vl model) to be used along with a similarity function 118 utilizing the cosine similarity to derive the semantic similarity of the specifications. In FIG 4, specifications that should be matched are indicated and their similarity score can be computed using the NLP model 114 (e.g., a transformer-based language model). As a result, the same or similar specifications can attain a high similarity score (e.g., about 0.8 or above), while dissimilar product specifications can have a lower similarity score (e.g., below 0.5). For instance, a phrase “Printing Technology Inkjet” and “Brand HP”, can have a similarity score of 0.32, while “Printing Technology Inkjet” and “Technology Inkjet” can have a similarity score of 0.9. As such, NLP model 114 can be used to match product specifications, validating the product attributes and defining a clear specification that can be used for hierarchical data mapping.

[0093] DPS 102 can then utilize attributes 148 and classifications 142 (e.g., categories) derived from the extraction process to feed them into a are fed into a DRG 150 which canform semantic connections in the taxonomy. DRG 150 can give the attributes 148 their corresponding weights 152 based on syntactic and semantic similarity and can link them. This can allow traversal of lookup surfaces of the highest density of attributes 148 resulting in the improved (e.g., more granular) results based on NLP tokens from the original query 170 received from a client device 106, providing improved performance and accuracy of the results to the queries 170 for product discovery.

[0094] DRG 150 can use non-recursive normalized (e.g., nested) structure. Each hierarchical entity can have different attributes 148. This can provide flexibility to the model. As far as data access is concerned, query JOINs can be less exhaustive than recursive joins (self joins) because records can be split into smaller entities. Adding a new hierarchy can include adding structural changes (schema changes). In recursive models adding a new hierarchy can be achieved through introduction of a new value to the column that defines hierarchy type. Granularity of the attribute 148 coverage can depend on the features identified based on the data from the local (e.g., first) entity 104, second entity 104 and any other entities 104 on the network 108.

[0095] FIG. 5 illustrates an example 500 of a knowledge graph 156 including embeddings 122 of classifications 142 (e.g., classes 502 and sub-classes 504) and any attributes 148 associated with the classifications 142 for any number of products. Knowledge graph 156 can include vector representations (e.g., embeddings 122) of any characteristics, categories or features (e.g., classifications 142) and any specifications of such characteristics, categories or features (e.g., attributes 148) for any number of different products.

[0096] Classifications 142 can be arranged in a hierarchy of various classes 502 and subclasses. A class 502 can include any classification 142 that includes or corresponds to a plurality of specific sub-categories (e.g., sub-classes 504). For instance, with respect to electronics, a class 502 can be directed to a smartphone with sub-classes 504 of memory size and display type. For instance, with respect to apparel, a classification 142 can include a class 502 for a shoe or shoes, which can include or correspond to sub-classes 504 of shoes, including, sandals, boots and sports shoes. A sub-class 504, such as a sub-class 504 of boots of a class 502 shoe, can include additional sub-classes 504 of its own, such as a sub-class 504called high boots and a sub-class 504 called ankle boots. Similarly, sub-classes 504 of sports shoes (e.g., of class 502 of shoes) can include additional sub-classes 504 of running shoes sub-class 504 or sneakers 504 (e.g., within sub-class 504 called sports shoes). Each of these classes 502 and sub-classes can have attributes 148 describing or specifying them, such as specific colors or brand names.

[0097] Hierarchy between the classes 502 or sub-classes 504 and among different subclasses 504 and their own sub-classes 504 can be explicit or implicit. For instance, explicit relations E can include relations or mappings between different classes 502 and sub-classes 504 determined or defined explicitly, such as by defining relations between them based on data or determined using ML models (e.g., 112, 114 or 116). Explicit relations 510 can include defined relations between two or more characteristics (e.g., classes or sub-classes) or attributes. Implicit relations 512 can be determined using the DRG 150 or via ML modeling (e.g., using ML models 112, 114 or 116) based on distances or similarity determinations or values between the vector embeddings 122 of different classifications 142 or attributes 148. Relations between embeddings 122 can be defined, for example, via a distance in the knowledge graph 156, such as by where they map with respect to other embeddings 122. For instance, relations can be defined using values of similarity determinations (e.g., as determined by a similarity function 118) in the vector space with respect to two embeddings 122 of two classifications or attributes compared.

[0098] FIG. 6 is an example a process flow of a method 600 for ML-based generating of hierarchical data relations, in accordance with embodiments of the present solution. Method 600 can be implemented using the systems and components described, for example, in system 100 and using computing system 200. Method 600 can include ACTS 605-625. At ACT 605, a first subset of attributes can be identified from image data. At ACT 610, a second subset of attributes can be identified from textual data. At ACT 615, a third subset of attributes can be identified from a remote website. At ACT 620, a weight of an attribute from the plurality of attributes can be determined. At ACT 625, an indication can be provided using the weight of the attribute in response to a query.

[0099] At ACT 605, a first subset of attributes can be identified from image data. Method 600 can include a DSP comprising one or more processors coupled with memory identifying, based on an image of an object of a first entity input into a first machine learning model trained on a plurality of images of a plurality of objects, a first subset of a plurality of attributes of the object. The first machine learning model can be an image model trained on a number of images of a number of products. The image model can be trained to identify and point out objects and attributes of the objects from product images. The first entity can include a computer system (e.g., server or cloud-based service) of an entity (e.g., enterprise, corporation, e-commerce company or an organization) providing, illustrating or describing a product using or more images or textual descriptions (e.g., phrases, names, values, numbers or identifiers) of the product.

[0100] For example, an image model of a DSP can identify one or more objects and their corresponding attributes from an image of a product. Objects can include one or more categories of features, characteristics or descriptors of a product, such as for example, a silhouette, style, color and length of a product that is a dress. Attributes of such a dress can include descriptors or specifications of the object, such as a wrap dress for the silhouette, black for the color, mini for a dress length and feminine / romantic for a style. Objects can include, for example, a make and model, color, transmission type, engine type and sunroof option of a product that is a vehicle. Attributes for such a vehicle can include the name of the manufacturer and the model of the vehicle for the make and model, a white for a color, an automatic for a transmission type, electric for engine type and an indicator of presence or absence of the sunroof on the vehicle (e.g., yes or no).

[0101] The attributes can describe or specify the objects, such as for example: a silhouette of the object, a color of the object, an occasion associated with the object, a style of the object, a material included in the object, a category of the object, an embellishment of the object, a length of the object, a width of the object, a type of a fit of the object, a pattern of the object, a length of a portion of the object, a width of a portion of the object, a type of a feature of the object, a technology of the object, a brand of the object, a manufacturer of the object, a model name of the object, a size of the object, a shape of the object, a weight of theobject, a product line of the object, an output of the object, a memory of the object, a performance of the object, a processing capability of the object, a power specification of the object and an energy specification of the object.

[0102] At ACT 610, a second subset of attributes can be identified from textual data. Method can include the DPS identifying based on a description of the object of the first entity input into a second machine learning model trained on a plurality of descriptions of the plurality of objects, a second subset of the plurality of attributes of the object. The second machine learning model can be an NLP model trained using descriptions, catalogues, webpages or other textual information corresponding to products.

[0103] The NLP model can identify the second subset of the plurality of attributes, where the second subset can include at least some attributes that are same as or similar to (e.g., included in) the first subset. The second subset of attributes can include one or more attributes that are not same as (e.g., not included in) the first subset of the plurality of attributes. The NLP model of the DPS can identify the second subset of the plurality of attributes based on a search of a website of the first entity.

[0104] At ACT 615, a third subset of attributes can be identified from a remote website. The method can include the DPS identifying from a website of a second entity, a third subset of the plurality of attributes of the object. The third subset of the plurality of attributes can include at least one or more attributes that are same or similar as (e.g., included in) the attributes from the first subset or the second subset. The third subset of the plurality of attributes can include at least one or more attributes that are different from (e.g., not included in) the attributes from the first subset or the second subset.

[0105] DPS can identify the third subset of the plurality of attributes based on a search of the website of the second entity. DPS can identify the third subset based on a search of textual information (e.g., catalogues, product information, user guides, instruction manual or any other textual information) of the product on another website or provided or described by another vendor or entity.

[0106] The method can include the ingestor function and / or DRP of the DPS for generating the plurality of attributes of the object using the first subset, the second subset andthe third subset. The DSP can identify the object based on a unique identifier of the product on the website of the second entity. The DSP can identify the third subset from the website of the second entity using at least one of: a serial identifier of the product, a model identifier of the object, a stock keeping unit (SKU) of the product or a universal product code (UPC) of the product, a European Article Number (EAN) of the product, an international standard book number (ISBN) of the product, an ASIN (amazon standard identification number) of the product, a manufacturer part number (MPN) of the product, and a uniform resource locator (URL) of the product.

[0107] At ACT 620, a weight of an attribute from the plurality of attributes can be determined. Method can include the DPS determining a weight of a first attribute of the plurality of attributes of the object. DPS can determine the weight of the first attribute according to a number of instances of the first attribute identified in the first subset, the second subset and the third subset. The weight can be a value, such as a value from 0-1 to scale the importance or strength (e.g., weight) of a particular attribute according to results of a similarity search of the given attribute with respect to other attributes.

[0108] For example, DPS can determine for each respective attribute of the plurality of attributes, a respective weight according to a number of instances of occurrence of each respective attribute in the first subset, the second subset and the third subset. DPS can identify the first attribute within the third subset of the plurality of attributes, wherein the first attribute in not identified in the first subset of attributes or the second subset of attributes.

[0109] DPS can train the third model to generate, for each respective attribute of the plurality of attributes, a respective embedding according to the respective weight of each respective attribute. The third model can be a vectorizing model for providing vectors for chunks of data corresponding to objects and / or attributes according to their weights.

[0110] At ACT 625, an indication can be provided using the weight of the attribute in response to a query. The method can include the DPS providing an indication of the object responsive to a query determined by a third model to correspond to the object based on the weight of the first attribute. For example, a user from a remote client device can send a query, via a network (e.g., internet) to the first entity housing or including a DPS.Vectorizing model of the DPS can determine, based on a similarity search between the content of the incoming query and the attributes of one or more objects of the product that the query corresponds to the particular product. In response to this determination, the response function of the DPS can provide an indication. The indication can include, for example, a webpage of the requested product, a link to the requested product, an image of the requested product or textual data (e.g., description) of the requested product.

[0111] DPS can receive the query from user of a website of the first entity. DPS can determine the weight of the first attribute based on the third subset. DPS can provide the indication within a webpage corresponding to the product on the website of the first entity. DPS can determine, based on the first attribute, that the query corresponds to the object.

[0112] FIG. 7 is an example a process flow of a method 700 for providing responses to client device queries using a hierarchical data relations. Method 700 can be implemented using the systems and components described, for example, in system 100 and using computing system 200. Method 700 can include ACTS 705-730. At ACT 705, a first embedding based on a first description of a product and a second embedding based on a second description of the product can be identified. At ACT 710, a classification of the product and an attribute of the classification can be generated using the first and the second embeddings. At ACT 715, a query from a client device can be received. At ACT 720, a match between an embedding of a query and an embedding of an attribute can be identified. At ACT 725, a response to the query can be provided.

[0113] At ACT 705, a first embedding based on a first description of a product and a second embedding based on a second description of the product can be identified. The method can include one or more processors coupled with memory and configured (e.g., via instructions and data stored in the memory) to identify a first embedding generated based on a first description of a product input into one or more machine learning models. The method can include the one or more processors also identifying a second embedding generated based on a second description of the product input into the one or more machine learning models.

[0114] The first description can be a description of the product from a first website, a first set of catalogues or a first group of documents of a product from a first entity (e.g., afirst corporation or organization describing or selling the product). The second description can be a description of the product from a second website or web page (e.g., or a second group of catalogues or documents) from a second entity (e.g., a second corporation or organization describing or selling the product). The first website can include a first product data on a product, including a first description that can include different nomenclature, descriptions or phrases describing the product than descriptions, phrases and nomenclature of a second description of the product. The ingestor function can gather the product data from the first website of the first entity and the product data from the second website of the second entity.

[0115] The method can include the ingestor function identifying the first description of the product using a first webpage of the product from a first website by a first entity. The method can include the ingest function identifying the second description of the product using a second web page of the product from a second website of a second entity. The ingestor function can utilize any one or more ML models for this purpose. For instance, the one or more ML models can include a transformer-based graph neural network model trained on a dataset of information on a plurality of products. The products can include any type and form of products, such as products of apparel, electronics, books, home decor, books, tools or equipment, toys and games, kitchenwarejewelry, food, sports gear, furniture, musical instruments or any other goods.

[0116] At ACT 710, a classification of the product and an attribute of the classification can be generated using the first and the second embeddings. The method can include the one or more processors using the one or more machine learning models to generate a classification of the product and an attribute of the classification based on a relation between the first embedding and the second embedding. For instance, a data relations generator can determine the relationship between the first embedding of a first description of the product from a first entity and the second embedding of a second description of the product from a second entity based on a weight. The weight can be determined or associated using a similarity function to determine a similarity (e.g., similarity value) between a portion of the first description and a portion of the second description, suchas an embedding of the portion of the first description and an embedding of the portion of the second description. The data relations generator can determine a weight for the attribute based on a number of instances of occurrence of a first word associated with the first embedding and a number of instances of occurrence of a second word associated with the second embedding.

[0117] The method can include the data relations generators using one or more machine learning models to generate a data structure comprising a taxonomy of the product. The taxonomy can include a hierarchical organization or arrangement of classifications. Each of the classifications can be associated with one or more classifications (e.g., classes having sub-classes) or attributes. For instance, the taxonomy can include the classification associated with the attribute and a second classification associated with a second attribute.

[0118] The data relations generator can map the query to the taxonomy of the product based on the match and a second match between a second embedding of the content with an embedding of the second attribute. The data relations generator can generate a knowledge graph of embeddings of a plurality of attributes associated with a plurality of classifications of a plurality of products. The data relations generator can generate the embedding of the attribute based on the first embedding and the second embedding. The data relations generator can associate the embedding of the attribute with the classification of a plurality of classifications of a taxonomy of the product.

[0119] The data processing system can utilize the data relations generators to store the embedding of the attribute into knowledge graph of embeddings. The knowledge graph can identify a plurality of embeddings of a plurality attributes corresponding to a plurality of classifications of a plurality of products. The plurality of products can include the product having the embedding of the attribute of the plurality of embeddings of the plurality of attributes.

[0120] The data relations generator can identify a third embedding of the product based on an image of the product input into a machine learning model of the one or more machine learning models trained on a plurality of images of a plurality of products. The data relations generator can generate, using the one or more machine learning models, a secondattribute for the classification of the product based on the third embedding. The second attribute of the classification can be utilized to identify the product from client queries.

[0121] At ACT 715, a query from a client device can be received. The method can include a data processing system receiving a query from a client device. The query can include a content. The content can include a statements from a user describing a product, features of a product, characteristics of the product or functionalities of the product. The content of the client device query can include, correspond to, or be contextually similar to one or more attributes or classifications (e.g., classes or sub-classes) involving the product.

[0122] At ACT 720, a match between an embedding of a query and an embedding of an attribute can be identified. The method can include the one or more processors configured to identify the product based on a match between an embedding of the content and an embedding of the attribute of the classification. For instance, the data relations generator can determine or detect (e.g., using one or more ML models, such as a vectorizing model) a match between an embedding of a portion of the content of the query from the client device and an embedding of the attribute of a class or a sub-class of the taxonomy of the product.

[0123] The data relations generator can identify a plurality of matches between embeddings of a plurality of attributes of one or more classifications (e.g., classes or subclasses) of the product and embeddings of a plurality of portions of the content of the query from the client device. The data relations generator can identify a plurality of matches between embeddings of a plurality of attributes of one or more attributes of the classifications of the product and embeddings of a plurality of portions of the content of the query from the client device. Based on such matches of the embeddings, the data relations generator can identify the product as the product that is suitable to include in the response to the query.

[0124] The data relations generator can identify, based on a distance in the knowledge graph between the embedding of the content and the embedding of the attribute satisfying a threshold for similarity, a data structure of the product. For example, the data relations generator can identify the product by comparing the distances or similarities of the embeddings of the incoming query content with the embeddings of various products (e.g., embeddings of attributes and classifications of various products) in the knowledge graph.Upon identifying the most similar (e.g., similarity function values satisfying or exceeding threshold similarity values) attributes or classifications, the data relations generator can identify the product of the most closely matching embeddings as the product to provide in response to the query.

[0125] The data relations generator can utilize one or more ML models (e.g., vectorizing model) to identify the similarity value (e.g., distance) between one or more embeddings of the query content and one or more embeddings of attributes or embeddings of classifications of the product. The data relations generator can identify the data structure of the matching product based on the matching of the attributes. The data structure can include any information about the product, such as a link to purchase the product, description of a product, attributes or classifications of the product, any product data from any number of websites of any number of entities. The data relations generator can generate, based on the identified data structure, the response comprising the information about the product or generate a content for the response.

[0126] At ACT 725, a response to the query can be provided. The method can include the one or more processors providing a response to the query, based on a match between the embedding of the attribute and the embedding of the content of the query. The response can include any information about the product. The information about the product can include the product description, specification, performance parameters, dimensions, color, materials utilized, brand name, make and model names, or any other information, characteristics or attributes of the product.

[0127] The response function can generate the response comprising the information about the product, based on, or using, the data structure identified based on a distance between the embedding of the content of the query and the embedding of the attribute of a classification of the product. For example, the data processing system can determine, based on the match between the embeddings, that the content of the query corresponds to the classification of the product and generate the response comprising the information corresponding to the classification and the attribute. For example, the response function or the data relations generator can identify, from the plurality of products, the product based onthe match of the attribute associated with the classification of the products in the knowledge graph.

[0128] The data relations generator and the response function can identify the product based on a second match between a second embedding of the content and an embedding of the second attribute and provide the response (e.g., for transmission to the client device, via the network), based on the second match.

[0129] The foregoing examples have been provided merely for the purpose of explanation and are in no way to be construed as limiting of the present disclosure. While aspects of the present disclosure have been described with reference to an exemplary embodiment, it is understood that the words which have been used herein are words of description and illustration, rather than words of limitation. Changes may be made, within the purview of the appended claims, as presently stated and as amended, without departing from the scope and spirit of the present disclosure in its aspects. Although aspects of the present disclosure have been described herein with reference to particular means, materials and embodiments, the present disclosure is not intended to be limited to the particulars disclosed herein; rather, the present disclosure extends to all functionally equivalent structures, methods and uses, such as are within the scope of the appended claims.

[0130] The systems described above can provide multiple ones of any or each of those components and these components can be provided on either a standalone system or on multiple instantiation in a distributed system. In addition, the systems and methods described above can be provided as one or more computer-readable programs or executable instructions embodied on or in one or more articles of manufacture. The article of manufacture can be cloud storage, a hard disk, a CD-ROM, a flash memory card, a PROM, a RAM, a ROM, or a magnetic tape. In general, the computer-readable programs can be implemented in any programming language, such as LISP, PERL, C, C++, C#, PROLOG, or in any byte code language such as JAVA. The software programs or executable instructions can be stored on or in one or more articles of manufacture as object code.

[0131] The subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware,including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. The subject matter described in this specification can be implemented as one or more computer programs, e.g., one or more circuits of computer program instructions, encoded on one or more computer storage media for execution by, or to control the operation of, data processing apparatuses. Alternatively, or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. While a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate components or media (e.g., multiple CDs, disks, or other storage devices include cloud storage). The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0132] The terms “computing device”, “component” or “data processing apparatus” or the like encompass various apparatuses, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize variousdifferent computing model infrastructures, such as web services, distributed computing and grid computing infrastructures.

[0133] A computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program can correspond to a file in a file system. A computer program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0134] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatuses can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Devices suitable for storing computer program instructions and data can include non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0135] The subject matter described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a web browser through which auser can interact with an implementation of the subject matter described in this specification, or a combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0136] While operations are depicted in the drawings in a particular order, such operations are not required to be performed in the particular order shown or in sequential order, and all illustrated operations are not required to be performed. Actions described herein can be performed in a different order.

[0137] Having now described some illustrative implementations, it is apparent that the foregoing is illustrative and not limiting, having been presented by way of example. In particular, although many of the examples presented herein involve specific combinations of method acts or system elements, those acts and those elements may be combined in other ways to accomplish the same objectives. Acts, elements and features discussed in connection with one implementation are not intended to be excluded from a similar role in other implementations or implementations.

[0138] The phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including” “comprising” “having” “containing” “involving” “characterized by” “characterized in that” and variations thereof herein, is meant to encompass the items listed thereafter, equivalents thereof, and additional items, as well as alternate implementations consisting of the items listed thereafter exclusively. In one implementation, the systems and methods described herein consist of one, each combination of more than one, or all of the described elements, acts, or components.

[0139] Any references to implementations or elements or acts of the systems and methods herein referred to in the singular may also embrace implementations including a plurality of these elements, and any references in plural to any implementation or element or act herein may also embrace implementations including only a single element. References inthe singular or plural form are not intended to limit the presently disclosed systems or methods, their components, acts, or elements to single or plural configurations. References to any act or element being based on any information, act or element may include implementations where the act or element is based at least in part on any information, act, or element.

[0140] Any implementation disclosed herein may be combined with any other implementation or embodiment, and references to “an implementation,” “some implementations,” “one implementation” or the like are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with the implementation may be included in at least one implementation or embodiment. Such terms as used herein are not necessarily all referring to the same implementation. Any implementation may be combined with any other implementation, inclusively or exclusively, in any manner consistent with the aspects and implementations disclosed herein.

[0141] References to “or” may be construed as inclusive so that any terms described using “or” may indicate any of a single, more than one, and all of the described terms. References to at least one of a conjunctive list of terms may be construed as an inclusive OR to indicate any of a single, more than one, and all of the described terms. For example, a reference to “at least one of ‘A’ and ‘B’” can include only ‘A’, only ‘B’, as well as both ‘A’ and ‘B’. Such references used in conjunction with “comprising” or other open terminology can include additional items.

[0142] Where technical features in the drawings, detailed description or any claim are followed by reference signs, the reference signs have been included to increase the intelligibility of the drawings, detailed description, and claims. Accordingly, neither the reference signs nor their absence has any limiting effect on the scope of any claim elements.

[0143] Modifications of described elements and acts such as substitutions, changes and omissions can be made in the design, operating conditions and arrangement of the disclosed elements and operations without departing from the scope of the present disclosure.

[0144] References to “approximately,” “substantially” or other terms of degree include variations of + / - 10% from the given measurement, unit, or range unless explicitly indicated otherwise. Coupled elements can be electrically, mechanically, or physically coupled with one another directly or with intervening elements. Scope of the systems and methods described herein is thus indicated by the appended claims, rather than the foregoing description, and changes that come within the meaning and range of equivalency of the claims are embraced therein.

Claims

CLAIMSWhat is claimed is:

1. A data processing system, comprising: one or more processors coupled with memory to: identify a first embedding generated based on a first description of a product input into one or more machine learning models; identify a second embedding generated based on a second description of the product input into the one or more machine learning models; generate, using the one or more machine learning models, a classification of the product and an attribute of the classification based on a relation between the first embedding and the second embedding; receive, from a client device, a query comprising a content; identify the product based on a match between an embedding of the content and an embedding of the attribute of the classification; and provide, based on the match, a response to the query comprising information about the product.

2. The system of claim 1, comprising the one or more processors to: generate, using the one or more machine learning models, a data structure comprising a taxonomy of the product, the taxonomy comprising the classification associated with the attribute and a second classification associated with a second attribute; map the query to the taxonomy of the product based on the match and a second match between a second embedding of the content with an embedding of the second attribute.

3. The system of claim 1, comprising the one or more processors to: identify the first description of the product using a first webpage of the product from a first website by a first entity; andidentify the second description of the product using a second web page of the product from a second website of a second entity.

4. The system of claim 1, comprising the one or more processors to: generate a knowledge graph of embeddings of a plurality of attributes associated with a plurality of classifications of a plurality of products; identify, based on a distance in the knowledge graph between the embedding of the content and the embedding of the attribute satisfying a threshold for similarity, a data structure of the product, the data structure comprising the information about the product; and generate, based on the identified data structure, the response comprising the information about the product.

5. The system of claim 1, comprising the one or more processors to: determine, based on the match, that the content of the query corresponds to the classification of the product; and generate the response comprising the information corresponding to the classification and the attribute.

6. The system of claim 1, comprising the one or more processors to: generate the embedding of the attribute based on the first embedding and the second embedding; and associate the embedding of the attribute with the classification of a plurality of classifications of a taxonomy of the product.

7. The system of claim 1, comprising the one or more processors to: store the embedding of the attribute into knowledge graph of embeddings, the knowledge graph identifying a plurality of embeddings of a plurality attributes corresponding to a plurality of classifications of a plurality of products, the plurality of products comprisingthe product having the embedding of the attribute of the plurality of embeddings of the plurality of attributes; and identify, from the plurality of products, the product based on the match of the attribute associated with the classification of the products in the knowledge graph.

8. The system of claim 1, wherein the one or more machine learning models comprise a transformer-based graph neural network model trained on a dataset of information on a plurality of products comprising the product.

9. The system of claim 1, comprising the one or more processors to: determine the relationship between the first embedding and the second embedding based on a weight associated with a similarity between a portion of the first description and a portion of the second description.

10. The system of claim 1, comprising the one or more processors to: determine a weight for the attribute based on a number of instances of occurrence of a first word associated with the first embedding and a number of instances of occurrence of a second word associated with the second embedding.

11. The system of claim 1, comprising the one or more processors to: identify a third embedding of the product based on an image of the product input into a machine learning model of the one or more machine learning models trained on a plurality of images of a plurality of products; generate, using the one or more machine learning models, a second attribute for the classification of the product based on the third embedding; identify the product based on a second match between a second embedding of the content and an embedding of the second attribute; and provide, based on the second match, the response.

12. The system of claim 1, comprising the one or more processors to: generate, using the one or more machine learning models, a data structure of the product comprising the classification and the information about the product including a link to a web page about the product; and generate, based on the match, the response comprising the link to the page about the product.

13. A method, comprising: identifying, by one or more processors coupled with memory, a first embedding generated based on a first description of a product input into one or more machine learning models; identifying, by the one or more processors, a second embedding generated based on a second description of the product input into the one or more machine learning models; generating, by the one or more processors, using the one or more machine learning models, a classification of the product and an attribute of the classification based on a relation between the first embedding and the second embedding; receiving, by the one or more processors, from a client device, a query comprising a content; identifying, by the one or more processors, the product based on a match between an embedding of the content and an embedding of the attribute of the classification; and providing, by the one or more processors, based on the match, a response to the query comprising information about the product.

14. The method of claim 13, comprising: generating, by the one or more processors, using the one or more machine learning models, a data structure comprising a taxonomy of the product, the taxonomy comprising the classification associated with the attribute and a second classification associated with a second attribute;mapping, by the one or more processors, the query to the taxonomy of the product based on the match and a second match between a second embedding of the content with an embedding of the second attribute.

15. The method of claim 13, comprising: identifying, by the one or more processors, the first description of the product using a first webpage of the product from a first website by a first entity; and identifying, by the one or more processors, the second description of the product using a second web page of the product from a second website of a second entity.

16. The method of claim 13, comprising: generating, by the one or more processors, a knowledge graph of embeddings of a plurality of attributes associated with a plurality of classifications of a plurality of products; identifying, by the one or more processors, based on a distance in the knowledge graph between the embedding of the content and the embedding of the attribute satisfying a threshold for similarity, a data structure of the product, the data structure comprising the information about the product; and generating, by the one or more processors, based on the identified data structure, the response comprising the information about the product.

17. The method of claim 13, comprising: determining, by the one or more processors, based on the match, that the content of the query corresponds to the classification of the product; and generating, by the one or more processors, the response comprising the information corresponding to the classification and the attribute.

18. The method of claim 13, comprising: generating, by the one or more processors, the embedding of the attribute based on the first embedding and the second embedding; andassociating, by the one or more processors, the embedding of the attribute with the classification of a plurality of classifications of a taxonomy of the product.

19. The method of claim 13, comprising: storing, by the one or more processors, the embedding of the attribute into knowledge graph of embeddings, the knowledge graph identifying a plurality of embeddings of a plurality attributes corresponding to a plurality of classifications of a plurality of products, the plurality of products comprising the product having the embedding of the attribute of the plurality of embeddings of the plurality of attributes; and identifying, by the one or more processors, from the plurality of products, the product based on the match of the attribute associated with the classification of the products in the knowledge graph.

20. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: identify a first embedding generated based on a first description of a product input into one or more machine learning models; identify a second embedding generated based on a second description of the product input into the one or more machine learning models; generate, using the one or more machine learning models, a classification of the product and an attribute of the classification based on a relation between the first embedding and the second embedding; receive, from a client device, a query comprising a content; identify the product based on a match between an embedding of the content and an embedding of the attribute of the classification; and provide, based on the match, a response to the query comprising information about the product.

Images