Machine learning based input query context control

Abstract

The solutions use machine learning (ML) models with parameters to maintain the response within the context of a query by validating ML model outputs during the generation of the response. A system can receive a query having elements to generate tokens for the elements. The system can provide, via one or more ML models having graph neural networks (GNNs) and transformers, a data structure to identify a relationship between a first node of for a first token and a second node for a second token. The system can validate the relationship between the first and second nodes based on a validation parameter of the data structure and provide, via a ML model trained using descriptions of a plurality of products and based on the data structure, a response to the query to send to the client device, responsive to the query.

Description

MACHINE LEARNING BASED INPUT QUERY CONTEXT CONTROLCROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application claims the benefit of priority under 35 U.S. C. § 119 to U.S. Provisional Patent Application No. 63 / 515,164, filed July 24, 2023, which is hereby incorporated by reference herein in its entirety and for all purposes.TECHNICAL FIELD

[0002] The present disclosure relates generally to machine learning (ML) based input query context, including, but not limited to, controlling the context of input queries via ML models.BACKGROUND

[0003] While searching for information online, users may provide queries for information concerning a particular field or a topic. Computing devices may receive user queries and provide responses that can be sent to the users.SUMMARY

[0004] The technical solutions of this disclosure are directed to machine learning (ML) based query response generation using validation and context control parameters. When using ML models to generate responses to queries from client devices, ML models can output falsepositive determinations, resulting in hallucinations of the ML model. These determinations can lead to increased consumption of computational and energy resources, including additional back and forth communications between the client and the ML-based system, reducing the system’s overall effectiveness and energy efficiency. The technical solutions overcome these and other challenges by validating determinations from neural networks using validation and context control parameters during the ML process.

[0005] In one aspect, the present solution relates to a system. The system can include a data processing system comprising one or more processors coupled with memory to receive a query comprising a plurality of elements and generate a plurality of tokens. Each of the plurality of tokens can correspond to an element of the plurality of elements of the query.The one or more processors can produce, via one or more machine learning (ML) models having a plurality of graph neural networks (GNNs) and a plurality of transformers, a data structure to identify a relationship between a first node of a plurality of nodes of the data structure corresponding to a first token of the plurality of tokens and a second node of the plurality of nodes corresponding to a second token of the plurality of tokens. The one or more processors can validate, via the one or more ML models and based on the data structure, the relationship between the first node and the second node based on a validation parameter of the data structure. The one or more processors can provide, via a model of the ML models, a response to the query. The model of the one or more ML models can be trained using a plurality of descriptions of a plurality of products. The one or more processors can send the response to the client device, responsive to the query.

[0006] In one aspect, the present solution relates to a method. The method can include a data processing system comprising one or more processors coupled with memory receiving a query comprising a plurality of elements. The method can include the data processing system generating a plurality of tokens. Each of the plurality of tokens can correspond to an element of the plurality of elements of the query. The method can include the data processing system producing, via one or more first machine learning (ML) models having a plurality of graph neural networks (GNNs) and a plurality of transformers, a data structure identifying a relationship between a first node of a plurality of nodes of the data structure corresponding to a first token of the plurality of tokens and a second node of the plurality of nodes corresponding to a second token of the plurality of tokens. The method can include the data processing system validating the relationship between the first node and the second node based on a validation parameter of the data structure. The method can include providing, via a model of the one or more ML models and based on the data structure, a response to the query. The model can be trained using a plurality of descriptions of a plurality of products. The method can include the data processing system sending the response to the client device, responsive to the query.

[0007] In one aspect, the present solution relates to a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to receive a query comprising a plurality of elements. The instructions, when executed by the one or more processors, can cause the one or moreprocessors to generate a plurality of tokens. Each of the plurality of tokens can correspond to an element of the plurality of elements of the query. The instructions, when executed by the one or more processors, can cause the one or more processors to produce, via one or more machine learning (ML) models having a plurality of graph neural networks (GNNs) and a plurality of transformers, a data structure to identify a relationship between a first node of a plurality of nodes of the data structure corresponding to a first token of the plurality of tokens and a second node of the plurality of nodes corresponding to a second token of the plurality of tokens. The instructions, when executed by the one or more processors, can cause the one or more processors to validate, via the one or more ML models, the relationship between the first node and the second node based on a validation parameter of the data structure. The instructions, when executed by the one or more processors, can cause the one or more processors to provide, via a model of the one or more ML models and based on the data structure, a response to the query. The model can be trained using a plurality of descriptions of a plurality of products. The instructions, when executed by the one or more processors, can cause the one or more processors to send the response to the client device, responsive to the query.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Aspects of the present disclosure are described in the detailed description which follows, in reference to the noted plurality of drawings by way of non-limiting examples of exemplary embodiments of the present disclosure.

[0009] FIG. 1 is an example block diagram of a system using an ML model to improve context of a query prior to its input into a field-specific trained ML model providing the response, in accordance with embodiments of the present solution.

[0010] FIG. 2 illustrates an example architecture of a computing system implemented in embodiments of the present disclosure.

[0011] FIG. 3 illustrates an example block diagram of a structure of the query context model, in accordance with embodiments of the present solution.

[0012] FIG. 4 is an example a process flow for using an ML model to validate and improve the context of a prompt prior to its input into a field-specific trained ML model providing the response, in accordance with aspects of the present disclosure.DETAILED DESCRIPTION

[0013] The technical solutions of this disclosure apply validation or context parameters to transformer- based graph neural network (GNN) models to improve the accuracy of ML- generated output responses to client device queries. The solutions can facilitate the use of a broad range of ML models trained using datasets directed to various specific fields (e.g., e- commerce, medicine, law, science or technology) to provide ML-generated responses to field-specific client device queries. The solution can use ML models that can include a multilayered transformer and GNN model structure that can use validation and context parameters to ensure that ML generated responses remain within the context of the user queries, resulting in contextually sound responses to user queries.

[0014] When providing ML-based search systems, it can be challenging to accurately determine the intended information that the user is seeking as well as maintaining the ML- generated responses within the intended context. As queries can be directed to different products or information and be worded using a variety of phrases or terms, responses from the ML models can vary based on the choice of words the user provides in the query. The reliance on mere keyword matching may not accurately capture the semantic meaning and the context of the user queries. ML-generated responses to client queries can include false positive results, such as hallucinations or drifts, which can include inaccurate or irrelevant information that can both hinder the user experience and adversely affect computational resources and energy efficiency of the system.

[0015] The technical solutions of this disclosure can overcome these challenges by utilizing validation or context parameters to maintain the transformer-based multi-layered GNN performance within the intended context, during the course ML performance. The technical solutions can utilize multi-layered structure of GNNs and transformers to determine relations between different query elements and provide context-bound input into the field-specific trained ML model, resulting in more accurate responses. The solutions can use encoders to dissect the queries into elements (e.g., words, phrases or sentences) that can be assigned to individual tokens. The tokens can be used to provide contextual information to establish or determine relationships between different elements of the query. The transformer-based multi-layered GNN can allow the query context model to determine and maintainrelationships between the tokens of the query elements and other tokens of the remaining portions of the query, or other related or similar elements (e.g., terms with similar meanings not included in the query). In doing so, the query context model can generate a context data structure having improved or enhanced representation of the context or meaning of the query across multiple layers of tokenized relations.

[0016] The query context model can generate the context data structure that is formed as a nested tree structure of tokens representing relations with respect to other tokens and further relations to other tokens (e.g., corresponding to words and phrases not included in the query). Within this nested tree structure, the relationships between tokens can be represented using values (e.g., weights), defining relations between each of the tokens of the query, creating a hierarchical network of interconnected elements. The query context model can navigate through these relationships, effectively traversing the various contextual paths within the user's query. By capturing and analyzing these contextual relationships, the system can provide highly accurate and contextually relevant responses to the user's query. By using validation and context parameters, the technical solutions can maintain the decisions from the ML models within the desired context, thereby reducing or precluding false positive determinations (e.g., hallucinations or drifts), improving the accuracy and the computational and energy resources of the ML models.

[0017] To train the field-specific ML models to provide the responses to the queries, a diverse and comprehensive dataset for a particular field of interest can be utilized. The training data can include a wide array of descriptors related to any field, such as for example, product catalogues, websites, and specification sheets of products for sale in the context of a ML model for e-commerce. In the context of an e-commerce model implementation, the dataset can include transcript or simulated conversations that customers might have with sales assistants during pre-sales and post-sales interactions, product specification documents and catalogues. The present solution can also include training data from expert sales professionals that can review and curate the training data, further improving the data set.

[0018] In one example, when a user query is received, the encoder can initiate the process by breaking down the query into elements and generating their corresponding individual tokens. The query context model can create and maintain relationships between the identified tokens that can be fed into query context model for context enhancement of the query. The resultingcontext data structure can provide a network of interconnected and interrelated weighted tokens, which can be used as an input into the field-specific trained ML model to produce the response. While generating the response, the transformer-based GNN models can utilize validation or context parameters to verify that the determinations are within the parameters, taking corrective actions when outputs do not align with the parameters and thus improving the accuracy of the determinations. The decoder can utilize context data structure to convert the ML model provided response into more contextual user-tailored response. The decoder can take advantage of the context provided by the GNN to ensure that the responses are delivered in a manner that resonates with the user, mirroring interactions and the context in which the query was given. In doing so, the present solution can provide a context-driven approach to provide improved responses to user’s queries for field-specific ML model-based search systems.

[0019] The solutions can provide an e-commerce aware large language model (LLM) trained using data of sales, post sales and e-commerce specific data that can handle efficient embedding of product data or catalogues for fast and efficient discovery using natural language queries. The solutions can include a data processing system using graph transformers based on multi-dimensional nested GNN combining encoder and decoder models to drive the field-specific (e.g., e-commerce) inference engine that can use ensemble models with validation or context parameters to maintain the transformer-based GNN determinations within the context of the relevant nodes and the context of the e-commerce queries.

[0020] FIG. 1 is an example block diagram of a system 100 for machine learning (ML) based query response generation using validation and context control parameters. System 100 can be directed to using an ML model to improve context of a query prior to its input into a fieldspecific trained ML model providing the response. Example system 100 can include a data processing system (DPS) 102 that can communicate with a client device 106, via network 104, one or more queries 108 having elements 110 as well as responses 116 responsive to the queries 108. DPS 102 can include one or more queries 108 along with their elements 110, one or more model trainers, NLP models 114, responses 116, similarity functions and tokenization functions 120 producing tokens 122. DPS 102 can include one or more encoders 124, decoders 126, training datasets 130 in repositories 132 and one or more query contextmodels (QCM) 140. Each QCM 140 can include or utilize one or more graph neural networks (GNNs), transformers, nodes 146, node features 148 and context data structures (CDSs) 150. Each context data structure can include one or more of validation parameters 152 and context parameters 154 for validating the determinations of the QCMs 140.

[0021] System 100 can include any combination of hardware and software for providing responses to client device queries using one or more ML models according to the context of the queries. System 100 can include the system for providing ML model generated responses to user queries using field-specific trained ML models and validation parameter controlled transformer-based graph neural networks (GNNs). System 100 can include a data processing system 102, which can include or be executed on any computing device (e.g., a server, a cloud-based service or any other computing device or a system. System 100 can include, or be executed using, a computing system 200 (e.g., described in EIG. 2), which can executed the data processing system 102 on one or more processors (e.g., 210) that can be coupled with one or more memories (e.g., 215) that can store any commands, instructions and / or data for executing or implementing the functionalities of the data processing system 102 or its components.

[0022] Query 108 can include any content, such as textual, media or image content, which can elicit or seek a response from the data processing system 102. Query 108 can include any textual or text-based input from a client device 106 seeking specific information, a response to a question, or content within, or according to, a given context, dataset or a domain (e.g., field, such as e-commerce, law, medicine, engineering or technology). Queries 108 can be generated by users on client devices 106, using applications (e.g., a web browser and a graphical user interface) allowing the user to enter any string of characters of the query.

[0023] Query 108 can include natural language elements 110, such as sentences, phrases, words, portions of words, or any other portions of a text. Elements 110 can include, be tied to, or related to, a particular meaning or a context. Elements 110 can be tokenized and processed by a tokenization function 120 of the data processing system 102. Upon tokenization, elements 110 can have their corresponding tokens 122, which can include numerical representations of the elements 110.

[0024] Tokenization function 120 can include any combination of hardware and software for breaking down the query 108 into elements 110 and creating tokens 122 for each element110 (e.g., each syllables, words or sentences). Tokenization function 120 can assign a numerical representation (e.g., a token identifier) for each individual token 122 (e.g., corresponding to each individual element 110). Tokenization function 120 can facilitate efficient information retrieval and understanding by the data processing system 102 using the numerical representations of the tokens. Once tokenized, the tokenization function 120 can generate embeddings corresponding to the tokens 122. The embeddings can represent one or more tokens 122 using vector representations (e.g., in a vector space). The embeddings can represent single token or a sequence of tokens (e.g., a contextual embedding of the tokens of the query 108). The tokenization function 120 can employ or work together with a QCM 140 (e.g., multi-layered arrayed GNNs 142 and transformers 144) to analyze the contextual relationships and dependencies between the tokens 122 (e.g., vector representations or embeddings of the tokens), such as for example to analyze contextual relations between nodes 146 of the QCM 140 corresponding to the tokens 122 to gain a comprehensive understanding of the context of the query 108.

[0025] Natural language processing (NLP) models 114 can include any ML models trained using a plurality of descriptions, texts or corpuses pertaining to a particular field. NLP model 114 can include any ML model (e.g., a computational framework), based on deep learning techniques. NLP model 114 can be trained to process and understand human language, enabling or performing tasks such as text classification, sentiment analysis, language translation, and question answering (e.g., providing responses 116 to queries 108), by leveraging patterns, semantics, and context within textual data.

[0026] An NLP models 114 can be trained based on a large training dataset 130 of descriptions or texts corresponding to any particular field. For example, NLP model 114 can be trained on e-commerce using various e-commerce related textual data, such as product descriptions, interactions between salespersons and customers, frequently asked questions (FAQs), or any other data pertaining to responding to user questions in an e-commerce environment, such as queries 108 for purchasing different products or services. For example, NLP model 114 can be trained in any specific field, such as medicine, law, engineering, technical support, history, geography, translation of different languages, or any other field that users may explore with their queries 108.

[0027] NLP model 114 can be trained for providing responses 116 with respect to a particular field by a model trainer 112. Model trainer can include any combination of hardware and software for training an NLP model 114 using a particular training dataset 130. Model trainer 112 can guide the learning process of a NPL model 114 by optimizing its parameters and weights based on the field-specific dataset.

[0028] The training dataset 130 can include input samples, such as user queries 108 or sentences of text corpuses, seller-customer interactions or other data along with the corresponding target responses or labels, forming a supervised learning setup. During training, the model trainer 112 can iteratively feed batches of input data into the NLP model 114, which can then predict the responses 116. Model trainer 112 can compare these predictions with the ground truth responses from the training dataset 130 and provide model's prediction errors. Through backpropagation, model trainer 112 can update the parameters of the NLP model, minimizing the errors and improving the NLP model’s ability to generate accurate responses 116. This process continues for multiple epochs, until the NLP model 114 reaches a satisfactory level of performance on the field-specific dataset 130, resulting in a well-trained NLP ML model.

[0029] Tokenization function 120 can include any combination of hardware and software for dividing query 108 into elements 110 and assigning tokens 122 to the elements 110. A token 122 can include one or more elements of a query 108 or a portion of an element 110 of a query 108. Each token, or a sequence of tokens 122, can be represented as one or more values or a vector, such as an embedding.

[0030] Encoder 124 can include any combination of hardware and software for transforming queries 108 and their elements into meaningful representations (e.g., context data structure 150) that can be input into the QCM 140 to generate a response 116. Encoder 124 can include a component of a ML model responsible for transforming input data (e.g., elements 110 of queries 108) into meaningful and compact representations (e.g., context data structure 150). Encoder 124 can process textual input, utilize a tokenization function 120 to tokenize the input into smaller units (e.g., words or sub- words) and map each token 122 to a corresponding one or more values, embeddings or feature vectors. Encoder 124 can utilize the context data structure 150 to convert the user’s choice of words from the query 108 intoan input that includes an improved contextual and semantic information within the input data, allowing NLP model 114 to understand relationships between different tokens 122.

[0031] Similar to encoder 124, decoder 126 can include any combination of hardware and software for transforming responses 116 into desired or expected representation to be provided to the client device 106. Decoder 126 can include the functionality to transform responses 116 into output sequences to be provided to the client device 106. For example, decoder 126 can take a context data structure 150 generated by the QCM 140 and / or encoder 124 and use it to generate coherent and contextually appropriate responses 116 to be provided to the client device 106. Decoder 126 can be used to provide language translation, text generation, and summarization for the response to be sent to the client device 106. Decoder 126 can utilize learned knowledge from the encoder 124 to produce accurate and coherent sequences of words for the response 116.

[0032] Query context model (QCM) 140 can include any combination of hardware and software for providing machine learning based validation, improvement or enhancement of context of the queries 108 prior to their input into the NLP model 114 providing the response 116. QCM 140 can include a nested tree structure of nodes 146 that can be used to represent hierarchical relationships between elements 110, tokens 122 or embeddings of tokens 122 of the query 108 (e.g., identified in node features 148). The nested tree structure of the QCM 140 can include arrays of GNNs 142 and transformers 144 facilitating the integration of both sequential and graph-based dependencies of the nodes 146. QCM 140 can include capabilities or functions to utilize validation parameters 152 or context parameters 154 to validate determinations or outputs from the ML models (e.g., QCM 140, transformers, GNNs 142 or NLP models 114). Such validations of the ML models can include utilizing validation parameters 152 or context parameters 154 to keep those ML model outputs or determinations within the predetermined context. QCM 140 can combine the transformers 144 and GNNs 142 to create context-rich representations (e.g., node features 148) between different nodes 146, corresponding to various elements 110 of the query 108 as well as similar elements 110 (e.g., words, phrases) not included in the query 108, but having a related or similar meaning (e.g., determined based on a similarity function 118).

[0033] Graph Neural Network (GNN) 142 can include any deep learning architecture designed to process and model data with complex graph structures. GNN 142 can operate ongraph-structured data in which elements 110 of queries 108 can be represented as nodes 146. GNN 142 can include the functionality to learn or develop relations in node features 148, providing representations of contextual information and dependencies between nodes 146 in the graph. At each layer of the GNN, nodes 146 can aggregate information from their neighboring nodes 146 in the GNN 142, updating their own representations based on this aggregated information from the neighboring nodes 146. This iterative process of updating nodes 146 and the node features 148 can allow the GNN to propagate information across the graph, capturing global dependencies and contextual relationships identified in node features 148 (e.g., for each layer of nodes 146).

[0034] Nodes 146 in the Query Context Model (QCM) 140 can include units, components or numerical constructs representing elements 110 of a query 108 within a nested tree structure. Nodes 146 can form the building blocks of a graph, acting as distinct points representing specific tokens 122 of elements 110 of the input query 108. For example, a query 108 of "Which mobile phones are best for gaming?" can include nodes 146 that represent individual words or phrases (e.g., elements 110), such as "mobile phones" and "gaming." Each node 146 can represent or indicate the identity of a given element 110 and can serve for establishing relations with other nodes 146 in the graph, which can be used the ML models to implement the hierarchical and sequential dependencies for the query 108.

[0035] Node features 148 can include any features, attributes or properties associated with each node 146. Node features 148 can provide contextual information for processing of the nodes 146 by the GNNs and transformers 144 within the QCM 140. Node features 148 can include, or be represented as, vectors or tensors encoding or numerically representing various aspects of the node 146. Node features 148 can include or represent word embeddings, positional encodings, semantic meanings, and contextual relevance. For instance, in a query 108 "Which mobile phones are best for gaming?", the node 146 representing "mobile phones" can include node features 148 such as its semantic embeddings derived from pretrained language models, its position in the query, and any relevant contextual clues that relate it to other nodes like "gaming." By associating the nodes 146 with the node features 148, the QCM 140 or the response function 160 can create context-rich representations that enhance the understanding and processing of complex queries 108, resulting in more accurate and relevant responses.

[0036] Transformer 144 can include an architecture for natural language processing tasks, characterized by its attention mechanism and parallel processing capabilities. For instance, transformers 144 can process input data in parallel (e.g., as opposed to processing data sequentially), making transformers 144 more efficient in terms of processing and more suitable for long-range dependencies. Transformer 144 can include a self-attention mechanism that can allow the transformer 144 to capture contextual relationships between words in a query 108 by assigning different weights to each element 110 based on its importance to other words in the same sequence (e.g., same layer). Through self-attention, transformer 144 can effectively learn complex linguistic patterns and context, making it useful for tasks, such as machine translation, sentiment analysis, and text generation.

[0037] After processing the elements 110 of a query 108, the tokens 122 of the elements 110 (e.g., or their embeddings) can be processed through layers of transformers 144 that can generate contextual relationships between the elements 110 of the query 108, as well as other elements 110 (e.g., words or phrases) not included to the query 108, but which can be related to it though meaning or context. A graph structure can be constructed in which nodes 146 represent the tokens 122 of the elements 110. Relationships between the tokens 122 can be represented as node features 148, allowing for the incorporation of graph- based information. The GNN-like information aggregation process can aggregate data from neighboring nodes 146 from the structure, enabling the QCM model 140 to capture contextual information and global relationships between the nodes 146. The outputs from the layers of transformers 144 and GNNs 142 can be combined to create hierarchical contextual representations, resulting in a multi-layered nested tree structure.

[0038] QCM 140 can utilize a similarity function 118 (e.g., Euclidean distance or cosine similarity function) to generate and apply weights to tokens 122 and determine relationships (e.g., similarities) between the nodes 146. For instance, depending on the similarity function 118, QCM 140 can determine that some nodes 146 (e.g., corresponding to some tokens 122) are more similar to each other (e.g., more relevant to each other) than other nodes 146. Such relationships between nodes 146 can be stored (e.g., in repository 132, such as a hard disk) as node features 148. The relations between the nodes 146 can include indications of relationship (e.g., similarity or contextual relation) of nodes 146 corresponding to each of the tokens 122 of the query 108 and other tokens 122 (e.g., words or phrases) not found in thequery 108, but available in the NLP model 114 (e.g., using phrases or descriptions of similar features or concepts in training datasets 130) to describe a related searched feature or topic.

[0039] QCM 140 can generate data structure 150 for identifying, determining or relationships between the nodes 146. Context data structure 150 can include, for example, numerical representations of nodes 146, node features 148 or contexts of the queries 108. Context data structure 150 can include embeddings or vector representations of nodes 146 corresponding to tokens 122 of the elements 110 of the queries 108. Context data structure 150 can include any one or more of validation parameters 152 or context parameters 154 for any node features 148, nodes 146 (e.g., of elements 110) or queries 108.

[0040] The context data structure 150 can include a comprehensive framework or a structure to facilitate managing and maintaining the contextual integrity of queries processed by the QCM 140. The context data structure 150 can include parameters, such as validation parameters 152 or context parameters 154 that can facilitate maintaining the accurate interpretation and response generation for queries 108. The context data structure 150 can include hierarchical relationships, validation routines, and contextual information used for precise query handling. For instance, in an e-commerce scenario, the context data structure 150 can include elements, such as, product categories, user preferences, and relevant attributes, organized in a way that preserves the logical flow and meaning of the original query 108.

[0041] Validation parameter 152 can include any numerical representation for validating a determination of a ML model (e.g., QCM 140 or NLP 114) corresponding to any node 146 or a node feature 148. Validation parameter 152 can include a vector (e.g., embedding) marking context for a given node 146 or a node feature 148, helping the QCM 140 maintain maintaining the contextual consistency of the outputs for each node 146. Validation parameters 152 can be used to maintain the accuracy and reliability of the machine learning (ML) determinations. Validation parameters 152 can define the criteria and thresholds against which the system's outputs are measured, such as checkpoints to confirm that the generated responses align with predefined context or factual correctness. Validation parameters 152 can include or represent factors such as price range, product availability, and user ratings, helping maintain that the recommendations are relevant, viable and accurate.

[0042] Context parameters 154 can include any numerical representations of the context of the query 108. Context parameter 154 can include a vector (e.g., embedding) of a series or a sequence of nodes 146 of the query 108, representing the context of the query 108 as a whole. Context parameters 154 can be used by the QCM 140 to facilitate maintaining the contextual consistency of the outputs for the nodes 146 within the meaning or context of the query 108. Context parameters 154 can provide additional information and constraints that help retain the original intent and context of the query 108 throughout the processing stages. They ensure that each node’s output remains relevant and coherent within the overall context of the query 108. Context parameters 154 can include or be derive from embeddings (e.g., vector representations) of the entire query 108 or a portion of the query 108 (e.g., a sequence of elements 110), and can correspond to the context of the context of the query, query history, session duration, and specific details about past interactions, allowing the QCM 140 to provide contextually appropriate and personalized responses.

[0043] Response functions 160 can include any combination of hardware and software for generating responses 116 to queries 108. Response function 160 can include computer code, instructions or data for utilizing QCM 140 and NLP models 114 to generate responses 116. Response function 160 can utilize the QCM 140 to generate context data structure 150 along with any validation parameters 152 with numerical representations of individual nodes 146 or node features 148 and context parameters 154 representing the context of the query 108. Response function 160 can generate responses 116 utilizing GNNs 142 and transformers 144 for various nodes 146 and node features 148 maintaining process outputs along the way in accordance with validation parameters 152 and context parameters 154.

[0044] Response function 160 can be implemented as a part of a data processing system 102 comprising, or implemented using, one or more processors 210 coupled with memory 215. The memory 215 can store instructions and computer code implementing various functionalities of the data processing system 102 and its components, such as the response function 160.

[0045] When a data processing system 102 receives a query 108 comprising a plurality of elements 110, the tokenization function 120 can generate a plurality of tokens 122. Each of the plurality of tokens 122 can correspond to an element 110 (e.g., a syllable, a word, a phrase or a sentence) of the plurality of elements 110 of the query 108. In response to thequery 108, the response function 160 can generate or produce, via one or more QCMs 140 having a plurality of GNNs 142 and a plurality of transformers 144, one more context data structures 150. The one or more context data structures 150 can be configured to identify a relationship between a first node 146 of a plurality of nodes 146 of the context data structure 150 corresponding to a first token 122 of the plurality of tokens 122 of the query 108 and a second node 146 of the plurality of nodes 146 corresponding to a second token 122 of the plurality of tokens 122 of the same query 108. The relationship or a relation between the two nodes 146 can be expressed using one or more node features 148 (e.g., vector representations or embeddings) expressing similarity (e.g., via similarity function 118) between the two nodes 146.

[0046] The response function 160 can utilize QCM 140 to validate (e.g., via one or more QCMs 140), the relationship between the first node and the second node based on a validation parameter 152 of the context data structure 150. For example ,the QCM 140 can check or compare the numerical representation (e.g., embedding) of the representation with the numerical representation (e.g., embedding) of the validation parameter 152 to check if the representation and the validation parameter 152 are within a predetermined threshold range. The QCM 140 can verify if the first and second nodes 146 (e.g., the two embeddings) are within a predetermined threshold range of similarity (e.g., via the similarity function 118) with respect to their respective validation parameters 152. For instance, each node 146 can have its own validation parameter 152 corresponding to the context of the element 110 to which the node corresponds, allowing the QCM 140 to maintain the context for each node determination within the contextual range of that node 146 (e.g., via the validation parameter 152 for the given node 146).

[0047] The response function 160 can provide, via an NLP model 114 (e.g., a model of the one or more ML models) and based on the context data structure 150, a response 116 to the query 108. For instance, the response function 160 can utilize the context data structure 150 with its validation parameters 152 (e.g., for each of the nodes 146) and the context parameters 154 (e.g., for the query 108) to validate the outputs from the GNNs 142, transformers 144 to form prompts into NLP models 114 to generate the responses 116. The QCM 140 can utilize the context data structure 150 (e.g., with its validation parameters 152 and context parameters 154) to prompt the ML models (e.g., GNNs 142, transformers 144 orNLP models 114) to make corrections and regenerate their determinations or responses, when such determinations or responses do not align with, fall within the range of, or satisfy any of the validation parameters 152 or context parameters 154. The ML model (e.g., NLP 114) used to provide the response 116 for the query 108 can be, or include, the model trained (e.g., via training datasets 130) using a plurality of descriptions of a plurality of products (e.g., to address e-commerce query responses).

[0048] Once the response function 160 generates the response 116, the response function 160 can send the response 116 to a client device 106. The response 116 can be sent, via a network 104 (e.g., any combination of a wireless local area network, a wired or optical network, a cellular network, or the Internet), to the client device 106, responsive to the query 108. The response function 160 can generate the response 116 and send the response to a transceiver circuitry of the computing device (e.g., 200) providing the data processing system 102. The transceiver circuitry can include signal processing circuitry and antennas or communication interfaces for transmitting the responses 116 to the client devices 106.

[0049] For example, the response function 160 can utilize the QCM 140 (e.g., including the transformers 144 and GNNs 142) or NLP models 114 to validate the relationship between the first node and the second node. The relationship between the first and the second nodes 146 can be validated using or based on a context parameter 154. The context parameter 154 can correspond to information derived from the plurality of elements 110 of the query 108 (e.g., a sequence of elements 110 represented by a sequence of tokens 122). The context parameter 154 can include one or more embeddings (e.g., vector representations of a sequence of elements 110 or tokens 122 of a query 108).

[0050] For example, to validate the relationship between the first node 146 and the second node 146, a transformer 144 can evaluate the sequential context between tokens 122 derived from the query 108. The transformer 144 can utilize attention mechanisms to assess the importance and relevance of each token 122 in relation to other tokens 122 in a series of tokens 122 (e.g., corresponding to a series of elements 110 of a query 108), to validate or verify that the generated relationships are contextually accurate (e.g., within a similarity threshold of each other’s contextual meaning). For example, in an e-commerce query 108 like "What is the best smartphone for gaming under $500?", the transformer 144 can validate the relevance of "smartphone" in connection with "gaming" and "under $500", to validatethat these tokens 122 are correctly understood in context before proceeding to generate a response 116.

[0051] For example, a GNN 142 can validate the relationship between the first and second nodes 146 based on structural dependencies within the data. For example, each node 146 representing an element 110 of the query 108 can be associated with an edge that can signify or indicate the relationships between the nodes. The GNN 142 can validate this relationship (e.g., or any other relationships between the nodes 146) by aggregating information from neighboring nodes 146 (e.g., with respect to a given node being processed) to validate or verify (e.g., ensure) the contextual accuracy of the connections. For example, in a product recommendation scenario, if the first node represents "14-y ear-old" and the second node represents "gaming console", the GNN 142 can validate this relationship between the two nodes 146 by considering the context provided by additional nodes like "birthday gift" and "interest in gaming", verifying that the recommendation aligns with the user's context.

[0052] Validation can also include evaluating the relationships using validation parameters 152. Validation parameters 152 can be derived from domain-specific knowledge or statistical thresholds established during the model's training phase (e.g., based on the training dataset 130 including materials on particular types of products or their use). For example, if the validation parameter 152 is associated with a certain level of confidence for a relationship to be considered valid, the QCM 140 can use one or more probabilistic measures to validate or verify the relationship between "smartphone" and "under $500" (e.g., from the query 108) to check if it meets the desired confidence threshold before it is accepted. For instance, context parameter 154 can include parameters to validate that the relationships adhere to specific contextual constraints, such as relevance to the overall parameters of the request (e.g., that it is within $500) or for “gaming” purposes, as well as using user's profile or historical data, thus maintaining the integrity and accuracy of the response.

[0053] The response function 160 can determine (e.g., via one or more ML models, such as the QCM 140 or NLP model 114) that the relationship between the first node 146 and the second node 146 does not match a context parameter 154 corresponding to information derived from the plurality of elements 110 of the query 108 within a predetermined context parameter threshold. The context parameter threshold can include, for example, a predetermined threshold value or a range of values indicative of, or corresponding to, anagreement with the context of the query 108. The response function 160 can utilize the QCM 140 (e.g., GNN 142 or transformer 144) or the NLP model 114 to produce a modified context data structure 150, responsive to such a determination, to regenerate the relationship, the first node or the second node, and maintain the determinations or outputs of the ML models within the context parameter threshold.

[0054] The response function 160 can utilize the ML models (e.g., QCM 140 or NLP model 114) to determine, responsive to the validation determination using the validation parameters 152, that the relationship between the first node 146 and the second node 146 does not match the validation parameter 152 within a predetermined validation parameter threshold corresponding to at least one of the first token 122 or the first node 146 or the second token 122 or the second node 146. The validation parameter threshold can include, for example, a predetermined threshold value or a range of values indicative of, or corresponding to, an agreement with the context of the node 146, its token 122 or its corresponding element 110. The validation parameter threshold can include, for example, a predetermined threshold value or a range of values indicative of, or corresponding to, an agreement with the context of the node 146, its token 122 or its corresponding element 110 in relation with other nodes neighboring nodes 146, their tokens 122 or corresponding elements 110, the neighboring nodes 146 preceding or following the node being determined or evaluated. The response function 160 can instruct the one or more ML models to produce a modified context data structure 150, responsive to such a determination with the validation parameters 152, to regenerate or produce a modified first and second node 146 or a modified relationship between the two.

[0055] The response function 160 can provide, via a first layer of a QCM 140, a first node feature 148 of the first layer corresponding to the relationship between the first node and the second node. The first layer can include a first GNN 142 and a first subset of the plurality of transformers 144 to provide a first subset of the plurality of nodes 146 corresponding to the plurality of tokens 122. For instance, a layer of the QCM 140 can include a GNN 142 communicatively coupled with a plurality of transformers 144 processing a node feature 148 and neighboring nodes 146. The first node feature 148 can represent any attributes or properties of the first node 146, such as its semantic meaning, context, and relationship with other nodes 146. The node feature 148 can include information such as word embeddings,positional encodings, attention scores, or other relevant features that capture or indicate the node's role within the context of the query 108. The node feature 148 can be improved by the transformers 144 to improve the context and relevance of the information shared between nodes 146, facilitating generation of the accurate and contextually appropriate response 116.

[0056] The response function 160 can provide, via a second layer of the QCM 140 that can have a second GNN 142 and a second subset of the plurality of transformers 144 to provide a second subset of the plurality of nodes 146, a second node feature 148 of the second layer. The second node feature 148 can include or serve the same or similar functions as the first node feature 148 and can correspond to a relationship between a third node 146 corresponding to a third token 122 and a fourth node 146 corresponding to a fourth token 122 (e.g., of the sequence of tokens 122 of the elements 110 of the query 108). The third node 146 can relates to the first node 146, such as by being adjacent to the first node 146 (e.g., preceding or following the first node 146 in the sequence of nodes 146 of the query 108). For instance, at least one of the third node 146 or the fourth node 146 can correspond to an element 110 different from each of the plurality of elements 110 of the query 108.

[0057] The response function 160 can utilize the one or more ML models (e.g., QCM 140 and its GNNS 142 or transformers 144) to assign, for each of the plurality of nodes 146, a weight corresponding to significance of one or more elements 110 of the query 108. The weight can include a numerical representation of levels of relevance of the given node 146 (e.g., and its corresponding elements 110), such as most weighted element 110 being the most important or relevant and least weighted element 110 being the least important or relevant element 110 for the context of the query 108. The response function 160 can utilize the QCM 140 to produce, via the one or more ML models (e.g., QCM 140 or NLP model 114), the context data structure 150 using the respective weight of each of the plurality of nodes 146.

[0058] The decoder 126 can decode the response 116 using the context data structure 150. The data processing system 102 can send the decoded response 116 to the client device 106, via the network 104. For instance, the decoder 126 can utilize the context parameters 154 within the data structure 150, validating or verifying that the response 116 aligns with specific user preferences for gaming laptops when responding to a query like "What are the best laptops for gaming?" The decoder 126 can validate the relevance of the response 116(e.g., using the context parameters 154) and validate the accuracy by considering the validation parameters 152 for the individual portions of the response 116 (e.g., elements 110 of the response 116).

[0059] The response function 160 or the tokenization function 120 can produce, via the one or more ML models (e.g., QCM 140 or NLP model 114), the context data structure 150 that can be arranged as, or to include, a tree structure. The tree structure can have a first layer of the plurality of nodes 146 corresponding to the plurality of tokens 122 along with a weight for each respective token 122 of the plurality of tokens 122 and a second layer of the plurality of nodes 146 corresponding to tokens 122 from a second plurality of tokens 122 different from the first plurality of tokens 122. Each token 122 of the second plurality of tokens 122 can be related to at least one other token 122 of the first plurality of tokens 122 based on a score of a similarity comparison (e.g., by a similarity function 118). For instance, tokens of the second plurality of tokens 122 can be processed or inferred tokens 122 corresponding to the first plurality of tokens 122.

[0060] FIG. 2 illustrates a block diagram of an example computing system 200, also referred to as the computer system 200, which can be used to implement elements of the systems and methods described and illustrated herein. Computing system 200 can include or be used to implement any computation or functionality described herein, using for example, commands, instructions or data described herein. Computing system 200 can be included in and run any device (e.g., a server of various entities or corporations executing a data processing system, a client device 106 or any other devices communicating over a network 104). Computing system 200 can be used for operating, executing or providing a model trainer 112, NLP model 114, QCM 140, tokenization function 120, repositories 132 (e.g., storage devices 225) and implement any functionality of these components described herein.

[0061] Computing system 200 can include at least one bus data bus 205 or other communication component for communicating information. Computing system 200 can include at least one processor 210 or processing circuit coupled to the data bus 205 for executing instructions or processing data or information. Computing system 200 can include one or more processors 210 or processing circuits coupled to the data bus 205 for exchanging or processing data or information along with other computing systems 200. Computing system 200 can include one or more main memories 215, such as a random-access memory(RAM), dynamic RAM (DRAM) or other dynamic storage device, which can be coupled to the data bus 205 for storing information and instructions to be executed by the processor(s) 210. Main memory 215 can be used for storing information (e.g., data, computer code, commands or instructions) during execution of instructions by the processor(s) 210.

[0062] Computing system 200 can include one or more read only memories (ROMs) 220 or other static storage device 225 coupled to the bus 205 for storing static information and instructions for the processor(s) 210. Storage devices 225 can include any storage device, such as a solid-state device, magnetic disk or optical disk, which can be coupled to the data bus 205 to persistently store information and instructions.

[0063] Computing system 200 may be coupled via the data bus 205 to one or more output devices 235, such as speakers or displays (e.g., liquid crystal display or active-matrix display) for displaying or providing information to a user. Input devices 230, such as keyboards, touch screens or voice interfaces, can be coupled to the data bus 205 for communicating information and commands to the processor(s) 210. Input device 230 can include, for example, a touch screen display (e.g., output device 235). Input device 230 can include a cursor control, such as a mouse, a trackball, or cursor direction keys, for communicating direction information and command selections to the processor(s) 210 for controlling cursor movement on a display.

[0064] Computer system 200 can also include one or more interfaces 240 coupled via data buses 205. Interfaces 240 can include any physical or virtual components enabling communication between the computer system 200 and any external networks (e.g., the Internet of the network 104). Interface 245 can include a network interface providing transfer of data between the processor(s) 210, memories 215 and any external networks (e.g., network 104) to communicate with other devices on the network 104 (e.g., client devices 106, third party entities or a server or a cloud providing data processing system 102).

[0065] The processes, systems and methods described herein can be implemented by the computing system 200 in response to the processor 210 executing an arrangement of instructions contained in main memory 215. Such instructions can be read into main memory 215 from another computer-readable medium, such as the storage device 225. Execution of the arrangement of instructions contained in main memory 215 causes the computing system 200 to perform the illustrative processes described herein. One or moreprocessors 210 in a multi-processing arrangement may also be employed to execute the instructions contained in main memory 215. Hard-wired circuitry can be used in place of or in combination with software instructions together with the systems and methods described herein. Systems and methods described herein are not limited to any specific combination of hardware circuitry and software.

[0066] Although an example computing system has been described in FIG. 2, the subject matter including the operations described in this specification can be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.

[0067] FIG. 3 illustrates an example block diagram of a structure 300 of the query context model 140 utilizing graphical transformers 144 with graph neural networks 142. The QCM model 140 structure 300 can include an architecture of layer-wise (e.g., multi-layered) GNN 142 components nested alongside and across the transformers 144 (e.g., transformer blocks of language models). By utilizing the example structure 300 of the QCM 140, a more context accurate prompt for the pretrained field-specific NLP models 114 can be achieved. Encoders 124 can be used to provide outputs (e.g., context data structure 150) that can drive the decoders 126. Each layer determinations (e.g., from transformers 144) can be validated using the context data structure 150 and its validation parameters 152 or context parameters 154.

[0068] Rather than taking a random section of a text from the query 108 as prompt of the NLP model 114 (e.g., without a further analysis of the meaning of this text), the example structure 300 of the QCM 140 can allow DPS 102 to analyze the query 108 text to understand the semantic role and relations between the elements 110 of the query (e.g., words and phrases). For instance, the QCM 140 can generate or determine validation parameters 152 or context parameters 154 for each of the nodes 146 or relationships between the nodes (e.g., node features 148). Using the example structure 300, QCM 140 can identify enhanced or improved prompt for the NLP model 114 and more accurately control the response 116 from the NLP model 114. For instance, the QCM 140 can utilize the context data structure 150 to maintain the nodes 146 of the prompts to the NLP models 114 in accordance with the validation parameters 152 as well as in accordance with the context parameters 154.

[0069] Example structure 300 makes use of graphical transformers, having GNN components nested in layers alongside and across the transformers 144 of language models. Example structure 300 can allow the text encoding and the graph aggregation to be fused or combined into an iterative workflow. This can improve the accuracy of the comprehension of each node 146 from the global perspective (e.g., to maintain in within the context of the query). The solution provides a progressive learning strategy in which a model can be successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.

[0070] Example structure 300 of the QCM 140 uses GNNs nested alongside and across each layer of transformers 144 of a model, such as a pretrained language model. As a result, the underlying semantic of each textual node 146 can be precisely captured and effectively integrated for high-quality textual graph representation. The present architecture can introduce a two-stage progressive training strategy to improve the representation quality of the graphical transformers (e.g., structure 300). The present solution can simplify the model with the unidirectional graph aggregation as this can eliminate some unnecessary computation cost, resulting in improved efficiency and scalability.

[0071] In example structure 300, the GNN 142 components can be nested alongside and across from the transformers 144 organized in layers or rows. As a result, the text encoding and graph aggregation can be fused or combined into an iterative workflow. Each iteration of the workflow can include or utilize nodes 146 that can be linked to each other and cause the nodes 146 to exchange information with other nodes 146 in the lay er- wise GNN 142 component (e.g., in the single layer of GNN 142 and transformers 144). As a result, each node 146 can be augmented by other nodes 146 in its surrounding (e.g., its neighbors) in the structure 300.

[0072] Each transformer 144 component can work on the augmented node features 148, in which increasingly informative representations of the nodes 146 and their relationships (e.g., contextual similarity determined from similarity functions 118) can be generated for the next iteration. As a result, QCM 140 of the example structure 300 can continuously improve the contextual representation of the nodes 146 corresponding to the elements of the incoming queries, thereby generate an output in the form of context data structure 150 which can include or reference node features 148 and which can be used as an impot into the NLPmodel 114. As a result, graphical transformers of the example structure 300 can achieve an improved utilization of the cross-node information on a graph, improving the representation quality of the data, while reducing computational or running costs.

[0073] QCM 140 can be trained using a variety of techniques. For example, the training of QCM 140 (e.g., graphical transformers) can involve the center node 146 being “sufficiently informative” to provide the desired response. In such instances, the training task can be accomplished without leveraging the information from neighboring nodes 146. To ensure sufficiently trained GNNs 142, QCM 140 can be trained progressively. In the first round of training, manipulated data can be used in which nodes 146 can be randomly polluted with incorrect data. This can make it more difficult to make prediction merely relying on the center nodes 146 (e.g., without utilizing neighboring nodes 146). The training of the QCM 140 model can then force the QCM model 140 to leverage the whole input nodes 146 (e.g., the neighborhood of nodes 146, including for example the nodes 146 in one or more layers).

[0074] The second round of training can use the unpolluted data, in which the QCM 140 can be fit into the targeted distribution. QCM 140 can be arranged to include a unidirectional graph to avoid instances in which all the linked nodes 146 are mutually dependent in the representation process, thereby avoiding that a presentation of a new node 146 triggers encoding of all the neighboring nodes 146 from scratch. In the unidirectional graph approach, only the center node 146 can make a reference to its neighbor nodes 146, while the neighbor nodes 146 remain independently encoded. By this means, the existing neighbors’ encoding results can be cached and reused, which significantly saves the computation cost.

[0075] FIG. 4 is an example a process flow of a method 400 for using an ML model to improve context of a query prior to its input into a field-specific trained ML model providing the response. Method 400 can be implemented using the systems and components described, for example, in system 100 and using computing system 200. Method 400 can include acts 405-425. At 405, a data processing system can receive a query from a client device. At 410, the data processing system can generate tokens for the element of the query. At 415, the data processing system can produce a data structure. At 420, the data processing system can validate the relationship between the first node and the second node. At 425, the data processing system can provide a response to query using the data structure. At 430, the data processing system can send the response to the query to the client device.

[0076] At 405, a data processing system can receive a query from a client device. For instance, a data processing system can include, or be implemented using, one or more processors coupled with memory. The one or more processors can receive a query from a remote client device. The query can include a plurality of elements. The elements can include paragraphs, sentences, phrases words or portions of words or sub-words (e.g., “un” in the word “unrecognized”, or “non” in the word “nonissue”). The query can include text (e.g., one or more sentences) inquiring about a particular product, requesting information corresponding to the purchase of a product or a service, or corresponding to a question or inquiry related to any field, such as e-commerce, medicine, technology, law, or any other field or topic.

[0077] The query can request information about a product for sale, such as a question about a product’s performance, availability, terms of delivery, information about warranty, service plans, or technical or physical features of the product. The query can include a description of a product or its features or seeking to purchase a product. The query can indicate a price range for the product, a delivery date or specification, or information about use or purpose of the product (e.g., a gift for a child), or technical specifications of the product.

[0078] The query can be directed to any field or a topic aside from e-commerce, such as a medical question about a medical or a health-related diagnosis. The query can be directed to a legal question, such as a question about a legal or a procedural issue. The query can be directed to a topic about a technical or scientific field, such as a question about a scientific determination. The query can be directed to a question about one or more events, history, political developments, geography, markets, traffic, instructions or any other field or topic.

[0079] At ACT 410, the data processing system can generate tokens for the element of the query. For instance, the method can include the data processing system generating a plurality of tokens. Each of the plurality of tokens can correspond to an element of the plurality of elements of the query (e.g., tokens for each of the words or the phrases of the query). Each token can include a numerical representation of a corresponding one or more elements of the query. Each token can be represented as a numerical value (e.g., integer or a floating value), or as a vector of one or more vector values (e.g., embedding). Each token can correspond to an element of the query, such as a word or a phrase of the query.

[0080] The tokenization function or the response function can utilize one or more ML models (e.g., query context models including any of their graph neural networks or transformers, or any natural language processing models) to generate one or more embeddings (e.g., vector representations) for the one or more elements of the query or the one or more tokens of the elements. The embeddings can include vector representations of the meaning or context of each of the tokens. The tokenization function or the response function can utilize the one or more ML models to generate an embodiment representing the context of the query, or a vector representation of the sequence of tokens corresponding to the sequence of elements of the query (e.g., a context parameter of the context data structure). The tokenization function or the response function can utilize the one or more ML models to generate vector representations (e.g., embeddings) of individual tokens or the corresponding elements, indicating contextual representation or meaning of each of the elements (e.g., along with any weights assigned to individual elements).

[0081] At ACT 415, the data processing system can produce a data structure. The method can include the data processing system that can produce, construct or generate, via a first machine learning (ML) model (e.g., query context model) having a plurality of graph neural networks (GNNs) and a plurality of transformers, a data structure, such as a context data structure. The data structure (e.g., context data structure) can identify a relationship between a first node of a plurality of nodes of the data structure corresponding to a first token of the plurality of tokens (e.g., and a first element of the query) and a second node of the plurality of nodes corresponding to a second token of the plurality of tokens (e.g., and a second element of the query). The relationship can be indicated, for example, via a first node feature having or including a numerical representation indicative of the relationship between the first node and the second node.

[0082] The relationship between the first and the second nodes can be a contextual relationship. The contextual relationship can be defined, quantified, indicated or determined based on a distance in the vector space (e.g., embedding space of a knowledge graph) between a first numerical representation (e.g., first embedding) of the first node and a second numerical representation (e.g., second embedding) of the second node. The relationship can be determined or defined based on one or more contextual parameters for the query or one or more validation parameters for the nodes. The context data structure can include a pluralityof tokens organized in layers such that the graphical transformers of the first ML model update each node based on the values of the neighboring node (e.g., a node that is adjacent to the node being updated).

[0083] The data processing system can provide a first node feature via a first layer of the first ML model (e.g., a layer of transformers of a GNN of a query context model). The first layer can include a first GNN and a first subset of the plurality of transformers providing a first plurality of nodes corresponding to the plurality of tokens. The first node feature can include, or be, a numerical representation indicative of the relationship between the first node and the second node, such as a distance in vector space between two or more nodes or a distance between the nodes and an embedding of the context of the query.

[0084] The data processing system can provide, via a second layer of the first ML model having a second GNN and a second subset of the plurality of transformers to provide a second subset of the plurality of nodes, a second node feature. The second node feature can be of the second layer and can be indicative of a relationship between a third node corresponding to a third token and a fourth node corresponding to a fourth token. The third node of the second layer can relate to the first node of the first layer. For instance, the third node can correspond to an element of the query that is adjacent to the element of the query of the first node in the sequence of elements of the query.

[0085] For example, the at least one of the third node or the fourth node can correspond to an element different from each of the plurality of elements of the query. The data processing system can assign each of the plurality of nodes a weight corresponding to significance of one or more elements of the query. The data processing system can produce, via the first ML model, the data structure using the respective weight of each of the plurality of nodes.

[0086] The data processing system can produce, via the first ML model, the data structure arranged as a tree structure. The tree structure can include a first layer of the plurality of nodes corresponding to the plurality of tokens along with a weight for each respective token of the plurality of tokens and a second layer of the plurality of nodes corresponding to tokens from a second plurality of tokens different from the first plurality of tokens. Each token of the second plurality of tokens can relate to at least one other token of the first plurality of tokens based on a score of a similarity comparison.

[0087] At 420, the data processing system can validate the relationship between the first node and the second node. The method can include the response function of the data processing system validating the relationship between the first node and the second node based on a validation parameter of the data structure. The validation can be done via the one or more ML models (e.g., query context model) to validate, verify or maintain the prompts input into the NLP model within their respective context or contextual range. For example, the response function can utilize the one or more of the GNNs or transformers of the query context model to validate, verify or check if the relationship between the first node and the second node (e.g., corresponding to the node feature) is within a validation parameter of a context data structure. The validation parameter can include a validation parameter threshold corresponding to numerical representation of the context of the relationship (e.g., expressed using embeddings and their relationship in a vector space).

[0088] The data processing system can validate, via the one or more ML models (e.g., query context model), the relationship between the first node and the second node based on a context parameter. The context parameter can relate or correspond to information derived from the plurality of elements of the query. For instance, the context parameter can include a numerical representation (e.g., vector or embedding) for a sequence of elements of the query. For instance, the response function can utilize the query context model to determine whether the relationship (e.g., node feature) between the first and the second nodes is within the range of context parameters.

[0089] For instance, the data processing system can determine, using the one or more ML models (e.g., query context models), that the relationship between the first node and the second node does not match a context parameter. For instance, the query context model can determine that the numerical relationship (e.g., embedding of the node feature) of the first and second nodes does not correspond to, or is different from, the numerical representation (e.g., embedding of the contextual parameter) derived from the plurality of elements of the query. This determination can be made according to a predetermined context parameter threshold. For example, the data processing system can determine that the relationship between the first node and the second node does not match a context parameter information derived from the plurality of elements of the query to within a predetermined range (e.g., similarity determination) of a threshold for the context parameter of the query. Responsive tosuch a determination, the data processing system can instruct the one or more ML models to produce a modified context data structure, which would have a different relationship that satisfies the context parameter threshold.

[0090] The method can include the data processing system determining, responsive to the validation using the validation parameters, that the relationship between the first node and the second node does not match the validation parameter within a predetermined validation parameter threshold. The validation parameter threshold can correspond to at least one of the first token (e.g., embodiment of the first token) or the second token (e.g., embodiment of the second token). The data processing system can instruct the one or more ML models to produce a modified data structure, responsive to the determination that the relationship does not match the validation parameters.

[0091] At ACT 425, the data processing system can provide a response to query using the data structure. The method can include the data processing system providing, by a second ML model trained using a plurality of descriptions of a plurality of products and based on the data structure, a response to the query. The second ML model can include an LLM or an NLP model that is trained using texts or materials from a particular field (e.g., e-commerce, science, technology, medicine, law or any other field)

[0092] Once the data processing system has verified, validated or ensured that the nodes and the relationships between the nodes are within their respective validation parameters (e.g., context of the nodes) or context parameters (e.g., the context of the query) at act 420, the data processing system can provide at least a portion of such context data structure as an input into the NLP model. The NLP model can utilize at least the portion of the context data structure to generate the response to the query.

[0093] The second ML model generating the response can include an NLP model. The NLP model can use as its input a prompt generated using the data structure (e.g., context data structure), rather than the query. For instance, the numerical representations of the nodes and the node features (e.g., relationships between the nodes) can be input as one or more prompts into the NLP model to be used for generating the response.

[0094] The prompt for the input to the second (e.g., NLP) ML model can include a modified query that is based on the received user query modified using the context data structure generated by query context model. For instance, the query context model can utilize thevalidation and context parameters for all the nodes and node features to validate or verify (e.g., make sure) that the nodes and node features satisfy the validation and context parameters, prompting the query context models to regenerate the nodes and node features when they do not satisfy the validation and context parameters. As such, the prompt provided to the NLP models includes nodes and node features that satisfy the validation and context parameters of the context data structure. For example, the second ML model can use the data structure as its input to produce the response to the query. The second ML model (e.g., NLP model) can generate the response based on the prompt input into the second ML model. The prompt can be generated by the query context model (e.g., transformer-based GNN) using the context data structure along with its validation and context parameters to maintain the prompt within the threshold ranges defined or implemented using the validation and context parameters.

[0095] At ACT 430, the data processing system can send the response to the query to the client device. The method can include the data processing system sending the response to the client device responsive to the query. The data processing system can receive the response from the second ML model and send it to the client device, via the network. The data processing system can decode the response using the data structure and then transmit to the client device the decoded response.

[0096] The foregoing examples have been provided merely for the purpose of explanation and are in no way to be construed as limiting of the present disclosure. While aspects of the present disclosure have been described with reference to an exemplary embodiment, it is understood that the words which have been used herein are words of description and illustration, rather than words of limitation. Changes may be made, within the purview of the appended claims, as presently stated and as amended, without departing from the scope and spirit of the present disclosure in its aspects. Although aspects of the present disclosure have been described herein with reference to particular means, materials and embodiments, the present disclosure is not intended to be limited to the particulars disclosed herein; rather, the present disclosure extends to all functionally equivalent structures, methods and uses, such as are within the scope of the appended claims.

[0097] The systems described above can provide multiple ones of any or each of those components and these components can be provided on either a standalone system or onmultiple instantiation in a distributed system. In addition, the systems and methods described above can be provided as one or more computer-readable programs or executable instructions embodied on or in one or more articles of manufacture. The article of manufacture can be cloud storage, a hard disk, a CD-ROM, a flash memory card, a PROM, a RAM, a ROM, or a magnetic tape. In general, the computer-readable programs can be implemented in any programming language, such as LISP, PERL, C, C++, C#, PROLOG, or in any byte code language such as JAVA. The software programs or executable instructions can be stored on or in one or more articles of manufacture as object code.

[0098] The subject matter and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. The subject matter described in this specification can be implemented as one or more computer programs, e.g., one or more circuits of computer program instructions, encoded on one or more computer storage media for execution by, or to control the operation of, data processing apparatuses. Alternatively, or in addition, the program instructions can be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. A computer storage medium can be, or be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. While a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate components or media (e.g., multiple CDs, disks, or other storage devices include cloud storage). The operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0099] The terms “computing device”, “component” or “data processing apparatus” or the like encompass various apparatuses, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones,or combinations of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatus can also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures.

[0100] A computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program can correspond to a file in a file system. A computer program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0101] The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatuses can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). Devices suitable for storing computer program instructions and data can include non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks;and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0102] The subject matter described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described in this specification, or a combination of one or more such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).

[0103] While operations are depicted in the drawings in a particular order, such operations are not required to be performed in the particular order shown or in sequential order, and all illustrated operations are not required to be performed. Actions described herein can be performed in a different order.

[0104] Having now described some illustrative implementations, it is apparent that the foregoing is illustrative and not limiting, having been presented by way of example. In particular, although many of the examples presented herein involve specific combinations of method acts or system elements, those acts and those elements may be combined in other ways to accomplish the same objectives. Acts, elements and features discussed in connection with one implementation are not intended to be excluded from a similar role in other implementations or implementations.

[0105] The phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including” “comprising” “having” “containing” “involving” “characterized by” “characterized in that” and variations thereof herein, is meant to encompass the items listed thereafter, equivalents thereof, and additional items, as well as alternate implementations consisting of the items listed thereafter exclusively. In one implementation, the systems and methods described herein consist of one, each combination of more than one, or all of the described elements, acts, or components.

[0106] Any references to implementations or elements or acts of the systems and methods herein referred to in the singular may also embrace implementations including a plurality of these elements, and any references in plural to any implementation or element or act herein may also embrace implementations including only a single element. References in the singular or plural form are not intended to limit the presently disclosed systems or methods, their components, acts, or elements to single or plural configurations. References to any act or element being based on any information, act or element may include implementations where the act or element is based at least in part on any information, act, or element.

[0107] Any implementation disclosed herein may be combined with any other implementation or embodiment, and references to “an implementation,” “some implementations,” “one implementation” or the like are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with the implementation may be included in at least one implementation or embodiment. Such terms as used herein are not necessarily all referring to the same implementation. Any implementation may be combined with any other implementation, inclusively or exclusively, in any manner consistent with the aspects and implementations disclosed herein.

[0108] References to “or” may be construed as inclusive so that any terms described using “or” may indicate any of a single, more than one, and all of the described terms. References to at least one of a conjunctive list of terms may be construed as an inclusive OR to indicate any of a single, more than one, and all of the described terms. For example, a reference to “at least one of ‘A’ and ‘B’” can include only ‘A’, only ‘B’, as well as both ‘A’ and ‘B’. Such references used in conjunction with “comprising” or other open terminology can include additional items.

[0109] Where technical features in the drawings, detailed description or any claim are followed by reference signs, the reference signs have been included to increase the intelligibility of the drawings, detailed description, and claims. Accordingly, neither the reference signs nor their absence has any limiting effect on the scope of any claim elements.

[0110] Modifications of described elements and acts such as substitutions, changes and omissions can be made in the design, operating conditions and arrangement of the disclosed elements and operations without departing from the scope of the present disclosure.

[0111] References to “approximately,” “substantially” or other terms of degree include variations of + / - 10% from the given measurement, unit, or range unless explicitly indicated otherwise. Coupled elements can be electrically, mechanically, or physically coupled with one another directly or with intervening elements. Scope of the systems and methods described herein is thus indicated by the appended claims, rather than the foregoing description, and changes that come within the meaning and range of equivalency of the claims are embraced therein.

Claims

CLAIMSWhat is claimed is:

1. A system, comprising: a data processing system comprising one or more processors coupled with memory to: receive a query comprising a plurality of elements; generate a plurality of tokens, each of the plurality of tokens corresponding to an element of the plurality of elements of the query; produce, via one or more machine learning (ML) models having a plurality of graph neural networks (GNNs) and a plurality of transformers, a data structure to identify a relationship between a first node of a plurality of nodes of the data structure corresponding to a first token of the plurality of tokens and a second node of the plurality of nodes corresponding to a second token of the plurality of tokens; validate, via the one or more ML models, the relationship between the first node and the second node based on a validation parameter of the data structure; provide, via a model of the one or more ML models and based on the data structure, a response to the query, the model trained using a plurality of descriptions of a plurality of products; and send, to a client device responsive to the query, the response.

2. The system of claim 1, comprising the data processor to: validate, via the one or more ML models, the relationship between the first node and the second node based on a context parameter corresponding to information derived from the plurality of elements of the query.

3. The system of claim 1, comprising the data processor to: determine, using the one or more ML models, that the relationship between the first node and the second node does not match a context parameter corresponding to informationderived from the plurality of elements of the query within a predetermined context parameter threshold; and instruct the one or more ML models to produce a modified data structure, responsive to the determination.

4. The system of claim 1, comprising the data processor to: determine, responsive to the validation, that the relationship between the first node and the second node does not match the validation parameter within a predetermined validation parameter threshold corresponding to at least one of the first token or the second token; and instruct the one or more ML models to produce a modified data structure, responsive to the determination.

5. The system of claim 1, comprising the data processor to: provide, via a first layer of a first ML model of the one or more ML models, a first node feature of the first layer corresponding to the relationship between the first node and the second node, the first layer having a first GNN and a first subset of the plurality of transformers to provide a first subset of the plurality of nodes corresponding to the plurality of tokens.

6. The system of claim 5, comprising the data processor to: provide, via a second layer of the first ML model having a second GNN and a second subset of the plurality of transformers to provide a second subset of the plurality of nodes, a second node feature of the second layer corresponding to a relationship between a third node corresponding to a third token and a fourth node corresponding to a fourth token, wherein the third node relates to the first node.

7. The system of claim 6, wherein at least one of the third node or the fourth node corresponds to an element different from each of the plurality of elements of the query.

8. The system of claim 1, comprising the data processor to: assign, for each of the plurality of nodes, a weight corresponding to significance of one or more elements of the query; and produce, via the one or more ML models, the data structure using the respective weight of each of the plurality of nodes.

9. The system of claim 1, comprising the data processor to: decode, using the data structure, the response; and send, to the client device, the decoded response.

10. The system of claim 1, comprising the data processor to: produce, via the one or more ML models, the data structure arranged to include a tree structure having a first layer of the plurality of nodes corresponding to the plurality of tokens along with a weight for each respective token of the plurality of tokens and a second layer of the plurality of nodes corresponding to tokens from a second plurality of tokens different from the first plurality of tokens, each token of the second plurality of tokens related to at least one other token of the first plurality of tokens based on a score of a similarity comparison.I L A method, comprising: receiving, by a data processing system comprising one or more processors coupled with memory, a query comprising a plurality of elements; generating, by the data processing system, a plurality of tokens, each of the plurality of tokens corresponding to an element of the plurality of elements of the query; producing, by the data processing system via one or more machine learning (ML) models having a plurality of graph neural networks (GNNs) and a plurality of transformers, a data structure to identify a relationship between a first node of aplurality of nodes of the data structure corresponding to a first token of the plurality of tokens and a second node of the plurality of nodes corresponding to a second token of the plurality of tokens; validating, by the data processing system via the one or more ML models, the relationship between the first node and the second node based on a validation parameter of the data structure; providing, by a model of one or more ML models and based on the data structure, a response to the query, the model trained using a plurality of descriptions of a plurality of products; and sending, by the data processing system to a client device responsive to the query, the response.

12. The method of claim 11, comprising: validating, by the data processing system via the one or more ML models, the relationship between the first node and the second node based on a context parameter corresponding to information derived from the plurality of elements of the query.

13. The method of claim 11, comprising: determining, by the data processing system, using the one or more ML models, that the relationship between the first node and the second node does not match a context parameter corresponding to information derived from the plurality of elements of the query within a predetermined context parameter threshold; and instructing, by the data processing system, the one or more ML models to produce a modified data structure, responsive to the determination.

14. The method of claim 11, comprising: determining, by the data processing system responsive to the validation, that the relationship between the first node and the second node does not match the validationparameter within a predetermined validation parameter threshold corresponding to at least one of the first token or the second token; and instructing, by the data processing system, the one or more ML models to produce a modified data structure, responsive to the determination.

15. The method of claim 11, comprising: providing, by the data processing system via a first layer of a first ML model of the one or more ML models, a first node feature of the first layer indicative of the relationship between the first node and the second node, the first layer having a first GNN and a first subset of the plurality of transformers providing a first plurality of nodes corresponding to the plurality of tokens.

16. The method of claim 15, comprising: providing, by the data processing system via a second layer of the first ML model having a second GNN and a second subset of the plurality of transformers to provide a second subset of the plurality of nodes, a second node feature of the second layer indicative of a relationship between a third node corresponding to a third token and a fourth node corresponding to a fourth token, wherein the third node relates to the first node.

17. The method of claim 16, wherein at least one of the third node or the fourth node corresponds to an element different from each of the plurality of elements of the query.

18. The method of claim 11, comprising: assigning, by the data processing system, each of the plurality of nodes a weight corresponding to significance of one or more elements of the query; and producing, by the data processing system via the one or more ML models, the data structure using the respective weight of each of the plurality of nodes.

19. The method of claim 11, comprising:decoding, by the data processing system using the data structure, the response; and sending, by the data processing system to the client device, the decoded response.

20. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: receive a query comprising a plurality of elements; generate a plurality of tokens, each of the plurality of tokens corresponding to an element of the plurality of elements of the query; produce, via one or more machine learning (ML) models having a plurality of graph neural networks (GNNs) and a plurality of transformers, a data structure to identify a relationship between a first node of a plurality of nodes of the data structure corresponding to a first token of the plurality of tokens and a second node of the plurality of nodes corresponding to a second token of the plurality of tokens; validate, via the one or more ML models, the relationship between the first node and the second node based on a validation parameter of the data structure; provide, via a model of the one or more ML models and based on the data structure, a response to the query, the ML model trained using a plurality of descriptions of a plurality of products; and send, to the client device responsive to the query, the response.

Images