Knowledge-driven conversation guidance system and method with relevance determination and irrelevance avoidance

Abstract

The present disclosure pertains to a knowledge-driven conversation guidance system and method that utilizes knowledge graphs, taxonomies, or ontologies to facilitate intelligent and contextually relevant conversations. It improves the adoption and use of existing public and private conversational systems by aligning their use with enterprise compliance and security mandates by guiding the conversation to relevant topics and avoiding content irrelevant to an organization's function. In addition to the extraction of structured knowledge representations, the system includes novel mechanisms for determining conversation relevance and avoiding discussions that stray into undesirable or irrelevant areas. The resulting system enables more informed, dynamic, and personalized interactions by leveraging the wealth of information encapsulated within the knowledge graphs, while also safeguarding against potential off-topic content.

Description

Knowledge-Driven Conversation Guidance System and Method with Relevance Determination and Irrelevance AvoidanceRELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 582,119, filed on September 12, 2023. The entire teachings of the above application are incorporated herein by reference.BACKGROUND

[0002] The effective management and utilization of information are paramount to the success of organizations across various sectors. Enterprises routinely grapple with vast collections of data, spanning structured and unstructured formats, often stored in diverse repositories. Extracting actionable insights from this trove of data has emerged as a crucial challenge, one that necessitates advanced techniques in knowledge management, natural language processing (NLP), Large Language Models (LLMs), and cutting-edge hardware acceleration such as graphics processing units (GPUs) and accelerated processing units (APUs).

[0003] A fundamental concept in knowledge management is the construction and utilization of knowledge graphs, taxonomies, and ontologies. These structures serve as repositories of domain-specific information, facilitating efficient organization, retrieval, and interpretation of data. Knowledge graphs represent entities, concepts, relationships, and attributes, interconnecting them in a coherent network. Taxonomies and ontologies, on the other hand, provide hierarchical frameworks for categorizing and structuring knowledge, allowing for systematic navigation through complex domains.

[0004] Advancements in NLP, LLMs and Generative Pre-trained Transformers (GPTs) have revolutionized how we interact with information and technology. These techniques enable machines to understand, process, and generate human language, opening doors to a wide range of applications, including chatbots, virtual assistants, and automated content analysis. Within this context, LLM and GPT technologies have played pivotal roles. These state-of-the-art LLMs, like GPT-3 and beyond, drive context-aware dialogues, facilitating knowledge extraction and personalized interactions.

[0005] In the realm of Conversational Al, policy-based conversation guidance is crucial in controlling how enterprises interact with powerful and transformative technologies such asLLMs and GPTs. Enterprises seek to ensure that conversations conducted by automated systems adhere to predefined rules and guidelines, aligning with the organization's domain expertise, compliance standards, and user expectations. These policies dynamically steer conversations, ensuring relevance, accuracy, and adherence to desired content boundaries.SUMMARY

[0006] The disclosed techniques build upon the foundations of taxonomy, ontology and knowledge graphs derived from target enterprise content to enable a knowledge-driven conversation guidance system (also referred to herein as guidance system). Innovative techniques for relevance determination and irrelevance avoidance based on knowledge graph analysis are deployed in a variety of form factors that optimize performance, reduce latency and increase security for enterprises deploying Large Language Models (LLMs) and Generative Al technologies (e.g., Generative Pre-trained Transformers (GPTs)) for enhancing efficiency and accelerating business functions within their organizations.

[0007] The guidance system has multiple form factors. It can be deployed as a dedicated hardware, virtual machine or software within an enterprise in such a way that the user, typically an employee within an organization, interacts with the system for conversations with chat engines and not directly. Depending on the latency and performance requirements, an organization may choose either hardware form factor with guidance processing accelerated via resource pinning, utilizing accelerators for guidance policies or may consider using standard virtual machine or software deployments for lower performance requirements. Based on corporate roles and responsibilities, corporate users stay on topic by binding user- groups to guidance policies derived by content.

[0008] Techniques such as semantic alignment and description compression may be employed to maintain context and relevance within conversations. Semantic alignment assesses the alignment of conversation content with the underlying knowledge structure. Description compression optimizes the representation of knowledge to enhance efficiency and reduce redundancy. Furthermore, the acceleration of these techniques using Graphics Processing Units (GPUs) and Accelerated Processing Units (APUs) significantly boosts the speed and scalability of knowledge-driven processes.

[0009] Human-In-The-Loop (HITL) verification remains integral to the validation of knowledge structures, ensuring their accuracy and reliability. In parallel, neural network training, often accelerated by GPUs and APUs, enhances the scalability of knowledge graphgeneration. This enables the system to autonomously expand its knowledge base while minimizing the need for continuous human intervention.

[0010] To cater to diverse user needs, personalization is introduced through user profiles, groups, and access control lists (ACLs). Each user or group may be associated with specific policies, tailoring the conversation experience to individual preferences and expertise. The efficient execution of these personalization strategies is further empowered by GPU and APU acceleration.

[0011] The integration of LLMs, GPTs, GPUs, and APUs into knowledge-driven conversation guidance signifies the convergence of cutting-edge Al models and hardware, enabling organizations to navigate the data-rich landscape with unprecedented efficiency and efficacy.

[0012] In one aspect, the invention may be a knowledge-driven conversation guidance system, comprising a processor and a memory with computer code instructions stored thereon. The memory is operatively coupled to the processor such that, when executed by the processor, the computer code instructions cause the guidance system to load content comprising at least one of structured content and unstructured content, convert the content into content corpora, build one or more embedding vectors based on the content corpora, convey the content corpora to one or both of a generative pre-trained transformer (GPT) and a large language model (LLM), and receive therefrom a preliminary taxonomy and a preliminary ontology. The computer code instructions may further cause the guidance system to verify the preliminary taxonomy to produce a verified taxonomy, verify the preliminary ontology to produce a verified ontology, produce one or more knowledge graphs related to the verified taxonomy, the verified ontology, or both, and store the verified taxonomy, the verified ontology, the one or more knowledge graphs, the one or more embedding vectors, and the content corpora in a database.

[0013] The unstructured content may comprise one or more of PDF, DOC, and IMG type files, and the structured content may comprise one or more of a CSV file, a direct database call, or an API invocation that returns JSON or XML structures. To verify the preliminary taxonomy, verify the preliminary ontology, and produce one or more knowledge graphs may further comprise verification through a human in the loop (HITL) process that employs a human to (i) manually verify the preliminary taxonomy to produce a verified taxonomy, (ii) manually verify the preliminary ontology to produce a verified ontology, and (iii) produce the one or more knowledge graphs from the verified taxonomy and the verified ontology.

[0014] To produce one or more knowledge graphs may further comprise configuration of a neural network using the verified taxonomy, the verified ontology, and the verified knowledge graph to produce a trained neural network, and use of the trained neural network to produce the one or more knowledge graphs from an unverified taxonomy and an unverified ontology. The one or more knowledge graphs may generalize a relationship between the verified taxonomy and the verified ontology. The unstructured content may be converted to content corpora using optical character recognition. The one or more embedding vectors may encapsulate one or more specific associations comprising relationships such as meaning, structure, and context between elements of the content corpora.

[0015] In another aspect, the invention may be a knowledge-driven conversation guidance system, comprising a processor and a memory with computer code instructions stored thereon. The memory may be operatively coupled to the processor such that, when executed by the processor, the computer code instructions cause the guidance system to intercept a communication between a user and an LLM or GPT, and evaluate the communication to determine a state of the communication. If the state of the communication aligns with one or more predefined categories of a taxonomy, an ontology, and a knowledge graph associated with an assigned guidance policy, the computer code instructions may further cause the guidance system to allow the communication to continue. If the state of the communication does not align with the one or more predefined categories of the taxonomy, the ontology, and the knowledge graph associated with an assigned guidance policy, the computer code instructions may cause the guidance system to prevent the communication from continuing.

[0016] The computer code instructions may further cause the guidance system to tie the user to a taxonomy, an ontology, and a knowledge graph in a way that guides the user to a relevant conversation. The user may be tied to a taxonomy, an ontology, and a knowledge graph by a determination of an alignment score to measure relevance of the communication, a comparison of the alignment score to a predetermined relevance threshold, and a determination that the communication is relevant when the alignment score exceeds the predetermined relevance threshold. The predetermined relevance threshold may be dynamically adjustable based on one or both of (i) specifics of the communication and (ii) context associated with the user.

[0017] The computer code instructions may further cause the guidance system to produce the knowledge graph to be under corporate control and to be configured based on proprietarycorporate data. The assigned guidance policy may be stored in a guidance policy storage database. The communication and associated communication evaluation result may be stored in an audit storage memory.

[0018] In another aspect, the invention may be a knowledge-driven method of guiding a conversation, comprising intercepting a communication between a user and an LLM or GPT, and evaluating the communication to determine a state of the communication. If the state of the communication aligns with one or more predefined categories of a taxonomy, an ontology, and a knowledge graph associated with an assigned guidance policy, the communication is allowed to continue. If the state of the communication does not align with the one or more predefined categories of the taxonomy, the ontology, and the knowledge graph associated with an assigned guidance policy, the communication is prevented from continuing.

[0019] In another aspect, the invention may be a method of configuring a guidance policy, comprising receiving, from a first storage memory, one or more of (i) a verified taxonomy, (ii) a verified ontology, (iii) one or more knowledge graphs, (iv) one or more embedding vectors, and (v) content corpora, designating a set of relevant topics based on the verified taxonomy, the verified ontology, the one or more knowledge graphs, the one or more embedding vectors, and the content corpora. For each relevant topic of the set of relevant topics, determining a relevance threshold, the method may further comprise limiting one or more users to communications within the set of relevant topics, receiving, from an administrator, a designation of at least one LLM or at least one GPT to be used by a user group and limiting members of the user group to use of only the designated LLM or designated GPT, and storing the guidance policies in a second storage memory, the guidance policies being the designated set of relevant topics, the relevance thresholds, the designated at least one LLM, and the designated at least one GPT.

[0020] The relevance thresholds may employ natural language processing techniques to map entities mentioned in the communications to nodes in the knowledge graph. The knowledge graph may be a custom knowledge graph that is under corporate control and is built on proprietary corporate data. The method may further comprise determining a semantic alignment score to measure relevance of a LLM or GPT communication, comparing the semantic alignment score to a predetermined relevance threshold, and determining that the communication is relevant when the alignment score exceeds the predetermined relevancethreshold. The predetermined relevance threshold may be dynamically adjustable based on one or both of (i) specifics of the communication and (ii) context associated with the user.BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The foregoing will be apparent from the following more particular description of example embodiments, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating embodiments.

[0022] FIG. 1 A shows a block diagram of example guidance system deployed within an enterprise mediating between a user and a conversation engine (public or private LLM).

[0023] FIG. IB shows a block diagram of example guidance system deployed in the cloud as and used by an enterprise mediating between a user an a conversation engine (public or private LLM).

[0024] FIG. 1C shows a block diagram of a node that may be used to implement aspects of the guidance techniques described herein.

[0025] FIG. 2A shows a flow chart of a sequence of design time steps used to develop a knowledge graph based on source content with Human-in-the-Loop validation

[0026] FIG. 2B shows a block diagram for automating the generation of knowledge graphs based on a neural network.

[0027] FIG. 3 shows a block diagram that illustrates binding between ontologies and taxonomies consumed at design time to build guidance policies.

[0028] FIG. 4 shows a block diagram of internal components of an example guidance system according to the embodiments.

[0029] FIG. 5 shows a flow chart of a sequence of steps that maybe used by the guidance system according to the embodiments.

[0030] FIG. 6A illustrates an example display of a guidance policy bound to a set of users and groups at design time.

[0031] FIG. 6B illustrates an example display of a guidance policy configured based on a knowledge graph at design time including taxonomy, ontology and related properties.

[0032] FIG. 6C illustrates an example display of a guidance policy configured based on a knowledge graph at design time including taxonomy, ontology and related properties that are then referenced by the sample permissible conversation prompts.

[0033] FIG. 7A illustrates an example display of a runtime conversation interface response with a relevant question.

[0034] FIG. 7B illustrates an example display of a runtime conversation interface response with an irrelevant question.

[0035] FIG. 8 shows an example enterprise deployment of guidance system in healthcare with policy development and enforcement based on knowledge graphs.DETAILED DESCRIPTION

[0036] A description of example embodiments follows.

[0037] Reference will now be made to the drawings in which aspects of the techniques are given numerical designations and in which the techniques will be discussed to enable one skilled in the art to make and use the techniques. It is to be understood that the following description is only an example of the principles of the techniques and should not be viewed as narrowing the claims that follow.

[0038] Aspects of the techniques described herein are deployed in a network for controlling interaction with conversation systems, internal or external.

[0039] FIG. 1 A is a block diagram that shows an example guidance system 102 that is deployed within a corporation 100 and intercepts conversations, requests and responses from users or other systems 101 which are not permitted to call the target systems 103 directly without going through the guidance systems 102. The target systems 103 are built on machine learning technologies, such as Large Language Models (LLM) or Generative Pretrained Transformers (GPT), examples of which are depicted in the target systems 103. It should be noted that the target systems 103 are examples and can be extended to any external similar systems or internally deployed LLMs and GPTs. Beyond LLMs and GPTs, any Generative Al (GenAI) systems that include additional techniques such as Reinforcement Learning Human Feedback (RLHF) may also have a guidance system 102 intercepting communication for conversation guidance.

[0040] FIG IB is a block diagram of the example guidance system 102 deployed in the cloud 110 outside the corporate boundary or datacenter 100. Corporate users and systems 101 within the corporate boundary 100 are not permitted to interact with the LLMs and GPTs directly. All such conversation is directed to the guidance systems - in this case - deployed in the Cloud 110.

[0041] FIG 1C is a high-level block diagram of an example console node 102. Console node 102 is illustratively a computer system comprising a memory 120 coupled to a processor 130 which is coupled via an input / output (I / O) bus 150 to a network interface 170 and one or more I / O interfaces 160. An example of a computer system that maybe used from Lambda Inc, 2510 Zanker Road, San Jose, California 95131. Processor 130 may be a conventional processor such as Intel Xeon 6448 Y or AMD EPYC 9124 that can execute instructions and manipulate data contained in memory 120. Network interface 170 contains logic configured to interface multiple external accelerators Graphical Processor Units (GPUs) 140 such as multiple NVIDIA RTX A6000 connected over, for example, Nvidia’s NVLink to each other and via PCIe I / O interfaces 160. This enables a variety of scalable clustered configurations to meet performance needs. A more integrated accelerator 140 configuration ties with memory 120 and processor 130, called an Accelerated Processor Unit (APU), for example the Ryzen 3 3200G APU from AMD, provides an additional configuration for higher efficiency for the guidance system 102.

[0042] Memory 120 may be implemented using, e.g., dynamic RAM (DRAM) devices. Memory 120 comprises an operating system 121, guidance engine 122, one or more private Large Language Models (LLMs) or GPTs 123, and Knowledge Graphs (KG) 124.

[0043] The operating system 121 is a conventional operating system that contains Software that enables Software, such guidance engine 122, the LLMs 123 and the Knowledge Graphs (KG) software, to, inter alia, be scheduled for execution and to access various devices on node 102, such as network interface 170. An operating system that may be used with the present invention is the Ubuntu 22.x, an open source operating system.

[0044] The guidance engine 122, one or more private Large Language Models (LLMs) or GPTs 123, and Knowledge Graphs (KG) 124 comprise computer executable instructions configured to implement aspects of the techniques described herein utilizing the example resources illustrated in Figure 1C with special consideration given to the use of accelerators 140 used to make relevance or irrelevance decisions for conversations.

[0045] FIG 2A is a flow chart of a sequence of design time steps used to develop a verified knowledge graph 207 based on the on source content 201 ingested during designtime. Once the content is ingested, the associated corpora is extracted 202 through standard well known techniques such as parsing for structured content and OCR for unstructured content with Human-in-the-Loop validation.

[0046] The extracted corpora may also be obfuscated with confidential and private information removed or replaced before sending it to an internal or external LLM or GPT 203. This obfuscation process is not shown here. The details of this process are captured in a provisional patent application filed by Yunus, et al.

[0047] Based on the content, the LLM / GPT 203 generates an ontology 204 and a taxonomy 205 that are manually verified by a Human-In-The-Loop (HITL) 206. The verified ontology and taxonomy are then used to generate a verified knowledge graph 207 via HITL 206. The ontologies 204, taxonomies 205 and related knowledge graphs now serve as human verified “golden records” that can then be used to automate the process as will be described herein with respect to FIG 2B.

[0048] The ontology, taxonomy, and verified relationships are consolidated into a JavaScript Object Notation Lines (JSONL) file format. This JSONL representation captures the domain's structured knowledge, ready for building guidance policies as shown in FIG 6A and FIG 6B into the conversation guidance system 102.

[0049] The verified ontology, taxonomy and knowledge graph generation is followed by a compression and generalization process 208. This process is enhanced by incorporating similarity techniques to compress descriptions. This step ensures a concise and efficient representation of entities and concepts within the knowledge graph. Descriptions are generalized and compressed based on a predetermined threshold of similarity, resulting in a more compact ontology and taxonomy.

[0050] Description Compression Policy Formulation: To implement the Description Compression Policy, the system utilizes semantic similarity algorithms to identify redundancies and similarities within entity descriptions. Descriptions that surpass a similarity threshold are compressed, replacing specific details with generalized terms. Sample code executed on node 102 for achieving this generalization and compression is shown below:1 from sklearn.feature_extraction.text import TfidfVectorizer2 from sklearn. metrics. pairwise import cosine_similarity34 def compress_description(entity_descriptions, similarity_threshold=0.8):5 vectorizer = TfidfVectorizer()6 tfidf_matrix = vectorizer.fit_transform(entity_descriptions)7 similarity_matrix = cosine_similarity(tfidf_matrix)89 compressed descriptions = []1011 for i, description in enumerate(entity_descriptions):12 similarjndices = [index for index, score in enumerate(similarity_matrix[i]) if score >= similarity_threshold]13 if len(similarjndices) > 1 : # Compress descriptions if multiple similar descriptions found14 generalized_description = generalize_description(description, similarjndices)15 compressed_descriptions.append(generalized_description)16 else:17 compressed_descriptions.append(description)1819 return compressed_descriptions2021 entity_descriptions = [22 "The American Bald Eagle is a large bird of prey.",23 "The Bald Eagle is native to North America.",24 "The Bald Eagle has a distinctive white head and tail feathers."25 ]2627 compressed_descriptions = compress_description(entity_descriptions, similarity_threshold=0.7)28 print(compressed_descriptions)

[0051] As shown in line 12 above, the knowledge graph description is evaluated for similarity and then compared to a similarity threshold that can be set by the user at design time. The similarity threshold may serve as a tuning parameter for deciding the level of compression desired. A low similarity threshold can result in a higher compression. Standard generalization techniques (not described here) can be implemented for line 14. The objective of this compression is to eliminate the need for a single knowledge graph per document ingested. For example, based on the similarity threshold setting of 0.7 as shown in line 27, all entities in line 21 may be compressed into a single description.

[0052] The compressed descriptions are integrated into the ontology, taxonomy and knowledge graph construction process. During the establishment of hierarchical relationships,the compressed descriptions are used to ensure that similar concepts and entities are grouped together, resulting in a more streamlined and concise representation.

[0053] The policies for conversation relevance may subsequently be configured using the compressed ontology, taxonomy and knowledge graphs, as shown in FIG 6B. The system uses the generalized and compact descriptions to enhance the efficiency of semantic alignment and relevance determination during conversations. If the conversation aligns with these parameters, it is considered relevant.

[0054] FIG. 2B is a block diagram for a neural network trained to generate knowledge graphs for additional source documents. The training data includes human-verified knowledge graphs and corresponding text corpora. This step is necessary to scale beyond the HITL verified content and rely on Neural Networks - such as Graph Neural Networks - to create a more comprehensive Knowledge Graph without incurring high manual labor.

[0055] This process automates the generation of knowledge graphs by using the generated and verified ontologies 204 and taxonomies 204 as input features to train a neural network 211 with the verified knowledge graph 207 as the target. The “golden” representations that are human verified as described in FIG. 2A are used to train the neural network since they represent the ground truth. Once the neural network is trained and tested satisfactorily (e.g., standard method of minimizing loss function without overtraining), new generated and unverified ontologies 214 and taxonomies 215 are sent to the trained neural network for inference. The resulting generated knowledge graphs 217 can then be selectively sampled to generate verified knowledge graphs with Human-In-The-Loop (HITL) 206. As with the manual process shown in FIG. 2A, once the knowledge graph is generated, a subsequent compression process 218 is invoked for a more optimal knowledge graph representation.

[0056] The objective of this process is to validate a small subset of the knowledge graph to ensure that the neural network is doing its job correctly. As ongoing HITL takes place, the verified knowledge graphs and their associated ontologies and taxonomies now become part of the golden representation or ground truth that can then be used to retrain the neural network and check for model drift between different versions. This ensures that all labor cost incurred by HITL is captured in the neural network for improved future results, and a reduced HITL burden as new content is ingested to create knowledge graphs for the guidance system. All training and inference for the neural network 211 utilize node 102 illustrated as an example system in FIG 1C.

[0057] FIG 3 is a block diagram that illustrates binding between ontologies, taxonomies and their relationships that are consumed at design time to build guidance policies as shown in FIG 6B. In FIG 3, the taxonomy 300 is derived from source content such as a healthcare benefits document used as an illustrative example. The content can be generalized to any domain and is not restricted to healthcare.

[0058] Using the process and techniques described in FIG 2A, the ingested document results in a taxonomy 300 with 4 elements, Prosthetic Devices 301, Physical Assistance Devices 302, Durable Medical Equipment 304 and Prescription Drugs 303. The taxonomy is hierarchical and generally static, since it builds a singular “subservice of’ relationship between the elements listed. These elements are business objects and are grouped to meet business needs. The groupings are then subdivided for example Physical Assistance Devices 302 are subdivided into Prosthetic Devices 301 and Durable Medical Equipment 304 for illustrative purposes. The subdivision can continue till it captures the required elements presented in a set of ingested documents. A taxonomy 300 also serves the purpose of categorizing documents associated with elements in the taxonomy.

[0059] An ontology 310 related to healthcare benefits provides a more contextual representation of the source documents and has a wider array of relationships options between the elements. Ontologies allow us to organize the jargon of a subject area, healthcare benefits in this case, into a controlled vocabulary to retain the context and understanding of the document. A Healthcare Plan Modification Document 311 “has prior coverage” of Prosthetic Devices 312 and Mental Health Services 313. The property “has prior coverage” captures the intent of the content in the ontology that can then be inferred by the guidance system 102. Along with the “has prior coverage” property, the “modified coverage” property for Prosthetic Devices 314 and Mental Health Services 315 provides the necessary understanding about the intent and understanding of the document 311.

[0060] With the taxonomy 300 and the ontology 310 in place, using techniques described with respect to FIG 2A-B, a knowledge graph is then constructed to bind the ontology and taxonomy as depicted in FIG 3 through the “has coverage of’ property.

[0061] This provides a unique capability for recommending new services based on the elements once the relationship is established between an entity in the ontology, or guiding the conversation to topics that are still relevant to the conversation, however, may not be directly related to the sources document 311. For example, Prosthetic Devices 312 in the ontologyand Prosthetic Devices 301 in the taxonomy are related with the property “has coverage of’ capturing the intent.

[0062] Since Prosthetic Devices 301 is a sub service of Physical Assistance Devices 302 that in turn has Durable Medical Equipment 304 as a subservice, the guidance system 102 can continue to intelligently consider these entities to be relevant conversation topics and not stop the user of system from communicating with a public or private LLM / GPT 103. One of the significant aspects of FIG 2A-B is to build and generalize the relationship between the taxonomy and the ontology as a knowledge graph with properties such as “has coverage of’ shown in dashed lines in FIG 3.

[0063] FIG 4 is a block diagram of internal components of an example guidance system that may employ the techniques described herein. The Loader 401 is responsible for loading structured and unstructured content including PDF, DOC and IMG types. Structured content can be loaded as CSV files, direct Database calls or API invocation that return JSON / XML structures in real time or through batch. The loader can integrate with existing corporate systems to retrieve and load the data as necessary.

[0064] Once the content is loaded, the Generation Engine 402 is responsible for converting the content into a corpus through the Corpora Converter. Unstructured content such as images, faxes, and scanned documents go through a standard Optical Character Recognition (OCR) for conversion. Once the corpora are generated, the Embedding Generator builds an Embedding vector. Word embeddings fundamentally revolve around the idea that each word in a language can be denoted by a unique set of real-number values, essentially a vector. These vectors serve as learned representations, positioning words within an n-dimensional space. In this space, words sharing similar meanings exhibit closely situated vector representations, resulting in their proximity. The primary objective when creating a word embedding space is to encapsulate specific associations within that space, whether they pertain to meaning, structure, context, or some other form of relationship. Word2Vec, GloVe, FastText, ELMo, BERT, Universal Sentence Encoder (USE), Transformer-based models (e g., GPT-2, GPT-3, T5), Doc2Vec, Roberta, XLNet, DistilBERT, ALBERT, Sentence- BERT, and OpenAI, are a few examples of embedding engines that generate vectors with dimensions ranging from 50 to over 12,228.

[0065] Within the Generation Engine 402, the Taxonomy and Ontology generator is used to take the content corpora, send it to an external or internal LLM / GPT to have it generate the preliminary taxonomies and ontologies in a designated format such as JSONL. Thiscomponent then utilizes the HITL process described in FIG 2A and its automation in FIG 2B to verify the taxonomies, ontologies and their related knowledge graphs and then utilize neural networks to automate this process. The generated ontologies, taxonomies, knowledge graphs along with the embeddings and original corpora and content are all stored in a database 403. This database can be a single database or a combination of databases. For example, a vector database such as Pineone, Weaviate, Chroma or Milvus can be used along with databases designed especially for knowledge graphs such as Neo4J, Amazon Neptune or GraphDB. A single database such as PostgreSQL may also be used as a store.

[0066] The Guidance Policy Configurator 404 is a design-time component. It serves as the heart of the knowledge-driven conversation guidance system in the development of conversation policies based on the insights derived from the extracted knowledge graph and the constructed taxonomy and ontologies. These policies are sets of rules, guidelines, or recommendations that steer the direction of conversations. They ensure that the interactions remain contextually relevant, accurate, and aligned with the available knowledge.

[0067] The Guidance Policy Configurator 404 ties user groups with the ontologies, taxonomies and knowledge graphs in a way that guides the users to relevant conversations. For a group of users once the relevant topics are configured, the Relevance Thresholds subcomponent of 404 employs NLP techniques to map mentioned entities to nodes in the knowledge graph. Semantic alignment scores are then calculated to measure relevance of the conversation and compare it to the thresholds set at design time. Conversations with alignment scores exceeding a predefined threshold are considered relevant.

[0068] The Guidance Policy Configurator 404 also enables the administrator to select public and private LLMs / GPT for a user group. The selection of the LLMs ensures that the users are only interacting with approved external or internal LLMs. For the relevant or irrelevant decision making process, the guidance system is designed to leverage a custom knowledge graph that does not leave the corporate control and is built on proprietary corporate data. The guidance system 102 therefore guides the conversation with external and internal LLMs / GPTs based on the knowledge inherent to the corporate information and boundaries. The policies configured in 404 are subsequently stored in the Guidance Policy Store 405.

[0069] The Enforcement Engine 406 loads policies from the Guidance Policy Store 405 for policy enforcement. Its request and response interceptor subcomponent intercept the conversation between the user or system making a request to LLMs / GPTs 103. The request isthen sent to the Guidance Policy Evaluator subcomponent to evaluate whether the dialogue aligns with the predefined categories of the taxonomy, ontology and the knowledge graph. If the alignment is within the prescribed relevance thresholds, as determined by the Relevance Threshold Decider, the requests and the responses are allowed to continue otherwise they are flagged as irrelevant, and the conversation is discontinued. The threshold for relevance determination can be dynamically adjusted based on the specifics of the conversation or the user context. For instance, a novice user might have a lower threshold than an expert, ensuring that the system remains adaptable to different user levels. The thresholds may also be associated with user authorization levels. The user or system 101 can provide feedback whether the dialogue was allowed or terminated incorrectly. This feedback captured by the RLHF subcomponent of 406 is then stored in the Audit Store 407 to be used for further training and policy refinement in conjunction with the knowledge graphs.

[0070] All requests and responses along with any transformed states such as obfuscated requests and responses are stored in the Audit Store 407. Compliance and security teams may then use the compliance report generator to audit reports to ensure compliance to corporate policies around privacy and ethics. The reports can also be used to further adjust the guidance engine policies.

[0071] FIG 5 is a flow chart of a sequence of steps that may be used by the guidance system with users or systems authenticating at step 501 with the guidance systems 102. Once the user or system identity is established, the user group is determined in step 502 based on the guidance policies in the guidance policy store 405 which also binds the groups to permissible conversation policies that are configured at design time using the Guidance Policy Configurator 405. The conversation request 504 is then sent for evaluation in the Enforcement Engine 406 and evaluated in step 505 against the policies set using the knowledge graph. If the relevance threshold is met in 506, the request is allowed to proceed to selected public or private LLMs to continue the conversation or blocked as irrelevant as shown in step 507. Although not shown in Fig 5, a similar sequence of steps can be deployed for responses sent back from the LLMs. As an illustrative example, an LLM that responds with inappropriate content should be blocked as irrelevant content by the guidance system 102 without the user or the system 101 ever seeing the irrelevant content.

[0072] The step 506 starts with a semantic alignment and relevance determination process such that during conversations, the system employs NLP techniques to map mentioned entities to nodes in the knowledge graph. Semantic alignment scores arecalculated to measure the relevance of the conversation. Sample code to implement such as relevance comparison between the content and the knowledge graphs entities are shown below for illustrative purposes:I import spacy23 def calculate_semantic_alignment(conversation_text, knowledge graph entities):4 nip = spacy. load("en_core_web_sm")56 conversation tokens = nlp(conversation text)7 knowledge tokens = [nlp(entity) for entity in knowledge graph entities]89 alignment score = 010 for conv token in conversation tokens:I I for kg token in knowledge tokens:12 alignment score += conv token.similarity(kg token)1314 return alignment score / (len(conversation tokens) * len(knowledge tokens))

[0073] As shown in line 14 an alignment score is calculated to determine if the conversation text aligns with knowledge graphs which may include taxonomy, ontology and properties as described in FIG 3. This technique checks for similarity between the knowledge graph tokens and the conversation as shown in line 12 and aggregates the alignment score across all conversation and knowledge graph tokens before normalizing it and returning it line 14.

[0074] Within step 506, the relevance is then determined by the semantic alignment score obtained from the alignment calculation in line 14 above and comparing it to the relevance threshold shown in line 3 below. Conversations with alignment scores exceeding a predefined threshold are considered relevant. Sample code for illustrative purposes to implement such as relevance score and threshold comparison is shown below:1 def is_conversation_relevant(conversation_text, knowledge graph entities, rel evance threshol d=0.6) :2 alignment score = calculate_semantic_alignment(conversation_text, knowl edge graph entiti es)3 return alignment score >= relevance threshold

[0075] The threshold for relevance determination can be dynamically adjusted based on the specifics of the conversation or the user context. Relevance thresholds may not be static for all members of an enterprise with varying responsibilities and functions. For instance, a novice user might have a lower threshold than an expert, ensuring that the system remains adaptable to different user levels.1 def dynamic_relevance_threshold(user_experience_level):2 if user_experience_level == "novice":3 return 0.54 elif user_experience_level == "expert":5 return 0.76 else:7 return 0.6 # Default threshold89 user experience = get_user_experience_level()10 threshold = dynamic_relevance_threshold(user_experience)11 is relevant = is_conversation_relevant(conversation_text, knowledge graph entities, threshold)

[0076] Although a couple of user experience levels are shown in the code above in lines 2 and 4, the concept can be extended to an enterprise identity authentication and authorization system like an LDAP store that can be integrated with Guidance Policy Configurator 404 for multiple-level employee and member roles within a corporation.

[0077] FIG 6A illustrates an example display 600 of a guidance policy bound to a set of users and groups at design time. A guidance policy shown in FIG 6B and FIG 6C stored in 405 is selected and associated with a group of users 602. This binding at design time is also stored in 405 along with the guidance policy. All available and unassociated user groups appear in 601

[0078] FIG 6B illustrates an example display of a guidance policy configured based on a knowledge graph at design time including taxonomy, ontology and related properties. The Healthcare guide 610 consists of details related to the Taxonomy 611 and the Ontology 612 and the relation between them as displayed in line 18 with “serviceName” pointing to a Taxonomy property “serviceName” with value “Premium Mental Health Services” on line 5 that is a service of the root taxonomy entity “Premium Services” on line 4 in this illustrative example.

[0079] As explained in FIG 4A and 4B the taxonomy, ontology and related knowledge graph is constructed based on the source ingested content to enable the guidance systems 102 to stay on topics closely related to the content.

[0080] FIG 6C illustrates an example display of a guidance policy 620 configured based on a knowledge graph at design time including taxonomy, ontology and related properties that are then referenced by the sample permissible conversation prompts. In 621, the guidance policy is configured to check against the taxonomy whereas in 622 the guidance policy checks against the ontology. Combined, together along with the prior artifacts defined in 611 and 612 along with their knowledge graph relationships, the guidance policy shown in 6A-C provides a technique to keep the conversation, both requests and response on topic and within the corporate compliance policies.

[0081] The guidance policy shown in 6A-C can be generated in a fully automated manner once a few examples are human verified via a neural network as shown in FIG 4B with the last nodes 207 and 208 including instructions such as 621 and 622 that are generated as a part of the knowledge graph and can therefore be used directly by the guidance policy directly obtained from the source ingested content 201.

[0082] FIG 7A illustrates an example display of a runtime conversation interface 700 with a relevant request 701 and a relevant response 702 that is enforced by the guidance enforcement engine 406. Since the question request is related the healthcare, as an example, the guidance engine evaluates this using the steps described in FIG 5 and determines that its within the thresholds set for alignment with the knowledge graph.

[0083] FIG 7B illustrates an example display of a runtime conversation interface response with an irrelevant question. The user veers off topic into personal interests unrelated to corporate content as shown in 710. The guidance engine 406 based on the guidance policies determines this to be an irrelevant question and responds with an “Out of Context” response 712.

[0084] Although not described here, it should be evident that the same process and systems is used for intercepting and inspecting responses from public or private LLMs / GPTs and determining their relevance using the Guidance Enforcement Engie 406.

[0085] The knowledge-driven conversation guidance system is designed to facilitate meaningful and relevant conversations. It comprises a Knowledge Graph Extraction component, which processes a collection of assets to extract a structured knowledge graph representing entities, concepts, relationships, and attributes within a specific domain. This knowledge graph serves as the foundation for guiding conversations. Additionally, there is a Conversation Guidance Policies component that defines and applies a set of rules. These rules help determine conversation relevance and prevent discussions from veering into undesirable or irrelevant areas, ensuring that the conversation stays on track.

[0086] Within the Conversation Guidance Policies component, there is a module responsible for defining and applying rules associated with user profiles, user groups, or access control lists (ACLs). Each user profile, user group, or ACL is assigned specific policies, allowing for the customization of conversation topics based on individual preferences or group requirements.

[0087] Furthermore, the system can employ dedicated accelerators to evaluate conversations against the Knowledge Graph efficiently, enhancing its responsiveness and accuracy. This specialized approach ensures that the guidance system can quickly assess the relevance of ongoing discussions.

[0088] To generate these guidance policies, the system utilizes a Guidance Policy Generation component, employing a neural network to bind taxonomies to ontologies. This process creates a knowledge graph that serves as the basis for defining the conversation rules. Additionally, the system can employ techniques such as knowledge graph compression to optimize these policies, making them more effective in guiding conversations.

[0089] In practice, a method for guiding conversations using this system involves several steps. First, it extracts a structured knowledge graph from a collection of assets, representing entities, concepts, relationships, and attributes within a specific domain. Then, it develops a taxonomy or ontology based on this knowledge graph. Subsequently, a set of conversation policies is formulated, defining rules for assessing conversation relevance and preventing discussions from straying into undesirable or irrelevant areas. These policies are then applied dynamically to guide conversations in real-time, ensuring they align with the insights derived from the knowledge graph and the constructed taxonomy.

[0090] Within the Conversation Guidance Policies component, various specific policies are applied, including Semantic Alignment Policy, Guidance Policy, and Relevance Threshold Policy. These policies collectively assess the alignment of conversation content with the knowledge graph, evaluate adherence to predefined categories, and trigger corrective measures when necessary, based on a predetermined relevance threshold.

[0091] Moreover, the application of conversation policies involves analyzing conversation content for semantic alignment, consistency with predefined categories, alignment with historical conversation progression, and adherence to sensitivity guidelines. The system can also adapt these policies over time by learning from user interactions and feedback, ensuring continuous improvement in guiding conversations.

[0092] Additionally, the Guidance Policy Audit component captures historical conversation data and user feedback regarding the relevance decisions made by the guidance system. This feedback loop allows for the modification of guidance policies based on user input and experience.

[0093] Finally, the Guidance Policy Engine component employs Reinforcement Learning with Human Feedback (RLHF) techniques to automate the process of defining and adjusting guidance policies, further enhancing the system's ability to facilitate meaningful and relevant conversations.

[0094] FIG. 8 shows an example enterprise deployment of guidance system in healthcare with policy development and enforcement based on knowledge graphs.Knowledge Graph Extraction:

[0095] In the first phase of the technique, a collection of assets such as documents, articles, databases, and other relevant sources is processed to extract a knowledge graph. This knowledge graph represents a structured and interconnected network of entities, concepts, relationships, and attributes within the domain of interest. Advanced natural language processing (NLP) and semantic analysis techniques are employed to identify and link these elements, resulting in a comprehensive and dynamic knowledge representation.Taxonomy or Ontology Development:

[0096] Upon the successful extraction of the knowledge graph, the technique involves the creation of a taxonomy or ontology that organizes the entities and concepts within a hierarchical framework. This taxonomy serves as a structured guide to categorize andnavigate the knowledge contained in the graph, enabling more efficient access to information during conversations.

[0097] Conversation Relevance Determination and Irrelevance Avoidance Techniques: Semantic Contextualization: The system leverages the interconnected nature of the knowledge graph to contextually analyze conversations. Entities, concepts, and relationships within the dialogue are mapped onto corresponding nodes and connections in the KG. This semantic contextualization enables the system to assess the alignment between the conversation's content and the knowledge encapsulated in the graph.

[0098] Named Entity Recognition INERT Advanced NLP techniques are employed to identify named entities within the conversation. These entities are compared against the entities present in the knowledge graph. A high presence of KG entities suggests topic relevance, aiding in the determination of whether the discussion remains on track.

[0099] Taxonomy -Based Analysis: The taxonomy developed from the KG's structure serves as a reference framework for categorizing conversation topics. The system evaluates whether the dialogue aligns with the predefined categories of the taxonomy. If the conversation strays outside these categories, it might be indicative of a deviation into irrelevant or undesirable territory.

[0100] Contextual Coherence: Drawing from the semantic relationships present in the KG, the system assesses how the conversation's content coheres with these relationships. Consistency with these relationships signifies that the conversation maintains its alignment with the intended topic.

[0101] Sentiment and Intent Analysis: The KG may include sentiment and intent information associated with entities. By analyzing sentiment and intent in context, the system can gauge if the conversation matches the intended topic and sentiment.

[0102] Threshold-based Triggers: The system can be configured with threshold parameters to detect conversations deviating from the desired topic. For instance, if a certain percentage of the conversation's content doesn't correspond to any KG entities or relationships, it may trigger a redirection to realign the conversation.

[0103] Historical Context: Considering the conversation's history, the system assesses whether the current dialogue aligns with the progression of topics. Drastic deviations from previous discussions may prompt further scrutiny.

[0104] Learning and Adaptation: Over time, the system learns from user interactions, incorporating feedback to refine its understanding of relevance and irrelevance. This learning enhances its ability to accurately identify topics and avoid unwanted content.

[0105] Conversation Policies: The heart of the knowledge-driven conversation guidance system lies in the development of conversation policies based on the insights derived from the extracted knowledge graph and the constructed taxonomy. These policies are sets of rules, guidelines, or recommendations that steer the direction of conversations. They ensure that the interactions remain contextually relevant, accurate, and aligned with the available knowledge.

[0106] Dynamic and Personalized Conversations: By incorporating the knowledge graphbased conversation policies, the system is capable of dynamically tailoring conversations to the user's preferences, needs, and the context of the discussion. As the conversation evolves, the system taps into the taxonomy and knowledge graph to provide timely and precise control, ensuring that the exchange does not veer from relevant topics whose policies are set via the knowledge graphs, ontologies and taxonomies precisely derived from the source content.

[0107] Advantages & Benefits: The disclosed technique offers several advantages, including:• Improved conversational accuracy and relevance.• Enriched user experiences through access to comprehensive knowledge.• Support for domain-specific conversations with deep insights.• Adaptation to evolving contexts and information updates.• Stay on topic based on the content of to knowledge graphs• Configurable access to users and groups to topic domains

[0108] Technical Details of the Knowledge-Driven Conversation Guidance System: Knowledge Graph Extraction and Development: The system's process starts with the ingestion of a wide range of content sources, including structured and unstructured data. Unstructured content, such as PDFs, undergoes parsing using advanced natural language processing (NLP) libraries. This parsing extracts textual data, forming text corpora for subsequent processing.import PyPDF2 def parse_pdf(pdf_path): text_corpora = [] with open(pdf_path, 'rb') as pdf_file: pdf_reader = PyPDF2.PdfFileReader(pdf_file) for page_num in range(pdf_reader.numPages): page = pdf_reader.getPage(page_num) text = page.extractText() text_corpora.append(text) return text_corpora

[0109] Ontology and Taxonomy Generation: A Subject Matter Expert uploads a document using the system and has the system construct an ontology that captures the relationships between entities and a taxonomy that organizes them hierarchically.

[0110] Human-in-the-Loop (HITL) Binding: The SME then verifies the ontology and taxonomy and binds them together. Selected documents undergo HITL verification, where human experts verify and refine relationships in the knowledge graph. This step ensures accuracy and resolves inconsistencies.

[0111] Neural Network Training for Knowledge Graph Generation: A neural network is trained to generate knowledge graphs for additional source documents. The training data includes human-verified knowledge graphs and corresponding text corpora. This step is necessary to scale beyond the HITL verified content and rely on Neural Networks - such as Graph Neural Networks - to create a more comprehensive Knowledge Graph without incurring high manual labor. import tensorflow as tf from transformers import T5ForConditionalGeneration, T5Tokenizer def trai n_neu ral_network(trai n i ng_data) : tokenizer = T5Tokenizer.from_pretrained('t5-smaH') model = T5ForConditionalGeneration.from_pretrained('t5-smaH')inputjext = training_data['input_text'] target_graph = training_data['target_graph'] inputs = tokenizer(input_text, return_tensors='pt', padding=True, truncation=True) targets = tokenizer(target_graph, return_tensors='pt', padding=True, truncations rue) inputjds = inputs['input_ids'] targetjds = targets['input_ids'] optimizer = tf.keras. optimizers. Adam(learning_rate=O.001) model. compile(optimizer=optimizer, loss=model. computejoss) model. fit(inputjds, targetjds, epochs=10, batch_size=32)

[0112] JSONL Representation of Knowledge Graph: The ontology, taxonomy, and verified relationships are consolidated into a JSONL file format. This JSONL representation captures the domain's structured knowledge, ready for integration into the conversation guidance system.

[0113] Ontology and Taxonomy Generation with Description Compression and Generalization: The process of ontology and taxonomy generation is enhanced by incorporating similarity techniques to compress descriptions. This step ensures a concise and efficient representation of entities and concepts within the knowledge graph. Descriptions are generalized and compressed based on a predetermined threshold of similarity, resulting in a more compact ontology and taxonomy.

[0114] Description Compression Policy Formulation: To implement the Description Compression Policy, the system utilizes semantic similarity algorithms to identify redundancies and similarities within entity descriptions. Descriptions that surpass a similarity threshold are compressed, replacing specific details with generalized terms. Sample code for achieving this generalization and compression is shown below: from sklearn.feature_extraction.text import TfidfVectorizer from sklearn. metrics. pairwise import cosine_similaritydef compress_description(entity_descriptions, similarity_threshold=0.8): vectorizer = TfidfVectorizer() tfidf_matrix = vectorizer.fit_transform(entity_descriptions) similarity_matrix = cosine_similarity(tfidf_matrix) compressed_descriptions = [] for i, description in enumerate(entity_descriptions): similarjndices = [index for index, score in enumerate(similarity_matrix[i]) if score >= similarity_threshold] if len(similarjndices) > 1 : # Compress descriptions if multiple similar descriptions found generalized_description = generalize_description(description, similarjndices) compressed_descriptions.append(generalized_descri ption) else: compressed_descriptions.append(descri ption) return compressed_descriptions entity_descriptions = ["The American Bald Eagle is a large bird of prey.","The Bald Eagle is native to North America.","The Bald Eagle has a distinctive white head and tail feathers." compressed_descriptions = compress_description(entity_descriptions, similarity hreshold = 0.7) print(compressed_descri ptions)

[0115] Integration into Ontology and Taxonomy: The compressed descriptions are integrated into the ontology and taxonomy construction process. During the establishment of hierarchical relationships, the compressed descriptions are used to ensure that similarconcepts and entities are grouped together, resulting in a more streamlined and concise representation.

[0116] Policy Application Based on Compressed Ontology: The policies for conversation relevance are applied to the compressed ontology and taxonomy. The system uses the generalized and compact descriptions to enhance the efficiency of semantic alignment and relevance determination during conversations. If the conversation aligns with these parameters, it is considered relevant.

[0117] Conversation Policy Formulation and Integration: The JSONL-formatted knowledge graph is integrated into the conversation guidance system. Policies are defined as rules that reference entities, concepts, and relationships within the graph.

[0118] Semantic Alignment and Relevance Determination: During conversations, the system employs NLP techniques to map mentioned entities to nodes in the knowledge graph. Semantic alignment scores are calculated to measure the relevance of the conversation. import spacy def calculate_semantic_alignment(conversation_text, knowledge_graph_entities): nip = spacy. Ioad("en_core_web_sm") conversation_tokens = nlp(conversation_text) knowledge_tokens = [nlp(entity) for entity in knowledge_graph_entities] alignment_score = 0 for conv_token in conversation_tokens: for kg_token in knowledge_tokens: alignment_score += conv_token.similarity(kg_token) return alignment_score / (len(conversation_tokens) * len(knowledge_tokens))

[0119] Relevance Scoring and Threshold: The semantic alignment score obtained from the alignment calculation is used to determine relevance. Conversations with alignment scores exceeding a predefined threshold are considered relevant.def is_conversation_relevant(conversation_text, knowledge_graph_entities, relevance_threshold=0.6): alignment_score = calculate_semantic_alignment(conversation_text, knowledge_graph_entities) return alignment_score >= relevance_threshold

[0120] Dynamic Threshold Adjustment: The threshold for relevance determination can be dynamically adjusted based on the specifics of the conversation or the user context. For instance, a novice user might have a lower threshold than an expert, ensuring that the system remains adaptable to different user levels. def dynamic_relevance_threshold(user_experience_level): if user_experience_level == "novice": return 0.5 el if user_experience_level == "expert": return 0.7 else: return 0.6 # Default threshold user_experience = get_user_experience_level() # Get user's experience level threshold = dynamic_relevance_threshold(user_experience) is_relevant = is_conversation_relevant(conversation_text, knowledge_graph_entities, threshold)

[0121] Integration with User Profiles and Groups: To tailor the conversation experience, the policy is integrated with user profiles and groups. Each user profile or group may have a unique threshold for relevance determination, ensuring conversations align with the user's expertise.

[0122] Policy Enforcement During Conversations: As conversations progress, the semantic alignment policy is applied at intervals or after each user input. The system calculates alignment scores and evaluates the relevance of the ongoing discussion. If theconversation remains above the relevance threshold, it is deemed relevant and aligned with the knowledge graph.

[0123] User Profiles, Groups, and ACLs: User profiles, groups, and access control lists (ACLs) personalize the experience. Policies are associated with users or groups using unique identifiers. These associations enable the application of tailored conversation rules.

[0124] While example embodiments have been particularly shown and described, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the embodiments encompassed by the appended claims.

Claims

CLAIMSWhat is claimed is:

1. A knowledge-driven conversation guidance system, comprising: a processor; and a memory with computer code instructions stored thereon, the memory operatively coupled to the processor such that, when executed by the processor, the computer code instructions cause the guidance system to: load content comprising at least one of structured content and unstructured content; convert the content into content corpora; build one or more embedding vectors based on the content corpora; convey the content corpora to one or both of a generative pre-trained transformer (GPT) and a large language model (LLM), and receive therefrom a preliminary taxonomy and a preliminary ontology; verify the preliminary taxonomy to produce a verified taxonomy; verify the preliminary ontology to produce a verified ontology; produce one or more knowledge graphs related to the verified taxonomy, the verified ontology, or both; and store the verified taxonomy, the verified ontology, the one or more knowledge graphs, the one or more embedding vectors, and the content corpora in a database.

2. The knowledge-driven conversation guidance system of claim 1, wherein the unstructured content comprises one or more of PDF, DOC, and IMG types, and wherein the structured content comprises one or more of a CSV file, a direct database call, or an API invocation that returns JSON or XML structures.

3. The knowledge-driven conversation guidance system of claim 1, wherein to verify the preliminary taxonomy, verify the preliminary ontology, and produce one or more knowledge graphs further comprises verification through a human in the loop (HITL) process that employs a human to (i) manually verify the preliminary taxonomy to produce a verified taxonomy, (ii) manually verify the preliminary ontology to producea verified ontology, and (iii) produce the one or more knowledge graphs from the verified taxonomy and the verified ontology.

4. The knowledge-driven conversation guidance system of claim 1, wherein to produce one or more knowledge graphs further comprises configuration of a neural network using the verified taxonomy, the verified ontology, and the verified knowledge graph to produce a trained neural network, and use of the trained neural network to produce the one or more knowledge graphs from an unverified taxonomy and an unverified ontology.

5. The knowledge-driven conversation guidance system of claim 1, wherein the one or more knowledge graphs generalize a relationship between the verified taxonomy and the verified ontology.

6. The knowledge-driven conversation guidance system of claim 1, wherein the unstructured content is converted to content corpora using optical character recognition.

7. The knowledge-driven conversation guidance system of claim 1, wherein the one or more embedding vectors encapsulates one or more specific associations comprising relationships such as meaning, structure, and context between elements of the content corpora.

8. A knowledge-driven conversation guidance system, comprising: a processor; and a memory with computer code instructions stored thereon, the memory operatively coupled to the processor such that, when executed by the processor, the computer code instructions cause the guidance system to: intercept a communication between a user and an LLM or GPT; evaluate the communication to determine a state of the communication; if the state of the communication aligns with one or more predefined categories of a taxonomy, an ontology, and a knowledge graph associated with an assigned guidance policy, allow the communication to continue; andif the state of the communication does not align with the one or more predefined categories of the taxonomy, the ontology, and the knowledge graph associated with an assigned guidance policy, prevent the communication from continuing.

9. The knowledge-driven conversation guidance system of claim 8, wherein the computer code instructions further cause the guidance system to tie the user to a taxonomy, an ontology, and a knowledge graph in a way that guides the user to a relevant conversation.

10. The knowledge-driven conversation guidance system of claim 9, wherein the user is tied to a taxonomy, an ontology, and a knowledge graph by determination of an alignment score to measure relevance of the communication, comparison of the alignment score to a predetermined relevance threshold, and determination that the communication is relevant when the alignment score exceeds the predetermined relevance threshold.

11. The knowledge-driven conversation guidance system of claim 10, wherein the predetermined relevance threshold is dynamically adjustable based on one or both of (i) specifics of the communication and (ii) context associated with the user.

12. The knowledge-driven conversation guidance system of claim 8, wherein the computer code instructions further cause the guidance system to produce the knowledge graph to be under corporate control and to be configured based on proprietary corporate data.

13. The knowledge-driven conversation guidance system of claim 8, wherein the assigned guidance policy is stored in a guidance policy storage database.

14. The knowledge-driven conversation guidance system of claim 8, wherein the communication and associated communication evaluation result is stored in an audit storage memory.

15. A knowledge-driven method of guiding a conversation, comprising: intercepting a communication between a user and an LLM or GPT; evaluating the communication to determine a state of the communication;if the state of the communication aligns with one or more predefined categories of a taxonomy, an ontology, and a knowledge graph associated with an assigned guidance policy, allowing the communication to continue; and if the state of the communication does not align with the one or more predefined categories of the taxonomy, the ontology, and the knowledge graph associated with an assigned guidance policy, preventing the communication from continuing.

16. A method of configuring a guidance policy, comprising: receiving, from a first storage memory, one or more of (i) a verified taxonomy, (ii) a verified ontology, (iii) one or more knowledge graphs, (iv) one or more embedding vectors, and (v) content corpora; designating a set of relevant topics based on the verified taxonomy, the verified ontology, the one or more knowledge graphs, the one or more embedding vectors, and the content corpora; for each relevant topic of the set of relevant topics, determining a relevance threshold; limiting one or more users to communications within the set of relevant topics; receiving, from an administrator, a designation of at least one LLM or at least one GPT to be used by a user group and limiting members of the user group to use of only the designated LLM or designated GPT; and storing the guidance policies in a second storage memory, the guidance policies being the designated set of relevant topics, the relevance thresholds, the designated at least one LLM, and the designated at least one GPT.

17. The method of claim 16, wherein the relevance thresholds employ natural language processing techniques to map entities mentioned in the communications to nodes in the knowledge graph.

18. The method of claim 16, wherein the knowledge graph is a custom knowledge graph that is under corporate control and is built on proprietary corporate data.

19. The method of claim 16, further comprising determining a semantic alignment score to measure relevance of a LLM or GPT communication, comparing the semantic alignment score to a predetermined relevance threshold, and determining that the communication is relevant when the alignment score exceeds the predetermined relevance threshold.

20. The method of claim 16, wherein the predetermined relevance threshold is dynamically adjustable based on one or both of (i) specifics of the communication and (ii) context associated with the user.

Images