Generating a unified knowledge graph from multimodal data sources using a collaborative multi-agent large-scale language model.

Abstract

This invention provides a method and system for generating a unified knowledge graph from multimodal data sources using a collaborative multi-agent LLM. [Solution] A system presents a collaborative multi-agent knowledge graph search extension generation framework using multiple collaborative multi-agent LLMs to extract information from multiple multimodal documents and build a unified knowledge graph to facilitate query-based searching, wherein a query generation agent generates multiple queries to reveal all possible information present in multiple multimodal documents, a domain model generation agent generates domain models based on the multiple queries, a domain model populator agent populates domain models using data extracted from multiple multimodal documents, and a unified knowledge graph is constructed from the populated domain models, systematizing the extracted information.

Description

Technical Field

[0001] (Cross - reference to related applications and priority) This application claims priority to Indian Patent Application No. 202421074061, filed on September 30, 2024.

[0002] (Technical Field) This disclosure generally relates to multi - agent large language models (LLMs), and more particularly to methods and systems for generating a unified knowledge graph from multimodal data sources using collaborative multi - agent LLMs.

Background Art

[0003] The spread of digital documents, especially PDF - formatted documents, has created important challenges in information retrieval and knowledge management across various industries. Large Language Models (LLMs) have shown remarkable capabilities in natural language processing tasks, but their application to complex information extraction from unstructured documents remains difficult. Conventional Retrieval - Augmented Generation (RAG) approaches have made great progress in enhancing document queries by combining the strengths of LLMs and information retrieval techniques. However, these approaches often struggle with maintaining context for long documents, handling complex multi - hop queries, and providing a transparent inference path. Furthermore, the flat structure of typical RAG approaches limits the ability to capture and utilize the hierarchical nature of information present in many documents.

[0004] The knowledge graph (KG) approach offers a superior alternative to document queries and information retrieval. By representing information as interconnected entities and relationships, the KG approach can more effectively capture the semantic structure of a document. This hierarchical representation enables more nuanced, context-aware queries, supports multi-hop reasoning, and provides a clear origin for extracted information. Furthermore, the KG approach allows for the integration of domain-specific knowledge and ontologities, improving the overall quality and relevance of query responses. However, generating high-quality KGs from unstructured text remains a significant challenge. Existing approaches to KG creation often produce graphs with superficial or less meaningful relationships between nodes, resulting in limited practicality for complex queries and knowledge discovery. The main challenges lie in accurately identifying related entities, establishing meaningful relationships, and capturing the hierarchical structure of information present in the source document. [Overview of the project]

[0005] Embodiments of this disclosure present technical improvements as solutions to one or more of the above-mentioned technical problems recognized by the inventors in conventional systems. For example, one embodiment is a method for generating a unified knowledge graph from a multimodal data source using a collaborative multi-agent large language model. The method includes receiving a plurality of multimodal documents and a plurality of domain history queries associated with the plurality of multimodal documents. Furthermore, the method includes processing the plurality of multimodal documents and identifying text and a plurality of images using a Python library. Furthermore, the method includes chunking the text and a plurality of images into a plurality of chunks, each containing a plurality of text chunks and a plurality of image chunks. Furthermore, the method includes generating a unified knowledge graph from the plurality of chunks using a plurality of collaborative multi-agent large language models (LLMs), the plurality of collaborative multi-agent LLMs including a query generation agent, a domain model generation agent, a domain model populator agent, and a knowledge graph curator agent.The generation of a unified knowledge graph involves (i) generating multiple queries associated with each chunk of multiple chunks via a query generation agent, (a) constructing a query agent by defining multiple query generation behavior parameters and multiple query generation function specification parameters via an input JavaScript Object Notation (JSON) model, wherein the multiple query generation behavior parameters comprise a query generation role, agent knowledge enriched with multiple domain history queries, a query generation interface, and the multiple query generation function specification parameters comprise multiple tasks including multiple query generation actions, multiple query generation tasks and multiple query generation evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple query generation tools, and (b) creating a query prompt template using the multiple instructions, multiple chunks, actions, and the query agent, and (c) generating a query prompt using the query prompt template. (d) generating multiple queries, including inputting query prompts to a query generation agent to generate multiple queries associated with each chunk of the multiple chunks; (ii) generating a domain model associated with each chunk of the multiple chunks based on the generated multiple queries via a domain model generation agent; (iii) populating the domain models generated using each chunk of the multiple chunks via a domain model populator agent to generate a popular domain model from among multiple populated domain models for each chunk of the multiple chunks; and (iv) generating a unified knowledge graph associated with each chunk of the multiple chunks using multiple populated domain models via a knowledge graph curator agent.

[0006] In another embodiment, a system that generates a unified knowledge graph from multimodal data sources using a collaborative multi-agent large-scale language model. This system includes a memory for storing instructions and one or more communication interfaces.The system comprises one or more hardware processors coupled to memory via one or more communication interfaces, the one or more hardware processors receiving, by instruction, a plurality of multimodal documents and a plurality of domain history queries associated with the plurality of multimodal documents, processing the plurality of multimodal documents and identifying text and a plurality of images using the Python library, chunking the text and a plurality of images into a plurality of chunks including a plurality of text chunks and a plurality of image chunks, and generating a unified knowledge graph from the plurality of chunks using a plurality of cooperative multi-agent large language models (LLMs), wherein the plurality of cooperative multi-agent LLMs generate a unified knowledge graph including a query generation agent, a domain model generation agent, a domain model populator agent, and a knowledge graph curator agent, and generating a unified knowledge graph means (i) generating a plurality of queries associated with each chunk of the plurality of chunks via the query generation agent, and (a) input JavaScript Object (b) Constructing a query agent by defining multiple query generation behavior parameters and multiple query generation function specification parameters via a Notation (JSON) model, wherein the multiple query generation behavior parameters comprise a query generation role, agent knowledge enriched with multiple domain history queries, and a query generation interface, and the multiple query generation function specification parameters comprise multiple tasks including multiple query generation actions, multiple query generation tasks, and multiple query generation evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple query generation tools; (b) creating a query prompt template using multiple instructions, multiple chunks, actions, and a query agent; (c) generating a query prompt using the query prompt template; and (d) inputting the query prompt to the query generation agent to generate multiple queries associated with each chunk of the multiple chunks; This includes (ii) generating multiple queries, including (ii) generating a domain model associated with each chunk of multiple chunks based on the generated multiple queries via a domain model generation agent, (iii) populating the domain models generated using each chunk of multiple chunks via a domain model populator agent to generate a popular domain model from among multiple populated domain models for each chunk of multiple chunks, and (iv) generating a unified knowledge graph associated with each chunk of multiple chunks using the multiple populated domain models via a knowledge graph curator agent.

[0007] In yet another embodiment, a non-temporary, computer-readable medium is provided for generating a unified knowledge graph from a multimodal data source using a collaborative multi-agent large-scale language model. The method includes receiving a plurality of multimodal documents and a plurality of domain history queries associated with the plurality of multimodal documents. Furthermore, the method includes processing the plurality of multimodal documents to identify text and a plurality of images using a Python library. Furthermore, the method includes chunking the text and a plurality of images into a plurality of chunks, each containing a plurality of text chunks and a plurality of image chunks. Furthermore, the method includes generating a unified knowledge graph from the plurality of chunks using a plurality of collaborative multi-agent large-scale language models (LLMs), the plurality of collaborative multi-agent LLMs including a query generation agent, a domain model generation agent, a domain model populator agent, and a knowledge graph curator agent.Unified Knowledge Graph generation involves (i) generating multiple queries associated with each chunk of multiple chunks via a query generation agent, (a) constructing a query agent by defining multiple query generation behavior parameters and multiple query generation function specification parameters via an input JavaScript Object Notation (JSON) model, wherein the multiple query generation behavior parameters comprise a query generation role, agent knowledge enriched with multiple domain history queries, a query generation interface, and the multiple query generation function specification parameters comprise multiple tasks including multiple query generation actions, multiple query generation tasks and multiple query generation evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple query generation tools, and (b) creating a query prompt template using the multiple instructions, multiple chunks, actions, and query agent, and (c) generating a query prompt using the query prompt template. (d) generating multiple queries, including inputting query prompts to a query generation agent to generate multiple queries associated with each chunk of the multiple chunks; (ii) generating a domain model associated with each chunk of the multiple chunks based on the generated multiple queries via a domain model generation agent; (iii) populating the domain models generated using each chunk of the multiple chunks via a domain model populator agent to generate a popular domain model from among multiple populated domain models for each chunk of the multiple chunks; and (iv) generating a unified knowledge graph associated with each chunk of the multiple chunks using multiple populated domain models via a knowledge graph curator agent.

[0008] Please understand that the above summary and the following detailed description are both illustrative and intended to further illustrate the embodiments of the present invention as described in the claims.

[0009] The accompanying drawings incorporated herein and constituting part thereof illustrate embodiments of the present disclosure and serve to illustrate the principles of the present disclosure together with this specification. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an exemplary system for generating a unified knowledge graph from multimodal data sources using multiple collaborative multi-agent large-scale language models (LLMs), according to some embodiments of the present disclosure.

[0011] [Figure 2] This is a functional architecture illustrating the process flow of a system for generating a unified knowledge graph from multimodal data sources using multiple collaborative multi-agent LLMs, according to some embodiments of the present disclosure.

[0012] [Figure 3A] A flowchart shows a method for generating a unified knowledge graph from multimodal data sources using multiple collaborative multi-agent LLMs, using the system of Figure 1, according to several embodiments of the present disclosure (collectively represented as Figure 3). [Figure 3B] A flowchart shows a method for generating a unified knowledge graph from multimodal data sources using multiple collaborative multi-agent LLMs, using the system of Figure 1, according to several embodiments of the present disclosure (collectively represented as Figure 3).

[0013] [Figure 4] The present disclosure shows partial knowledge graphs of a unified knowledge graph generated using multiple popularized domain models according to several embodiments of this disclosure. [Modes for carrying out the invention]

[0014] Those skilled in the art will understand that any block diagrams in this specification represent conceptual diagrams of exemplary systems and devices that embody the principles of this subject. Similarly, any flowcharts, flow charts, etc., whether or not such a computer or processor is explicitly shown, will be understood to represent various processes that can be performed by such a computer or processor, substantially represented in a computer-readable medium.

[0015] Exemplary embodiments will be described with reference to the accompanying drawings. In the drawings, the leftmost digit of the reference number identifies the drawing in which the reference number first appears. Where convenient, the same reference number will be used throughout the drawings to refer to the same or similar parts. While examples and features of the disclosed principles are described herein, modifications, adaptations, and other implementations are possible without departing from the scope of the disclosed embodiments.

[0016] A significant amount of knowledge exists in industry, primarily in an unstructured, non-queryable form. Industry constantly seeks ways to make this unstructured data accessible on demand and facilitate informed decision-making, as opposed to relying solely on individual expertise. The emergence of Large-Scale Language Models (LLMs) has given rise to a Search-Enhanced Generative (RAG) approach, which vectorizes unstructured data, stores it in a vector store, and searches based on word similarity matches. LLMs are neural networks trained on vast amounts of text data and perform a wide range of natural language processing tasks. Models such as the Generative Pre-trained Transformer (GPT) series, Bidirectional Encoder Representations from Transformers (BERT), and Text-To-Text Transfer Transformer (T5) have demonstrated remarkable capabilities in understanding and generating human-like text. LLMs have shown proficiency in tasks including text summarization, question answering, and language translation. However, applying LLMs to complex information retrieval and knowledge structuring tasks remains a challenging task.

[0017] A knowledge graph (KG) is a structured representation of information that captures entities and their relationships in a graph format. KGs are fundamental to a variety of applications, including semantic retrieval, question answering systems, and recommender systems. KGs provide a means to efficiently organize and query complex and interconnected information. Several approaches exist for automatically generating KGs from unstructured text, including rule-based approaches, supervised learning approaches, unsupervised and semi-supervised approaches, and neural network-based approaches. Rule-based approaches extract entities and relationships from text using predefined patterns and rules. While these rule-based approaches are effective for specific domains, they often lack flexibility and require significant manual effort to create and maintain the rules. Supervised learning approaches use machine learning models trained on annotated datasets to identify entities and relationships within text. These are more adaptable than rule-based approaches but may require large amounts of labeled training data. Unsupervised and semi-supervised approaches attempt to extract knowledge graph elements with minimal labeled or unlabeled data, often using techniques such as clustering or remote monitoring. These methods are more scalable, but they may suffer from lower accuracy.

[0018] While neural network-based approaches have shown promise in capturing complex semantic relationships, they often struggle to generate meaningful, structured, and hierarchical knowledge representations. However, a significant drawback of these approaches is their limited accuracy for factual data and aggregate-type queries, especially as corpus size increases. Furthermore, the Knowledge Graph (KG) approach offers a superior alternative for document queries and information retrieval. Additionally, the KG approach enables the integration of domain-specific knowledge and ontologities, improving the overall quality and relevance of query responses. However, generating high-quality KGs from unstructured text remains a significant challenge. Existing approaches for KG generation often produce graphs of superficial or less meaningful relationships between nodes, limiting their usefulness for complex queries and knowledge discovery. The main challenges lie in accurately identifying related entities, establishing meaningful relationships, and capturing the hierarchical structure of information present in the source documents. Furthermore, the Knowledge Graph Search Enhancement Generation (RAG) approach has demonstrated significant effectiveness in querying private, short, unstructured data. However, when dealing with large corpora, these methods often fail to provide accurate factual answers, frequently resulting in a lack of context and an inability to establish domain relationships.

[0019] Recent advances in artificial intelligence (AI) demonstrate the potential of multi-agent systems for tackling complex tasks. These approaches distribute cognitive load across multiple expert agents, often leading to more robust and effective solutions. The literature ("P. Chen, B. Han, and S. Zhang, "CoMM: Collaborative Multi-Agent, Multi-Reasoning-Path Prompting for Complex Problem Solving," arXiv:2404.17729v1 [cs.CL], Apr.26, 2024.") introduces the Collaborative Multi-Agent, Multi-Reasoning-Path (CoMM) prompting framework. In this approach, LLMs play different roles within the problem-solving team, facilitating collaborative problem solving. CoMM applies different reasoning paths to different roles, effectively implementing few-shot prompting in multi-agent scenarios. This research demonstrated significant improvements in solving complex scientific problems at the university level. Furthermore, the paper AutoGen ("Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, . Zhang, S. Zhang, J. Liu, AH Awadallah, RW White, D. Burger, and C. Wang, "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation") introduced a framework for building multi-agent systems using LLM. This research revealed how different agents can be assigned specific roles and cooperate to solve complex tasks, providing a foundation for multi-agent systems in various domains.Furthermore, ChatDev ("C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong, J. Xu, D. and M. Sun," "ChatDev: Communicative Agents for Software Development," "arXiv:2307.07924 [cs.SE], 2024.") presents a collaborative software development framework using multiple LLM-based agents. This demonstrates how a multi-agent approach can be applied to complex and creative tasks such as software development. Furthermore, task-oriented dialogue systems, as described in the literature ("W. He, Y. Sun, M. Yang, F. Ji, C. Li, and R. Xu, "Multi-goal multi-agent learning for task-oriented dialogue with bidirectional teacher-student learning," Knowledge-Based Systems, vol. 210, p. 106667, 2020. doi:"), are multi-agent frameworks for task-oriented dialogue systems that demonstrate how multiple expert agents can collaborate to handle complex conversational tasks. Existing multi-agent LLMs focus on document generation using techniques such as named entity recognition (NER), relation extraction (RE), and event extraction (EE). Existing multi-agent LLMs are used collaboratively, with each agent specializing in extracting entities, relationships, and events from a document. Moreover, existing multi-agent LLMs use a more generalized collaborative framework without specific customization. A generalized collaborative framework with limited contextual understanding can struggle to capture subtle context because agents operate on predefined tasks without adapting to the specific content or domain of the document. This could lead to missing or misinterpreting nuances in the extraction of entities, relationships, or events.In generalized collaborative frameworks, agents may face inefficiencies due to a lack of streamlined communication and collaboration protocols. This can lead to redundant efforts, conflicting outputs, or processing delays, as agents may not be able to effectively share insights or coordination based on collective knowledge. Furthermore, existing generalized collaborative frameworks lack flexibility in task expertise, which may limit agents' ability to adapt to the varying complexities of different documents. This lack of flexibility can hinder performance in specialized fields where unique extraction strategies or methodologies are required to achieve high accuracy. In addition, scalability issues in existing generalized frameworks may prevent them from scaling well as the complexity or volume of documents increases. As the number of agents and tasks increases, managing interactions and ensuring efficient processing becomes increasingly difficult, potentially leading to bottlenecks or performance degradation.

[0020] Embodiments of this specification provide methods and systems for generating a unified knowledge graph from multimodal data sources using multiple collaborative multi-agent LLMs, according to several embodiments of the disclosure. The proposed methods present a collaborative multi-agent knowledge graph RAG framework designed to enhance the capabilities of LLMs in complex information retrieval scenarios. The collaborative multi-agent knowledge graph RAG framework includes a domain model generation agent, a domain model populator agent, and a knowledge graph curator agent, each agent coordinated through a customization model. The query generation agent creates multiple queries related to multiple text chunks and multiple image chunks within multiple multimodal documents, and the domain model generation agent constructs a domain model based on the multiple queries. The domain model populator agent populates the data extracted from the multiple multimodal documents into a domain model. The knowledge graph curator agent generates a unified knowledge graph from the multiple populated domain models in Neo4j (e.g., a graph database management system) and organizes the extracted information for accurate retrieval. Each LLM in the multiple collaborative multi-agent LLMs interacts with each other, evaluates the output, and provides feedback to enhance the retrieval process. Furthermore, one or more queries are converted into cryptographic queries using LLM, processed by a unified knowledge graph engine, and returned in human-interpretable natural language. The disclosed method enhances information retrieval from multiple multimodal documents by reducing hallucinations, incomplete responses, and factual errors. The disclosed method is evaluated against the publicly available technical report "Operations & Maintenance Best Practices" and the conventional knowledge graph generation software Neo4j GraphBuilder. The results demonstrate that the proposed approach outperforms GraphBuilder in both the quantity and quality of extracted information, identifying significantly more entities and discovering unique contextually significant relationships.

[0021] Referring now to the drawings, and more particularly to FIGS. 1 - 4, like reference numerals throughout the figures consistently refer to related features, and preferred embodiments are shown, which are described in the context of the following exemplary systems and / or methods.

[0022] FIG. 1 is a functional block diagram of a system 100 for generating a unified knowledge graph from multimodal data sources using a plurality of cooperative multi - agent LLMs, according to some embodiments of the present disclosure. In one embodiment, system 100 includes one or more hardware processors 104, a communication interface device or input / output (I / O) interface 106 (also referred to as an interface), and one or more data storage devices or memories 102 operatively coupled to the one or more hardware processors 104. The one or more processors 104 can be one or more software processing components and / or hardware processors.

[0023] Referring to the components of system 100, in one embodiment, processor 104 can be one or more hardware processors 104. In one embodiment, the one or more hardware processors 104 can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and / or any device that operates on signals based on operational instructions. Among other capabilities, processor 104 is configured to fetch and execute computer - readable instructions stored in the memory. In one embodiment, system 100 can be implemented in various computing systems such as a laptop computer, notebook, handheld device (e.g., smartphone, tablet phone, mobile communication device, etc.), workstation, mainframe computer, server, network cloud, and the like.

[0024] The I / O interface 106 can include various software and hardware interfaces such as, for example, a web interface, a graphical user interface, etc., and can facilitate multiple communications within a wide variety of network N / W and protocol types, including, for example, wired networks such as LAN, cable, etc., and wireless networks such as WLAN, cellular, or satellite. In one embodiment, the I / O interface 106 can include one or more ports for connecting a number of devices to each other or to another server.

[0025] The memory 102 can include any computer-readable medium known in the art, including, for example, volatile memory such as static random access memory (SRAM) and dynamic random access memory (DRAM), and / or non-volatile memory such as read-only memory (ROM), erasable programmable ROM, flash memory, hard disk, optical disk, and magnetic tape. Thus, the memory 102 can configure information related to the input / output of each step executed by the processor 104 of the system 100 and method of the present disclosure. In one embodiment, a database 108 is configured within the memory 102, and the database 108 includes information regarding a plurality of multimodal documents, a plurality of domain history questions, text, a plurality of images, a plurality of text chunks, a plurality of image chunks, a unified knowledge graph, a plurality of queries, a plurality of query generation action parameters, a plurality of query generation functional specification parameters, a plurality of knowledge graph query action parameters, a plurality of knowledge graph query functional specification parameters.

[0026] The database 108 further includes a plurality of domain model generation action parameters, a plurality of domain model generation functional specification parameters, a plurality of domain model populater action parameters, a plurality of domain model populater functional specification parameters, a plurality of knowledge graph curator action parameters, a plurality of knowledge graph curator functional specification parameters, and information including these.

[0027] Memory 102 further comprises a Python library, a query generation agent, a domain model generation agent, a domain model populator agent, a knowledge graph curator agent, a knowledge graph query agent, a query agent, a unified knowledge graph query agent, a domain model agent, a domain model populating agent, a knowledge graph agent, and multiple modules for various technologies such as these. The technologies described above are implemented as a logically self-contained portion of a software program, a self-contained hardware component, and / or a self-contained hardware component having a logically self-contained portion of a software program embedded in each of the hardware components (e.g., hardware processor 104 or memory 102) that, when executed, perform the methods described herein.

[0028] Memory 102 further includes (or may further include) information relating to the inputs / outputs of each step performed by the system and method of the present disclosure. In other words, the inputs supplied at each step and the outputs generated at each step are contained in Memory 102 and can be used in further processing and analysis.

[0029] Figure 2 is a functional architecture showing the process flow of System 100 of Figure 1 for generating a unified knowledge graph from multimodal data sources using multiple collaborative multi-agent LLMs, according to several embodiments of the present disclosure. The functional architecture describes receiving multiple multimodal documents and multiple domain history queries associated with the multiple multimodal documents. The multiple multimodal documents include user manuals, maintenance reports, standard operating procedures (SOPs), and the like. The agent pool depicted in Figure 2 includes a query generation agent, a domain model generation agent, a domain model populator agent, a knowledge graph curator agent, and a unified knowledge graph query agent. Multiple text chunks and multiple image chunks are input to the query generation agent along with multiple domain history queries to generate multiple queries associated with each chunk of the multiple chunks. The domain model generation agent generates a domain model using the generated queries. The domain model populator agent populates the generated domain model using each chunk of the multiple chunks and generates a populated domain model from among the multiple populated domain models for each chunk of the multiple chunks. The knowledge graph curator agent uses multiple popular domain models to generate a unified knowledge graph associated with each of several chunks. The knowledge graph query agent queries the generated unified knowledge graph and generates graph data based on one or more queries. Finally, using LLM, the graph data is converted into human-readable natural language, providing answers to one or more queries.

[0030] Figures 3A and 3B show flowcharts of methods for generating a unified knowledge graph from multimodal data sources using multiple collaborative multi-agent LLMs with the system of Figure 1, according to several embodiments of the present disclosure.

[0031] In one embodiment, system 100 comprises one or more data storage devices or memories 102 operably coupled to a processor 104 and configured to store instructions for the processor 104 to execute steps of method 400. The steps of method 400 of this disclosure will now be described with reference to components or blocks of system 100 as shown in Figure 1, a functional architecture as shown in Figure 2, and steps in a flowchart as shown in Figure 3. Process steps, method steps, techniques, etc., may be described in a sequential order, but such processes, methods, and techniques may be configured to operate in an alternative order. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of the processes described herein may be performed in any practical order. Furthermore, several steps may be performed simultaneously.

[0032] Referring to the steps in Figure 3, in step 302 of method 300, one or more hardware processors 104 receive multiple multimodal documents and multiple domain history questions related to the multiple multimodal documents. The multiple domain history questions often refer to past queries made by the user in a particular context or domain. These questions may encompass a range of topics, issues, or themes relevant to the user base. By analyzing the multiple domain history questions, the query generation agent can better understand the user's intent, preferences, and the complexity of domain-specific language.

[0033] Multiple domain history questions provide crucial context that enables query generation agents to interpret current user queries more effectively. For example, if a user frequently asks about "boiler pressure checks," the query generation agent can associate this term with maintenance best practices and respond more accurately. By examining past questions across multiple domains, query generation agents can identify common trends in boiler maintenance inquiries, such as seasonal concerns about heating efficiency in winter. Multiple history questions help query generation agents produce responses that directly address user needs. If past queries frequently include "What should I do if my boiler leaks?", the query generation agent can be trained to provide detailed troubleshooting guides when similar queries occur. By reviewing multiple history questions and past interactions, query generation agents can learn from past inaccuracies. If a query regarding "boiler maintenance frequency" is repeatedly misinterpreted, adjustments can be made to improve accuracy. Furthermore, multiple history questions enable query generation agents to provide personalized responses. For example, if a user frequently asks about "boiler energy saving tips," the query generation agent can prioritize this information, making interactions more relevant and tailored. Furthermore, multiple past questions enable the query generation agent to engage in meaningful conversations. If a user previously inquired about "boiler safety checks," the query generation agent can follow up with questions such as, "Have you had your annual safety check done?" Moreover, multiple past questions contribute to a broader and more nuanced knowledge base. For example, queries about specific boiler brands or models help the query generation agent understand variations in maintenance methods. The range of questions submitted by users reflects diverse perspectives and experiences, enriching the query generation agent's ability to meet the diverse needs of various users.Multiple past questions create a feedback loop that supports continuous learning. The query generation agent can track which queries are used most frequently and adapt the training data accordingly. Multiple past questions serve as a benchmark for evaluating the performance of the query generation agent, allowing developers to measure improvements and identify areas for further training.

[0034] In step 304 of method 300, one or more hardware processors 104 process multiple multimodal documents and use the Python library to identify text and multiple images.

[0035] In step 306 of method 300, one or more hardware processors 104 chunk the text and multiple images into multiple chunks, each containing multiple text chunks and multiple image chunks.

[0036] In step 308 of Method 300, one or more hardware processors 104 generate a unified knowledge graph from multiple chunks using multiple collaborative multi-agent LLMs. These multiple collaborative multi-agent LLMs include a query generation agent, a domain model generation agent, a domain model populator agent, a knowledge graph curator agent, and a knowledge graph query agent. The multiple collaborative multi-agent LLMs cooperate by leveraging natural language processing capabilities and structured data mapping capabilities. They interact with integrated tools such as text analysis algorithms, graph modeling tools, and data cleaning systems, as well as external application programming interfaces (APIs) for industry standards and graph visualization. The disclosed collaborative multi-agent knowledge graph (RAG) framework enables the application of expertise at each stage of the unified knowledge graph generation process, from generating multiple queries to creating complex hierarchical unified knowledge graphs. The disclosed method efficiently handles large amounts of text and complex domain structures. Key features of multiple collaborative multi-agent LLMs include adaptive learning capabilities, performance enhancement through exposure to diverse manual content, seamless integration between multiple collaborative multi-agent LLMs with a feedback loop for continuous improvement, customizable options to address different domains, operational needs, and industry-specific requirements, a strong focus on data quality, and ensuring the accuracy, completeness, and relevance of the generated unified knowledge graph.

[0037] To generate a unified knowledge graph, a customized model is built for each of the multiple collaborative multi-agent LLMs. The customized model includes two scalable components: behavioral customization and functional specifications. Scalable components are modular elements that allow the customized model to grow and adapt to the requirements of various users without compromising accuracy. Behavioral customization involves coordinating the responses, interactions, and domains of multiple collaborative multi-agent LLMs, including roles, domain knowledge, and interfaces. This ensures that multiple collaborative multi-agent LLMs behave according to defined parameters and provide context-aware responses. For example, if the role is "maintenance engineer," the LLM of the collaborative multi-agent LLM will adjust its responses to provide maintenance-related guidance; for domain knowledge, the LLM will adjust its responses to leverage expertise in a specific field such as "maintenance of oil and gas industry equipment"; and for interfaces, if the LLM's interface type is text-based with natural language understanding capabilities, the LLM will interact through text and understand and generate responses accordingly. For example, if an LLM's task is to provide safety guidelines, the LLM will use domain knowledge and role definitions to ensure that their response is relevant to the oil and gas industry and complies with safety protocols.

[0038] The functional specification includes defining the specific functions, actions, tasks, and tools that the LLM should handle. It details what the LLM can do and how these functions should be performed based on predefined attributes. The capabilities and actions expected of the LLM are outlined in the functional specification. Actions include core actions and extended actions. Core actions include basic tasks that the LLM should handle, such as "Providing maintenance procedures" and "Troubleshooting assistance." Extended actions include additional tasks, such as "Generating equipment manuals" and "Providing safety guidelines." Tasks include primary tasks, which are mandatory tasks, such as "Scheduling periodic maintenance" and "Emergency repair guidance," and advanced tasks, such as more specialized tasks, such as "Optimizing equipment performance" and "Conducting safety audits." Tools include integrated tools, such as systems and databases that the LLM should use internally, such as "Equipment database" and "Maintenance tracking system," and external tools, such as external resources and APIs that the LLM should interact with, such as "Industry standards and regulatory APIs." For example, LLM should extract information from an integrated equipment database to provide detailed maintenance procedures and allow access to the latest information on industry standards and regulations using external APIs.

[0039] Generating a unified knowledge graph from multiple chunks using multiple collaborative multi-agent LLMs is described through steps 304a to 304d of Method 300. In step 304a, Method 300 generates multiple queries related to each chunk of multiple chunks via a query generation agent. For example, the query generation agent simulates a field engineer or operator and generates multiple queries related to a domain. In the boiler maintenance example, the main functions of the query generation agent include (a) creating questions about maintenance procedures for various boiler types, (b) generating queries about troubleshooting common boiler problems, (c) creating questions about inspection processes including non-destructive testing (NDE) methods, and (d) questions about safety protocols and compliance guidelines. The query generation agent is highly customizable and can be adapted to generate multiple queries for any industry or domain, not just boiler maintenance. The steps for generating multiple queries related to each chunk of multiple chunks via a query generation agent include (a) constructing the query agent by defining multiple query generation behavior parameters and multiple query generation function specification parameters via an input JavaScript Object Notation (JSON) model. A query agent is a customized model of a query generation agent. Multiple query generation behavior parameters include query generation roles, agent knowledge enriched with multiple domain history queries, and query generation interfaces. Multiple query generation function specification parameters include multiple query generation actions, multiple query generation tasks and multiple query generation evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple query generation tools. Multiple query generation tools help the query generation agent interact with the external environment to gather information useful for processing tasks. Multiple query generation tools encompass various techniques to facilitate information retrieval and processing.Examples of implementations include database management systems such as My Structured Query Language (MySQL) and MongoDB for storing and managing data, search engines such as Elasticsearch and Apache Solr for full-text search, application programming interface (API) query tools such as Graph Query Language (GraphQL) and Representational State Transfer (REST)ful API for data access, data analysis tools such as Apache Spark and Tableau, natural language processing libraries such as SpaCy and Natural Language Toolkit (NLTK), business intelligence platforms such as Power Business Intelligence (BI) and Looker for visual reporting, custom query builders such as Knex.js for constructing SQL queries, and collaboration tools such as Jupyter Notebooks and Google Data Studio for interactive data exploration. By using these tools together, the functionality and effectiveness of the query generation agent are improved. (b) Create a query prompt template using multiple instructions, multiple chunks, actions, and the query agent. (c) Generate a query prompt using the query prompt template. (d) Send the query prompt to the query generation agent and generate multiple queries associated with each chunk of multiple chunks.

[0040] A query prompt template created according to some embodiments of this disclosure is disclosed as follows: "You are an operator with expertise in cable installation and maintenance, and you are tasked with generating questions that a real operator of an oil and gas field in operation might ask. Your goal is to create questions based solely on the provided text content. Model definition (for reference): {query_generator_agent}. Text content {text_chunk / image} Generate {num_questions} questions from the above text content."

[0041] In step 304b, method 300 uses one or more hardware processors to generate domain models associated with each chunk of multiple chunks based on multiple generated queries via a domain model generation agent. The domain model generation agent generates multiple popularized domain models from multiple multimodal documents. The main functions of the domain model generation agent are detailed entity extraction, multilevel relationship identification, hierarchical structure creation, attribute refinement, and ontology integration and cross-domain connection identification. This domain model generation agent is essential for creating multiple popularized domain models with multilevel relationships and attributes that enable the construction of an accurate and semantically rich unified knowledge graph. The step of generating domain models associated with each chunk of multiple chunks based on multiple generated queries via the domain model generation agent includes: (a) Construct a domain model agent by defining multiple domain model generation behavior parameters and multiple domain model generation function specification parameters. The domain model agent is a customized model of the domain model generation agent. The multiple domain model generation behavior parameters include domain model generation roles, domain knowledge, and domain model generation interfaces. The multiple domain model generation function specification parameters include multiple domain model generation tasks consisting of multiple domain model generation actions, multiple domain model generation query tasks, and multiple domain model generation evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple domain model generation tools. The multiple domain model generation tools facilitate the creation of structured representations to enhance information ingestion. Examples include Unified Modeling Language (UML) tools such as PlantUML and StarUML for diagram creation, entity-relationship (ER) modeling tools such as Lucidchart and dbdiagram.io for database design, and code generation tools such as JHipster and Yeoman for scaffolding applications. NoSQL databases like Hackolade and MongoDB Compass offer visual modeling capabilities, while API modeling tools such as Swagger (OpenAPI) and Postman help define and generate API documentation. Furthermore, domain-specific language (DSL) tools like Xtext and ANTLR (Another Tool for Language Recognition) enable the development of custom programming languages ​​and domain-specific languages, streamlining the modeling process and improving efficiency. (b) Create a domain model generation prompt template using multiple instructions, multiple chunks, and multiple queries, actions, and domain model agents associated with each chunk. (c) Generate a domain model generation prompt from the domain model generation prompt template. (d) Enter the domain model generation prompt into the domain model generation agent and generate a domain model associated with each of the multiple chunks based on the multiple queries that are generated.

[0042] The creation of a domain model generation prompt according to some embodiments of this disclosure is as follows: "You are an AI agent acting as a comprehensive domain model generator. Your task is to create a detailed hierarchical schema of a neo4j graph database with nodes and relationships, based on the information provided." Agent model; {Domain model generation agent} Manual text:: {manual_text} Generated queries: {generated_queries} Create a comprehensive schema: 1. Main Entities and Their Descriptions 2. Detailed attributes of each entity 3. Subentities representing hierarchical structure 4. Relationships between entities, including the type and details of the relationship. Avoid generic relationships and ensure that all relationships are meaningful and contextual. 5. Cross-domain relationships, if applicable. 6. If there are implicit relationships or attributes, they can be inferred. The model is hierarchical, with primary entities containing secondary entities where appropriate. The focus is on creating a structure suitable for generating knowledge graphs. Important: The domain model should be provided as a raw JSON object without any formatting or Markdown syntax added. The response must be a valid JSON string that can be directly parsed by a JSON parser. Without additional characters or formatting, a JSON object begins with {{ and ends with {{}}. You should only refer to the JSON keys and relationships; you should not populate the node values.

[0043] In step 304c, method 300 populates the domain models generated using each chunk of multiple chunks in order to generate multiple populated domain models for each chunk of multiple chunks via a domain model populator agent. The domain model populator agent populates domain models with content from multiple multimodal documents. The main functions of the domain model populator agent include content extraction and mapping to domain models, relational population, and contextual analysis, implicit information inference, and consistency verification. The steps for generating multiple populated domain models for each chunk of multiple chunks via a domain model populator agent include: (a) Construct a Domain Model Populator agent by defining multiple Domain Model Populator behavior parameters and multiple Domain Model Populator function specification parameters. The Domain Model Populator agent is a customized model of the Domain Model Populator agent. The multiple Domain Model Populator behavior parameters include Domain Model Populator roles, domain knowledge, and Domain Model Populator interfaces. The Domain Model Populator function specification parameters include multiple Domain Model Populator actions, multiple Domain Model Populator tasks consisting of multiple Domain Model Populator query tasks and multiple Domain Model Populator evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple Domain Model Populator tools. The multiple Domain Model Populator tools are designed to automate and streamline the process of filling the domain model with data. Examples include data migration tools such as Talend and Apache Niagara Files (NiFi) to facilitate data transfer between systems, and mock data generation tools such as Mockaroo and Faker to create realistic test data. Database population tools like DBMonster and DbForge Data Generator generate large amounts of test data for the database, while API testing tools like Postman and SoapUI can populate multiple populated domain models with data during API testing. ETL (Extract, Transform, Load) tools like Apache Airflow and Informatica manage complex data pipelines for populating multiple populated domain models from multiple multimodal documents. Furthermore, custom scripting solutions using languages ​​like Python and Ruby on Rails fixtures can be developed to load and insert data as needed. (b) Create a domain model populator prompt template using multiple instructions, multiple chunks, domain models, actions, and a domain model populator agent. (c) Generate a domain model populator prompt from a domain model populator prompt template. (d) Enter the Domain Model Populator prompt into the Domain Model Populator agent to generate multiple populated domain models associated with each of the multiple chunks.

[0044] The creation of a domain model populator prompt template according to several embodiments of this disclosure is shown below: "You are an AI agent responsible for inputting the domain model using content from the technical manual." Use the following model definition to guide the task: {Domain Model Populator Agent} The following is the structure of the input domain model: {domain_model} Here is the text extracted from the manual: {input_text} Your task is to input relevant information from text into the domain model. Follow the following guidelines: 1. Maintain the structure and relationships defined in the model. 2. Extract the entity, attribute, relationship, and hierarchical information specified in the agent model. 3. Verify that the input model contains all the information necessary for effective knowledge graph generation. 4. Pay particular attention to maintaining the hierarchy and connections between entities. 5. If you encounter information that does not fit the current model structure, add it as an additional attribute or relationship. 6. Cross-reference information across different sections of the input text to ensure a comprehensive population. 7. Prioritize the most recent or most relevant data to resolve any inconsistencies or contradictions in the information. 8. Important: Omit fields, attributes, or relationships that do not contain information in the text. Do not include empty fields or fields marked "Not found in text". Provides the popularized domain model in JSON format. The response will contain only the pure JSON of the popularized domain model, without any agent or process description or metadata. Important: The domain model should be provided as a raw JSON object, without any formatting or Markdown syntax added. The response must be a valid JSON string that can be directly parsed by a JSON parser. This command starts a JSON object with {{ and ends with}}, without adding any characters or formatting.

[0045] In step 304d, method 300 / 1 or 2 or more hardware processors generate a unified knowledge graph associated with each chunk of multiple chunks using multiple popularized domain models through a knowledge graph curator agent. The knowledge graph curator agent creates a unified knowledge graph in Neo4j from multiple popularized domain models. The main functions of the knowledge graph curator agent include converting domain models into graph structures, generating cipher queries for graph creation, ensuring full connectivity and hierarchy, performing data cleaning and normalization, optimizing the graph structure, and implementing advanced graph algorithms. Generating a unified knowledge graph associated with each chunk of multiple chunks using multiple input domain models through the knowledge graph curator agent includes: (a) Construct a knowledge graph agent by defining multiple knowledge graph curator behavior parameters and multiple knowledge graph curator function specification parameters. The knowledge graph is a customization model for the knowledge graph curator agent. Multiple knowledge graph curator behavior parameters include the role of the knowledge graph curator, domain knowledge, and knowledge graph curator interface. Multiple knowledge graph curator function specification parameters include multiple knowledge graph curator actions, multiple knowledge graph curator query tasks and multiple knowledge graph curator evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple knowledge graph curator tools. Multiple knowledge graph curator tools help create, manage, and enhance a unified knowledge graph that structures information for better accessibility. Notable examples include graph database platforms such as Neo4j and Amazon Neptune, which facilitate the construction and querying of a unified knowledge graph. Ontology editors like JPEG2026062517000002.jpg838 enable the creation and management of knowledge representations. Visualization tools like Gephi and Graphistry assist in the analysis and exploration of graph data. Data integration tools such as Apache Jena and Talend connect various sources and input them into a unified knowledge graph. Build tools like Kgtk and Stardog provide capabilities for manipulating graph data. Furthermore, AI-powered tools like Diffbot automate data extraction from web pages, and OpenRefine assists with data cleanup, further enhancing the development and maintenance of the unified knowledge graph. (b) Create a knowledge graph curator prompt template using instructions, multiple populated domain models, actions, and knowledge graph agents. (c) Generate a knowledge graph curator prompt from a knowledge graph curator prompt template. (d) Enter the knowledge graph curator prompt into the knowledge graph curator agent to generate a unified knowledge graph associated with each of the multiple chunks.

[0046] The creation of a knowledge graph curator prompt template according to some embodiments of this disclosure is as follows: "Knowledge graph instructions for #GPT-4o" ##1. Overview You are a top-level algorithm designed to extract information in a structured format to build a knowledge graph. **Nodes** represent entities and concepts. **Relationships** connect nodes and define the nature of those connections. A **property** is a key-value pair associated with a node and its relationship. ## 2. Cypher Query Format For each entity Create a node in the following format: `CREATE (n: NODE_LABEL {{id: 『NODE_ID』, property1: 『value1』,...}})`. Ensure that the NODE_ID property is correctly enclosed in quotation marks and escaped. Regarding each relationship Create a relationship using the following format: `MATCH (a: NODE_LABEL {{id: 『NODE_ID』}}), (b: NODE_LABEL {{ id: 『NODE_ID』}}) ​​CREATE (a)-[:REL_TYPE {{ property1: 『value1』,...}}]->(b)`. ## 3. Guidelines for Nodes and Relationships Use consistent labels for nodes of similar types (for example, "Person" for all people entities). **Meaningful Relationships**: Create specific, descriptive, and context-appropriate relationships between nodes. Avoid generic types like "is part of" or "includes." Interpret the context of the entities and define relationships that accurately reflect their interactions and connections. For example, use a relationship type like "WORKS_FOR" instead of "ASSOCIATED_WITH." ## 4. Handling Numerical Data and Dates Numerical data, such as age and other relevant information, should be incorporated as attributes or properties of each node. **Do not create separate nodes for dates / numbers:** Do not create separate nodes for dates or numbers. Always attach them as attributes or properties of a node. **Property Format**: Properties must be in key-value format. **Quotation Marks**: Escaped single or double quotation marks must not be used within property values. **Naming conventions**: For example, `birthdate`. ## 5. Cross-reference resolution **Maintaining Entity Consistency**: Ensuring consistency is crucial when extracting entities. If an entity (e.g., "John Doe") is mentioned multiple times in the text but referred to by different names or pronouns (e.g., "Joe" or "he"), Throughout the entire knowledge graph, always use the most complete identifier for that entity. In this example, we use "John Doe" as the entity ID. Remember that maintaining consistency in entity references is crucial because knowledge graphs must be consistent and easy to understand. ## 6. Hierarchical structure Verify that the entities are structured hierarchically. Use relationships like "IS_PART_OF" or "BELONGS_TO" to establish clear parent-child relationships. ## 7. Complete Connectivity Ensure that all nodes are connected to at least one other node in the graph. If an entity appears isolated, consider its potential relationships with existing entities.

[0047] Multiple collaborative multi-agent LLMs provide feedback to each other via a text-based interface, evaluating each other's output based on criteria such as completeness, accuracy, logical correctness, and intent. If the output is unsatisfactory, the LLMs in the collaborative multi-agent LLMs provide specific feedback highlighting areas for improvement. Each LLM in the collaborative multi-agent LLMs acts as both a task processor and evaluator (feedback provider) for the remaining LLMs in the collaborative multi-agent LLMs. This iterative process continues until the output meets quality criteria and the collaborative multi-agent LLMs are satisfied with the results.

[0048] Once a unified knowledge graph is generated, it is queried through the knowledge graph query agent by the following: (i) Receive one or more queries related to multiple multimodal documents from one or more users. (ii) Construct a unified knowledge graph query agent by defining multiple knowledge graph query behavior parameters and multiple knowledge graph query function specification parameters. The knowledge graph query agent is a customized model of the knowledge graph query agent. Multiple knowledge graph query behavior parameters include domain model generation roles, domain knowledge, and domain model generation interfaces. Multiple knowledge graph query function specification parameters include multiple domain model generation tasks, including multiple domain model generation actions, multiple knowledge graph query tasks, and multiple knowledge graph evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple domain model generation tools. (iii) Create a knowledge graph query template using multiple instructions, a schema model, one or more queries, and a unified knowledge graph agent. (iv) Generate knowledge graph query prompts from knowledge graph query prompt templates. (v) Provide the Knowledge Graph Query Agent with a Knowledge Graph Query Prompt in order to generate Cipher queries for the Knowledge Graph Engine. (vi) The knowledge graph engine generates a response consisting of graph data retrieved from a unified knowledge graph using pattern matching. (vii) Process the responses from the knowledge graph engine along with one or more queries and prompts to convert the graph data into human-readable natural language and use a Large Language Model (LLM) to provide answers to one or more queries.

[0049] The creation of knowledge graph query templates according to some embodiments of this disclosure is as follows: "Human: Task: Task: Generate a Cypher statement to execute a query on a graph database." Instructions: Use only the relationship types and properties provided within the schema. Do not use any other relationship types or properties that are not provided. schema {schema} Note: Do not include explanations or apologies in your response. Do not answer questions that ask anything other than how to create a Cypher statement. Do not include any text other than the generated Cypher statement. When asked about the incident timeline, ensure that your cipher query is configured with a date data type when writing your query. The question is {question}.

[0050] Experimental results For the experiment, we utilized the publicly available technical report "Operations & Maintenance Best Practices: A Guide to Achieving Operational Efficiency" published by Pacific Northwest National Laboratory. This chapter focuses specifically on boilers, describing their types, components, maintenance, and efficiency. This chapter, originally in PDF format, was converted to a text file while maintaining the document's structure. Importantly, while the experiment focused on boiler maintenance, the multi-agent collaborative LLM is highly flexible and designed to be applicable to any industry or domain. By customizing the query generation agent and coordinating the domain knowledge of the remaining LLMs in the multi-agent collaborative LLM, the proposed method can be applied to diverse fields such as healthcare, finance, manufacturing, or other areas requiring structured knowledge extraction from technical documentation. The disclosed method is capable of generating a comprehensive, hierarchical, unified knowledge graph that accurately represents complex domains, such as those described in multiple multimodal documents, and is adaptable to a wide range of industries and applications.

[0051] Quantitative results Knowledge graph statistics: Number of entities extracted: 112 entities (e.g., AirHeater, Boiler, Burner, CableManagement, EfficiencyImprovement, WaterTubeBoiler, Document, Deaerators, DeficientAir, DiagnosticTools, etc.); Number of relationships identified: 18 relationships (e.g., AFFECTS, HAS_COMPONENT, SUPPORTS, MEASURES, IMPLEMENTED_IN, REDUCES, USE, BETTER_THAN, COMMONLY_ASSOCIATED_WITH, CRITICAL_FOR, etc.).

[0052] Question Generation Metrics: Total number of questions generated by the query generation agent: The query generation agent successfully generated a total of 100 queries designed to investigate various aspects of boiler operation and efficiency. The total number of questions can be adjusted based on user requirements. Some of the generated queries are as follows: Maintenance: Examples: "What equipment is used to determine combustion efficiency in a boiler?" and "How can scaling be prevented in a boiler?". Safety: Examples: "What could be the consequences of insufficient air in the boiler?", "What are the safety implications of improper boiler maintenance?". Troubleshooting examples: "What are the main problems related to heat recovery from exhaust gases?", "How can blowdown energy be recovered in the boiler?". Other topics include: "What is cogeneration in boiler operation?" and "How does the vertical temperature in a boiler room indicate air stratification?"

[0053] Comparative Analysis: Comparison with Baseline Method

[0054] Traditional rule-based information extraction: Rule-based methods rely on manually created rules to extract entities and relationships from text. While effective in specific, narrow domains, these approaches often lack scalability and are inadequate for diverse datasets.

[0055] Single LLM Approaches Without Multi-Agent Frameworks: Large-Scale Language Models (LLMs) such as GPT-3 and ChatGLM have shown promise in natural language processing tasks. However, attempts to apply single LLM systems to knowledge graph construction (KGC) have observed significant limitations in generating meaningful relationships and schema alignment. Single-agent approaches often struggle with complex multi-stage reasoning and fail to achieve optimal performance in extracting and cohering relationships between entities. This highlights the superiority of multiple collaborative multi-agent LLMs over single LLMs by enhancing knowledge selection, modification, and aggregation through agent synergies.

[0056] Comparison with GraphRAG: Neo4j Ecosystem Tools: Objective: To evaluate and compare the performance of multiple collaborative multi-agent LLMs against Neo4j's GraphRAG tool from the perspective of the number of nodes created in the unified knowledge graph, relationships, and other relevant metrics. Table 1 shows a comparison of multiple collaborative multi-agent LLMs against Neo4j's GraphRAG tool. Neo4j's GraphRAG ecosystem tools enhance GenAI applications by integrating knowledge graphs and search-aided generation (RAG). These open-source tools address issues such as hallucinations and lack of domain-specific context by combining structured and semi-structured data. This improves response quality and accelerates application development. JPEG2026062517000003.jpg226150 JPEG2026062517000004.jpg54164

[0057] The following JSON snippet shows the initial hierarchical structure of the domain model for the "Boilers" entity. At this stage, attributes such as boiler horsepower and maximum pressure are placeholders awaiting further input from the domain model populator agent. { "Boilers": { "description": "Fuel-burning appliances that produce either hot water or steam for heating or process uses.", "attributes": {}, "sub-entities": { "Types of Boilers": { "description": "Classifications of boiler designs.", "attributes": {}, "sub-entities": { "Fire-Tube Boilers": { "description": "Boilers where hot gases destroyed through tubes submerged in water.", "attributes": { "Boiler Horsepower": "", "Maximum pressure": "" }, "relationships": { "Efficiency Institute": { "type": "Reprinted with permission of", "details": "The Boiler Efficiency Institute, Auburn, Alabama" } } },

[0058] The following JSON snippet shows the input data for "Fire-Tube Boilers," a subtype within the "Boilers" entity. {"Boilers": { "description": "Fuel-burning appliances that produce either hot water or steam for heating or process uses.", "attributes": {}, "sub-entities": { "Types of Boilers": { "description": "Classifications of boiler designs.", "attributes": {}, "sub-entities": { "Fire-Tube Boilers": { "description": "Boilers where hot gases destroyed through tubes submerged in water.", "attributes": { "Boiler Horsepower": "20 through 800 bhp", "Maximum Pressure": "150 psi" }, "relationships": { "Efficiency Institute": { "type": "Reprinted with permission of", "details": "The Boiler Efficiency Institute, Auburn, Alabama" } } },

[0059] Figure 4 shows a partial knowledge graph of a unified knowledge graph generated using multiple popularized domain models according to some embodiments of the present disclosure.

[0060] Experimental results demonstrate that the disclosed method generates more meaningful relationships between entities, improves the accuracy of query responses, and reduces the time spent on manual document reading. By incorporating domain-specific knowledge and creating a more interconnected, context-aware, unified knowledge graph, the disclosed method provides a more powerful tool for knowledge management tasks in data-intensive environments. The disclosed method highlights the potential of multiple collaborative multi-agent LLMs to transform information retrieval processes and points to further opportunities to refine knowledge graph generation, particularly when dealing with larger documents and more complex queries.

[0061] This specification describes the subject matter in such a way that any person skilled in the art can create and use embodiments. The scope of embodiments of the subject matter is defined by the claims and may include other modifications that would be useful to a person skilled in the art. Such other modifications are intended to be included in the claims if they have similar elements that do not differ from the language of the claims, or if they contain equivalent elements that do not differ substantially from the language of the claims.

[0062] The disclosed method presents a collaborative multi-agent knowledge graph (RAG) framework by utilizing multiple collaborative multi-agent LLMs specifically tailored to generate a unified knowledge graph from multiple multimodal data sources. Unlike traditional approaches that employ generalized collaborative frameworks, the disclosed method introduces customized agents for generating multiple queries, generating domain models, and inputting domain models. The populated multiple domain models are generated for each chunk of multiple chunks, enabling the incorporation of information tailored to the specific context of that chunk. As a result, the populated multiple domain models collectively enrich the unified knowledge graph, enhancing the capture of domain-specific insights rather than simply focusing on extracting entities, relationships, and events. Thus, a strategic emphasis on generating populated multiple domain models ensures a more comprehensive understanding of the underlying data and its nuances.

[0063] It should be understood that the scope of protection extends not only to such programs but also to computer-readable means containing messages. Such computer-readable storage means include program code means for implementing one or more steps of the method when the program is executed on a server or mobile device or any suitable programmable device. Hardware devices can be any type of programmable device, including, for example, any type of computer such as a server or a personal computer, or any combination thereof. Devices can also include hardware means such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or a combination of hardware and software means, such as an ASIC and an FPGA, or means such as at least one microprocessor and at least one memory having software processing components located therein. Thus, means can include both hardware and software means. Embodiments of the method described herein can be implemented in hardware and software. The device can also include software means. Alternatively, embodiments can be implemented using different hardware devices, for example, multiple CPUs.

[0064] Embodiments of this specification may consist of hardware and software elements. Software-implemented embodiments include, but are not limited to, firmware, resident software, and microcode. Functions performed by the various components described herein may be implemented by other components or combinations thereof. In this specification, a computer-usable medium or computer-readable medium may be any device capable of configuring, storing, communicating, propagating, or transporting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0065] The illustrated steps are set up to illustrate the illustrated exemplary embodiments, and it should be anticipated that the way in which specific functions are performed may change due to ongoing technological development. These examples are presented herein for illustrative purposes only and are not limiting. Furthermore, boundaries of functional units are arbitrarily defined herein for the convenience of explanation. Different boundaries may be defined as long as the specified functions and their relationships are adequately performed. Alternative forms (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to those skilled in the art based on the teachings contained herein. Such alternatives are within the scope of the disclosed embodiments. The terms “comprising,” “having,” “containing,” and “including,” as well as other similar forms, are intended to be semantically equivalent and open-ended, in that the item or word following any one of these terms does not mean that the item or word following any one of these terms is an exhaustive list of such items or words, nor does it mean that the item or word following any one of these terms is limited to or exclusive to the listed items or words. Furthermore, it should be noted that, as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural anaphora unless the context clearly indicates otherwise.

[0066] Furthermore, one or more computer-readable storage media may be used when carrying out embodiments consistent with the present disclosure. Computer-readable storage media refers to any type of physical memory capable of storing information or data that is readable by a processor. Thus, computer-readable storage media can store instructions for execution by one or more processors, including instructions for causing a processor to perform one or more steps consistent with the embodiments described herein. The term “computer-readable media” should be understood to include tangible objects, excluding carrier waves and transient signals, i.e., non-transient. Examples include random-access memory (RAM), read-only memory (ROM), volatile memory, non-volatile memory, hard drives, CD-ROMs, DVDs, flash drives, disks, and any other known physical storage media.

[0067] This disclosure and examples are merely illustrative, and the true scope of the disclosed embodiments shall be indicated by the appended claims. [Explanation of symbols]

[0068] 100 Systems 102 memory 104 Hardware Processors 106 I / O interfaces 108 Databases

Claims

1. A processor implementation method (300), Receiving multiple multimodal documents and multiple domain history queries related to the multiple multimodal documents via one or more hardware processors (302), Processing the multiple multimodal documents via the one or more hardware processors and identifying text and multiple images using a Python library (304), Chunking the text and the plurality of images into a plurality of chunks including a plurality of text chunks and a plurality of image chunks via the one or more hardware processors (306), (308) Generating a unified knowledge graph from the plurality of chunks using a plurality of cooperative multi-agent large-scale language models (LLMs) via the one or more hardware processors, wherein the plurality of cooperative multi-agent LLMs include a query generation agent, a domain model generation agent, a domain model populator agent, and a knowledge graph curator agent, and generating a unified knowledge graph. Includes, Generating the aforementioned unified knowledge graph means (i) generating a plurality of queries associated with each of the plurality of chunks via the query generation agent (308a), (a) Constructing a query agent by defining a plurality of query generation behavior parameters and a plurality of query generation function specification parameters via an input JavaScript Object Notation (JSON) model, wherein the plurality of query generation behavior parameters comprises a query generation role, agent knowledge enriched using a plurality of domain history queries, a query generation interface, and the plurality of query generation function specification parameters comprises a plurality of tasks including a plurality of query generation actions, a plurality of query generation tasks and a plurality of query generation evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of query generation tools. (b) Creating a query prompt template using multiple instructions, the multiple chunks, actions, and the query agent, (c) Generating a query prompt using the query prompt template, (d) Inputting the query prompt to the query generation agent in order to generate the plurality of queries associated with each of the plurality of chunks, (308a) generating multiple queries including, (ii) via the domain model generation agent, generate a domain model associated with each of the multiple chunks based on the multiple queries generated (308b), (iii) Populating the domain models generated using each chunk of the plurality of chunks in order to generate a populated domain model from a plurality of populated domain models for each chunk of the plurality of chunks via the domain model populator agent (308c), (iv) generating the unified knowledge graph associated with each of the multiple chunks using the multiple popular domain models via the knowledge graph curator agent (308d), including, Processor implementation method.

2. The generated unified knowledge graph is accessed via the knowledge graph query agent. (a) Receiving one or more queries from one or more users relating to the plurality of multimodal documents, (b) Constructing a unified knowledge graph query agent by defining a plurality of knowledge graph query behavior parameters and a plurality of knowledge graph query function specification parameters, wherein the plurality of knowledge graph query behavior parameters include a domain model generation role, domain knowledge, and a domain model generation interface, and the plurality of knowledge graph query function specification parameters include a plurality of domain model generation actions, a plurality of domain model generation tasks including a plurality of knowledge graph query tasks and a plurality of knowledge graph query evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of domain model generation tools, (c) Creating a knowledge graph query template using multiple instructions, schema models, one or more queries, and a unified knowledge graph query agent, (d) Generating a knowledge graph query prompt from the knowledge graph query prompt template, (e) Inputting the knowledge graph query prompt to the knowledge graph query agent in order to generate a cipher query for the knowledge graph engine, (f) The knowledge graph engine generates a response that includes graph data obtained from the unified knowledge graph using pattern matching, (g) Processing the response from the knowledge graph engine along with one or more queries and prompts, and converting the graph data into human-interpretable natural language using a Large Language Model (LLM), A processor implementation method according to claim 1, which is queried by

3. Through the domain model generation agent, based on the generated multiple queries, the domain model associated with each chunk of the multiple chunks is: (a) Constructing a domain model agent by defining a plurality of domain model generation behavior parameters and a plurality of domain model generation function specification parameters, wherein the plurality of domain model generation behavior parameters include a domain model generation role, domain knowledge, and a domain model generation interface, and the plurality of domain model generation function specification parameters include a plurality of domain model generation tasks including a plurality of domain model generation actions, a plurality of domain model generation query tasks, and a plurality of domain model generation evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of domain model generation tools, (b) Using the plurality of instructions, the plurality of queries associated with each chunk of the plurality of chunks, the action, and the domain model agent, create a domain model generation prompt template; (c) Generating a domain model generation prompt from the domain model generation prompt template, (d) Inputting the domain model generation prompt to the domain model generation agent in order to generate a domain model associated with each of the multiple chunks based on the multiple queries generated, A processor implementation method according to claim 1, which is generated by the method described above.

4. The generated domain model is populated using each of the multiple chunks via the domain model populator agent to generate a populated domain model for each of the multiple chunks, (a) Constructing a domain model populate agent by defining multiple domain model populate behavior parameters and multiple domain model populate function specification parameters, wherein the multiple domain model populate behavior parameters include domain model populate roles, domain knowledge, and domain model populate interfaces, and the domain model populate function specification parameters include multiple domain model populate actions, multiple domain model populate query tasks, and multiple domain model populate evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple domain model populate tools, and (b) Creating a domain model populator prompt template using the plurality of instructions, the plurality of chunks, the domain model, the action, and the domain model populate agent, (c) Generating a domain model populator prompt from the domain model populator prompt template, (d) Inputting the domain model populator prompt to the domain model populator agent and generating multiple populated domain models associated with each of the multiple chunks, The processor implementation method according to claim 1, including the method described in claim 1.

5. Through the knowledge graph curator agent, a unified knowledge graph associated with each of the multiple chunks is generated using the multiple popularized domain models. (a) Constructing a knowledge graph agent by defining multiple knowledge graph curator behavior parameters and multiple knowledge graph curator function specification parameters, wherein the multiple knowledge graph curator behavior parameters include knowledge graph curator roles, domain knowledge, and knowledge graph curator interfaces, and the multiple knowledge graph curator function specification parameters include multiple knowledge graph curator actions, multiple knowledge graph curator query tasks and multiple knowledge graph curator evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple knowledge graph curator tools. (b) Create a knowledge graph curator prompt template using instructions, multiple popularized domain models, actions, and knowledge graph agents, (c) Generating knowledge graph curator prompts from knowledge graph curator prompt templates, (d) Inputting the knowledge graph curator prompt to the knowledge graph curator agent and generating the unified knowledge graph associated with each of the multiple chunks, The processor implementation method according to claim 1, including the method described in claim 1.

6. System (100), A memory (102) for storing instructions; One or more communication interfaces (106) and; One or more hardware processors (104) connected to the memory (102) via the one or more communication interfaces (106), Equipped with, The one or more hardware processors (104) described above, in accordance with the instructions, Receiving multiple multimodal documents and multiple domain history questions related to those multiple multimodal documents, The process involves processing the aforementioned multiple multimodal documents and identifying text and multiple images using the Python library. Chunking the aforementioned text and the aforementioned multiple images into multiple chunks, which include multiple text chunks and multiple image chunks. The method involves generating a unified knowledge graph from multiple chunks using multiple collaborative multi-agent large-scale language models (LLMs), wherein the multiple collaborative multi-agent LLMs include a query generation agent, a domain model generation agent, a domain model populator agent, and a knowledge graph curator agent. It is configured to do the following: Generating the aforementioned unified knowledge graph means (i) generating a plurality of queries associated with each chunk of the plurality of chunks via the query generation agent, (a) Constructing a query agent by defining a plurality of query generation behavior parameters and a plurality of query generation function specification parameters via an input JavaScript Object Notation (JSON) model, wherein the plurality of query generation behavior parameters comprises a query generation role, agent knowledge enriched using a plurality of domain history queries, a query generation interface, and the plurality of query generation function specification parameters comprises a plurality of tasks including a plurality of query generation actions, a plurality of query generation tasks and a plurality of query generation evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of query generation tools. (b) Creating a query prompt template using multiple instructions, the multiple chunks, actions, and the query agent, (c) Generating a query prompt using the query prompt template, (d) Inputting the query prompt to the query generation agent in order to generate the plurality of queries associated with each of the plurality of chunks, This involves generating multiple queries, (ii) via the domain model generation agent, generate a domain model associated with each chunk of the plurality of chunks based on the plurality of generated queries, (iii) Populating the domain models generated using each chunk of the plurality of chunks in order to generate a populated domain model from a plurality of populated domain models for each chunk of the plurality of chunks via the domain model populator agent, (iv) via the knowledge graph curator agent, generate the unified knowledge graph associated with each of the multiple chunks using the multiple popular domain models, A system that includes this.

7. The generated unified knowledge graph is accessed via the knowledge graph query agent. (a) Receiving one or more queries from one or more users relating to the plurality of multimodal documents, (b) Constructing a unified knowledge graph query agent by defining a plurality of knowledge graph query behavior parameters and a plurality of knowledge graph query function specification parameters, wherein the plurality of knowledge graph query behavior parameters include a domain model generation role, domain knowledge, and a domain model generation interface, and the plurality of knowledge graph query function specification parameters include a plurality of domain model generation actions, a plurality of domain model generation tasks including a plurality of knowledge graph query tasks and a plurality of knowledge graph query evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of domain model generation tools, (c) Creating a knowledge graph query template using multiple instructions, schema models, one or more queries, and a unified knowledge graph query agent, (d) Generating a knowledge graph query prompt from the knowledge graph query prompt template, (e) Inputting the knowledge graph query prompt to the knowledge graph query agent in order to generate a cipher query for the knowledge graph engine, (f) The knowledge graph engine generates a response that includes graph data obtained from the unified knowledge graph using pattern matching, The response from the knowledge graph engine is processed along with one or more queries and prompts, and the graph data is converted into human-interpretable natural language using a Large Language Model (LLM). Queryed by, The system according to claim 6.

8. Through the domain model generation agent, based on the generated multiple queries, the domain model associated with each chunk of the multiple chunks is: (a) Constructing a domain model agent by defining the plurality of domain model generation behavior parameters and the plurality of domain model generation function specification parameters, wherein the plurality of domain model generation behavior parameters include a domain model generation role, domain knowledge, and a domain model generation interface, and the plurality of domain model generation function specification parameters include a plurality of domain model generation tasks including a plurality of domain model generation actions, a plurality of domain model generation query tasks, and a plurality of domain model generation evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of domain model generation tools, (b) Using the plurality of instructions, the plurality of queries associated with each chunk of the plurality of chunks, the action, and the domain model agent, create a domain model generation prompt template; (c) Generating a domain model generation prompt from the domain model generation prompt template, (d) Inputting the domain model generation prompt to the domain model generation agent in order to generate a domain model associated with each of the multiple chunks based on the multiple queries generated, Generated by The system according to claim 6.

9. The generated domain model is populated using each of the multiple chunks via the domain model populator agent to generate a populated domain model for each of the multiple chunks, (a) Constructing a domain model populate agent by defining multiple domain model populate behavior parameters and multiple domain model populate function specification parameters, wherein the multiple domain model populate behavior parameters include domain model populate roles, domain knowledge, and domain model populate interfaces, and the domain model populate function specification parameters include multiple domain model populate actions, multiple domain model populate query tasks, and multiple domain model populate evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple domain model populate tools, and (b) Creating a domain model populator prompt template using the plurality of instructions, the plurality of chunks, the domain model, the action, and the domain model populate agent, (c) Generating a domain model populator prompt from the domain model populator prompt template, (d) Inputting the domain model populator prompt to the domain model populator agent and generating multiple populated domain models associated with each of the multiple chunks, including, The system according to claim 6.

10. Through the knowledge graph curator agent, a unified knowledge graph associated with each of the multiple chunks is generated using the multiple popularized domain models. (a) Constructing a knowledge graph agent by defining multiple knowledge graph curator behavior parameters and multiple knowledge graph curator function specification parameters, wherein the multiple knowledge graph curator behavior parameters include knowledge graph curator roles, domain knowledge, and knowledge graph curator interfaces, and the multiple knowledge graph curator function specification parameters include multiple knowledge graph curator actions, multiple knowledge graph curator query tasks, and multiple knowledge graph curator evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple knowledge graph curator generation tools, (b) Create a knowledge graph curator prompt template using instructions, multiple popularized domain models, actions, and knowledge graph agents, (c) Generating knowledge graph curator prompts from knowledge graph curator prompt templates, (d) Inputting the knowledge graph curator prompt to the knowledge graph curator agent and generating the unified knowledge graph associated with each of the multiple chunks, including, The system according to claim 6.

11. One or more non-temporary machine-readable information storage media containing one or more instructions, wherein when the one or more instructions are executed by one or more hardware processors, Receiving multiple multimodal documents and multiple domain history questions related to those multiple multimodal documents, The process involves processing the aforementioned multiple multimodal documents and identifying text and multiple images using the Python library. Chunking the aforementioned text and the aforementioned multiple images into multiple chunks, which include multiple text chunks and multiple image chunks. The method involves generating a unified knowledge graph from multiple chunks using multiple collaborative multi-agent large-scale language models (LLMs), wherein the multiple collaborative multi-agent LLMs include a query generation agent, a domain model generation agent, a domain model populator agent, and a knowledge graph curator agent. Have them do it, Generating the aforementioned unified knowledge graph means (i) generating a plurality of queries associated with each chunk of the plurality of chunks via the query generation agent, (a) Constructing a query agent by defining a plurality of query generation behavior parameters and a plurality of query generation function specification parameters via an input JavaScript Object Notation (JSON) model, wherein the plurality of query generation behavior parameters comprises a query generation role, agent knowledge enriched using a plurality of domain history queries, a query generation interface, and the plurality of query generation function specification parameters comprises a plurality of tasks including a plurality of query generation actions, a plurality of query generation tasks and a plurality of query generation evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of query generation tools. (b) Creating a query prompt template using multiple instructions, the multiple chunks, actions, and the query agent, (c) Generating a query prompt using the query prompt template, (d) Inputting the query prompt to the query generation agent in order to generate the plurality of queries associated with each of the plurality of chunks, This involves generating multiple queries, (ii) via the domain model generation agent, generate a domain model associated with each chunk of the plurality of chunks based on the plurality of generated queries, (iii) Populating the domain models generated using each chunk of the plurality of chunks in order to generate a populated domain model from a plurality of populated domain models for each chunk of the plurality of chunks via the domain model populator agent, (iv) via the knowledge graph curator agent, generate the unified knowledge graph associated with each of the multiple chunks using the multiple popular domain models, including, One or more non-temporary machine-readable information storage media.

12. The generated unified knowledge graph is accessed via the knowledge graph query agent. (a) Receiving one or more queries from one or more users relating to the plurality of multimodal documents, (b) Constructing a unified knowledge graph query agent by defining a plurality of knowledge graph query behavior parameters and a plurality of knowledge graph query function specification parameters, wherein the plurality of knowledge graph query behavior parameters include a domain model generation role, domain knowledge, and a domain model generation interface, and the plurality of knowledge graph query function specification parameters include a plurality of domain model generation actions, a plurality of domain model generation tasks including a plurality of knowledge graph query tasks and a plurality of knowledge graph query evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of domain model generation tools, (c) Creating a knowledge graph query template using multiple instructions, schema models, one or more queries, and a unified knowledge graph query agent, (d) Generating a knowledge graph query prompt from the knowledge graph query prompt template, (e) Inputting the knowledge graph query prompt to the knowledge graph query agent in order to generate a cipher query for the knowledge graph engine, (f) The knowledge graph engine generates a response that includes graph data obtained from the unified knowledge graph using pattern matching, (g) Processing the response from the knowledge graph engine along with one or more queries and prompts, and converting the graph data into human-interpretable natural language using a Large Language Model (LLM), Queryed by, One or more non-temporary machine-readable information storage media according to claim 11.

13. Through the domain model generation agent, based on the generated multiple queries, the domain model associated with each chunk of the multiple chunks is: (a) Constructing a domain model agent by defining a plurality of domain model generation behavior parameters and a plurality of domain model generation function specification parameters, wherein the plurality of domain model generation behavior parameters include a domain model generation role, domain knowledge, and a domain model generation interface, and the plurality of domain model generation function specification parameters include a plurality of domain model generation tasks including a plurality of domain model generation actions, a plurality of domain model generation query tasks, and a plurality of domain model generation evaluation tasks, feedback from a plurality of collaborative multi-agent LLMs, and a plurality of domain model generation tools, (b) Using the plurality of instructions, the plurality of queries associated with each chunk of the plurality of chunks, the action, and the domain model agent, create a domain model generation prompt template; (c) Generating a domain model generation prompt from the domain model generation prompt template, (d) Inputting the domain model generation prompt to the domain model generation agent in order to generate a domain model associated with each of the multiple chunks based on the multiple queries generated, Generated by One or more non-temporary machine-readable information storage media according to claim 11.

14. The generated domain model is populated using each of the multiple chunks via the domain model populator agent to generate a populated domain model for each of the multiple chunks, (a) Constructing a domain model populate agent by defining multiple domain model populate behavior parameters and multiple domain model populate function specification parameters, wherein the multiple domain model populate behavior parameters include domain model populate roles, domain knowledge, and domain model populate interfaces, and the domain model populate function specification parameters include multiple domain model populate actions, multiple domain model populate query tasks, and multiple domain model populate evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple domain model populate tools, and (b) Creating a domain model populator prompt template using the plurality of instructions, the plurality of chunks, the domain model, the action, and the domain model populate agent, (c) Generating a domain model populator prompt from the domain model populator prompt template, (d) Inputting the domain model populator prompt to the domain model populator agent and generating multiple populated domain models associated with each of the multiple chunks, One or more non-temporary machine-readable information storage media according to claim 11, including the following:

15. Through the knowledge graph curator agent, a unified knowledge graph associated with each of the multiple chunks is generated using the multiple popularized domain models. (a) Constructing a knowledge graph agent by defining multiple knowledge graph curator behavior parameters and multiple knowledge graph curator function specification parameters, wherein the multiple knowledge graph curator behavior parameters include knowledge graph curator roles, domain knowledge, and knowledge graph curator interfaces, and the multiple knowledge graph curator function specification parameters include multiple knowledge graph curator actions, multiple knowledge graph curator query tasks and multiple knowledge graph curator evaluation tasks, feedback from multiple collaborative multi-agent LLMs, and multiple knowledge graph curator tools. (b) Create a knowledge graph curator prompt template using instructions, multiple popularized domain models, actions, and knowledge graph agents, (c) Generating knowledge graph curator prompts from knowledge graph curator prompt templates, (d) Inputting the knowledge graph curator prompt to the knowledge graph curator agent and generating the unified knowledge graph associated with each of the multiple chunks, One or more non-temporary machine-readable information storage media according to claim 11, including the following:

Images