System and method for hybrid artificial intelligence enhancement and optimization

Abstract

Examples are disclosed for validating and optimizing responses generated by a large language model (LLM). A multi-management method includes implementing a logical programming engine that parses a response as a verifiable logical predicate to detect a contradiction with a real-world knowledge base. The responses may also be tradeed for category verification by modeling the responses as mathematical radios using a category theory to identify semantic inconsistencies that may indicate false information associated with the responses. These responses may be modified and supplemented in view of the verification operation.

Description

[0001] Cross-reference to related applications This is a PCT patent application that claims the benefit of U.S. Provisional Application No. 63 / 539102, filed on September 19, 2023, which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure generally relates to artificial intelligence (AI) including large language models (LLM); and particularly to systems and methods for enhancing the capabilities of conversational AI systems to provide more accurate and reliable responses in enterprise settings across different industries. Background Technology

[0003] Large language models such as ChatGPT can sometimes produce erroneous or hallucinatory responses unsuitable for enterprise applications. These models generate answers probabilistically through massive statistical computations, providing human-like insights. However, the probabilistic nature of these models leads to insufficient accuracy in their outputs. Furthermore, the lack of mechanisms to maintain realism allows these models to fabricate false information, and the augmentation model itself often fails to address such hallucination problems.

[0004] It is based on these observations and other considerations that the various aspects of this disclosure were conceived and developed. Summary of the Invention

[0005] This disclosure provides examples of systems and methods for enhancing the capabilities of conversational AI systems such as ChatGPT to provide more accurate and reliable responses in enterprise settings across diverse industries. In the context of the disclosed methods, apparatuses, techniques, devices, systems, etc., the terms “operable to,” “configured to,” and “capable” are used interchangeably.

[0006] In general, the innovations described in this article aim to enhance the capabilities of conversational AI systems such as ChatGPT to provide more accurate and reliable responses in enterprise settings across various industries, including: • Energy – Utilizing public grid data to provide outage diagnostics and recovery strategies. Planning renewable energy expansion based on generation assets.

[0007] • Finance – Providing personalized investment advice based on an individual's portfolio. Using knowledge of normal trading patterns to detect fraudulent transactions.

[0008] • Transportation – Consider factors such as weather, traffic, and fleet availability to suggest optimized logistics plans. Use geospatial databases to track assets.

[0009] Manufacturing – Interfaces with IoT sensors for predictive maintenance. Optimizes production scheduling based on demand forecasts and machine availability.

[0010] • Retail – We recommend interfaced with IoT sensors for predictive maintenance. Optimize production scheduling based on demand forecasts and machine availability.

[0011] The innovation is based on logical reasoning, mathematical optimization, and integration with enterprise data systems, which allows it to go beyond general conversational capabilities. It can provide tailored, validated solutions that are connected to the specific operational realities of different enterprises and industries.

[0012] In a first set of illustrative examples, the innovation may take the form of a method comprising the steps of: (a) accessing at least one response to a query from a large language model; (b) resolving the at least one response into a logical programming predicate, the logical programming predicate defining multiple facts and actions associated with the at least one response; (c) performing an accuracy (correctness) verification operation on the at least one response, given a knowledge base for logical programming, including: (c)(i) verifying and retaining facts from the multiple facts in the at least one response that are indicated as true via the knowledge base, and (c)(ii) removing any other facts from the multiple facts in the at least one response, the any other facts defining negations indicated as true in the knowledge base; and (d) generating an output that evaluates the at least one response and returns a verified version of the at least one response based on the accuracy verification operation.

[0013] The method may further include the following steps: determining that the knowledge base cannot verify multiple facts; and performing category verification, including: generating one or more hints derived from the query but semantically equivalent to the query; for each of the one or more hints, obtaining a corresponding response from a large language model; establishing a structural relationship between the generated hints and the corresponding response from the large language model; and using category theory to detect any semantic inconsistencies, wherein the identification of semantic inconsistencies reflects a lack of reliability indicating the association with the at least one response.

[0014] In a second set of illustrative examples, the innovation may take the form of a system comprising a processor that communicates with one or more computing devices implementing a Large Language Model (LLM). The processor parses LLM responses from the Large Language Model into logical programming predicates, verifies the logical programming predicates and thereby extends this to verifying the accuracy of the LLM responses, and performs category verification as described herein.

[0015] In the third set of illustrative examples, the innovation may take the form of computer-executable instructions stored in a non-transitory medium (memory), which can be executed by a processor to perform one or more of the operations implemented by the method, such as parsing a large language model response into logical programming predicates, verifying logical programming predicates and thereby extending to verifying the accuracy of the LLM response, and performing category verification as described herein.

[0016] The foregoing examples broadly outline various aspects, features, and technical advantages of the examples according to this disclosure in order to better understand the following detailed description. It should also be understood that the operations described above in the context of the illustrative example methods, apparatus, and computer-readable media are not essential and may exclude one or more operations and / or may include other additional operations discussed herein. Additional features and advantages will be described below. The illustrated and specific examples described herein can be readily utilized as the basis for modifying or designing other structures to achieve the same purpose of this disclosure. Such equivalent constructions do not depart from the spirit and scope of the appended claims. Attached Figure Description

[0017] Figure 1 This is an overall systems diagram associated with the inventive concepts described in this paper for enhancing the capabilities of conversational AI systems to provide more accurate and reliable responses.

[0018] Figure 2A This is a diagram of an example process and exemplary logic for verifying the accuracy of the response to an LLM triggered by a query.

[0019] Figure 2B It is used for Figure 2A The diagram illustrates an example process and exemplary logic for performing category verification on the response.

[0020] Figure 3 yes Figure 1 The system and Figure 2A-2B Additional example diagrams of the operation's architecture.

[0021] Figure 4 Is with Figure 1 The system and Figure 2A-2B The diagram illustrates the example data flow associated with the operation.

[0022] Figure 5 This is an illustration of an example aspect associated with the multi-agent solver operation used to complement the LLM response described herein.

[0023] Figure 6 This is a diagram of the IEEE node test feeder as described in this article.

[0024] Figure 7This is a simplified block diagram of an example computing device that can be implemented based on the inventive concept described herein.

[0025] The corresponding reference numerals in the various views of the accompanying drawings indicate the corresponding elements. The headings used in the drawings do not limit the scope of the claims. Detailed Implementation

[0026] The inventive concept described in this paper relates to examples of systems and methods for enhancing the ability of conversational AI systems to provide more accurate and reliable responses. In particular, this innovation uniquely integrates logical programming representing truth and falsehood with a large language model. The operation of the logic engine is defined using an enterprise knowledge base encoded in logical programming to filter out incorrect responses. The inventive concept may also include logic for generating optimal solutions that match user queries by combining model outputs with enterprise knowledge. An example software architecture named GPTProX parses model responses into logical programming predicates. Note that a Horn clause can represent first-order logic; therefore, an LLM response can be parsed as a predicate used as an item within a Horn clause. The logical programming predicates are validated against the knowledge base to eliminate illusions. By fusing AI with mathematical optimization, this innovation generates accurate, customized responses based on reality. This principled approach promises to overcome inaccuracies through structured knowledge representation and reasoning, thereby unlocking enterprise applications for large language models.

[0027] In some examples, GPTProX includes a new software system that combines chatbots like ChatGPT with logic programming and mathematical optimization tools. It does this to provide more accurate and reliable answers for business use. GPTProX can extract ChatGPT's responses and examine them using logic programming (e.g., in Prolog). This helps filter out incorrect or fabricated answers from ChatGPT. Category-based validators can also provide additional checks for accuracy.

[0028] GPTProX can connect to various databases, such as Wikipedia, financial databases, and map databases. These databases can be leveraged to provide better answers based on real-world data. When asked to provide helpful advice, GPTProX can use templates for linear programming, dynamic programming, and optimal control, and these templates can be run in Matlab to yield optimal solutions.

[0029] ChatGPT can then be rewritten from solutions derived from traditional LLMs to make them more understandable. GPTProX can also use ChatGPT, along with validated facts and a multi-agent mathematical solver, to make fully automated choices. This allows for real-time automation using both dialogue and structured business data.

[0030] Overall, GPTProX's examples combine chatbots, logical reasoning, mathematical optimization, and databases to provide companies with customizable and accurate technical solutions to overcome the challenges of accuracy in handling currently available LLM and similar platforms.

[0031] Introduction and Technical Issues Today's business IT systems, even those equipped with AI models, are limited in terms of performance and the types of tasks they can perform. While LLMs such as ChatGPT and other chatbots are less limited in this respect, they may produce incorrect or hypothetical answers and lack real-world business knowledge. Generally, LLMs suffer from known technical problems associated with the accuracy of responses triggered by a variety of queries.

[0032] Example technical solution ("GPTProx") An example of the inventive concept, sometimes referred to herein as "GPTProX," addresses the aforementioned technical problems by integrating Chat-GPT with a real-world company system. Logic can be programmed and configured to check ChatGPT answers against a knowledge base. This improves the answers to fit the company's context.

[0033] GPTProX can include an implementation of a dynamic knowledge base within Prolog, which connects to one or more databases, GIS systems, niche data sources, and Wikipedia based on a given end-user, objective, and application. GPTProX can automatically expand its knowledge base using inductive logic programming. When the answer remains uncertain, GPTProX can use category theory to capture inconsistencies, which may imply fabricated information.

[0034] GPTProX can break down the answer into facts and actions. For actions, it can mathematically formulate the problem using templates for linear programming, dynamic programming, and optimal control. Then, it can call a solver to find the optimal solution.

[0035] ChatGPT can further rewrite the optimal solution in clear language. This allows GPTProX to provide users with accurate, tailored answers that fit their business realities.

[0036] Therefore, in summary, GPTProX combines chatbots, logic, mathematical optimization, and live data to overcome limitations and inaccuracies. This enhances ChatGPT for use in real-world enterprise examples and significantly improves the accuracy of outputs derived from LLM models.

[0037] Example Implementation Figure 1-5Example implementations of operations and components for enhancing the capabilities of conversational AI systems to provide more accurate and reliable responses are shown. As indicated, the inventive concept in response to the foregoing technical problems and challenges can take the form of a computer-implemented system (referred to as System 100), comprising any number of computing devices or processing elements. Generally, System 100 utilizes logic programming and Large Language Models (LLMs) to validate and enhance responses to queries posed to the LLM. While the inventive concept is primarily described as an implementation of a system, it should be understood that the inventive concept can also take the form of a tangible, non-transitory computer-readable medium (on which processor-executable instructions are encoded), and any number of methods associated with the examples of the systems described herein.

[0038] Figure 1 The illustration shows example system components associated with the inventive concepts described herein. For example... Figure 1 As indicated herein, system 100 includes at least one processor 102 and at least one of a memory 103 or storage device 103 storing instructions 104 accessible by processor 102 to perform various functions and operations described herein. System 100 verifies at least one response 132 generated by at least one large language model 130 and additionally provides output associated with response 132. System 100 may also include a network interface 106 (or multiple network interfaces) and a bus (or wireless medium) for establishing communication and / or interconnection between the aforementioned components. Network interface 106 includes mechanical, electrical, and signaling circuitry for transmitting data over links (e.g., wired or wireless links) within a network (e.g., the Internet). As those skilled in the art will understand, network interface 106 can be configured to transmit and / or receive data using a wide variety of different communication protocols.

[0039] Generally, processor 102 is configured (via instructions 104) to perform operations including verifying at least one response 132 generated by the at least one Large Language Model (LLM) 130 in response to at least one query 128 fed to or received by the LLM (by an end user or otherwise). The LLM 130 can be implemented via any number or type of computing elements, including cloud implementations, and examples of the LLM 130 are described herein. In various examples, processor 102 accesses (e.g., through interaction with a user interface (UI) 112 presented along display 110) input data 114A provided to external computing device 108, and processor 102 can return output (e.g., a modified and / or verified response to a query made to the LLM) in the form of output data 114B for access by computing device 108. Input data 114A may include response 132 or data associated with a response generated by the LLM 130, and other information required to perform the operations described herein. Output data 114B may include information about verification and enhancements associated with response 132. For example, output data 114B may include a modified response (e.g., some changes to the original response 132 generated by LLM 130), confirmation that response 132 has been verified and / or validated, and / or supplementary information or functionality used to enhance response 132 as described herein.

[0040] In some examples, processor 102 may access data from one or more data source devices 120 (e.g., devices 120A, 120B, and 120C). Device 120 may provide any datasets or information required to perform LLM response verification and / or LLM response enhancement operations or any other operations described herein. Datasets and other information may be preprocessed and stored in database 118, as shown. Any artificial intelligence (AI) models cited herein may include classification models, supervised or unsupervised learning models (such as K nearest neighbors), linear regression models, neural networks, deep learning models, etc.

[0041] Generally, instruction 104 can be implemented as machine-executable instructions and / or code executable by processor 102. These instructions and / or code can represent one or more of the following: procedures, functions, subroutines, programs, routines, subroutines, modules, objects, software packages, classes, or any combination of instructions, data structures, or program statements, etc. In other words, the instructions 104 or any operation described herein that is executable by processor 102 can be implemented in hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., a computer program product) that perform the necessary tasks can be stored in a computer-readable or machine-readable medium (e.g., memory 103), and processor 102 executes the tasks defined by the code.

[0042] Figure 2A Diagram and Figure 1 The system-related example process 200 or method implementation, and Figure 3-5 Diagrams and Procedures 200 and Figure 1 Further exemplary architecture and data flow aspects associated with the system components described herein. Referring to block 201 of Figure 2, processor 102 can access response 132 and / or data defining response 132 generated by LLM 130 via user interface 112 or other means. Processor 102 can also access query 128 and / or data defining query 128 provided to LLM 130; query 128 is the communication that causes response 132 to be raised from LLM 130.

[0043] In some examples, processor 102 may be provided with or otherwise access the data defining the response 132 generated by LLM 130 without communicating with LLM 130. Alternatively, processor 102 may directly receive query 128 and / or may directly receive response 132 from LLM 130, and may otherwise facilitate communication between the end user and LLM 130 to initiate response 132 and its modified or validated output. In some examples, query 128 may be defined as part of input data 114A, which may be sent to an application programming interface (API) wrapper (such as...) associated with LLM 130. Figure 3 (as indicated in the document) to trigger response 132. Any such changes have been considered, but the final processor 102 will at least access response 132 and query 128 to perform the validation and optimization functions described herein.

[0044] exist Figure 3In the specific (non-limiting) example shown, query 128 can be accessed or performed on user interface 112. Query 128 is any query, prompt, or other communication performed to elicit a response 132 from LLM 130. Sample queries are provided herein, but an example of query 128 could be as follows.

[0045] For example, this type of query could be performed via a chatbot associated with the LLM 130 (e.g., fed into ChatGPT). Although Figure 2A The query 128 originates from the user interface 112, but in other examples, the processor 102 may access the response 132 generated from the LLM 130, while the query 128 may be generated elsewhere.

[0046] This document provides a specific example of response 132 generated by LLM 130 or the output of query 128, but continuing with the above example, an example of response 132 associated with the above specific example query (“Logician Hao Wang Affiliations”) may include the following.

[0047] While the example response 132 provided above is text-based, other iterations of response 132 may include graphical images, numerical outputs and / or functions, or other output information as shown in other examples provided in this document.

[0048] Referring to box 202 in Figure 2, processor 102 parses response 132 into logical programming predicates, which define multiple facts and actions / rules associated with the response. In some examples, processor 102 may implement a Prolog verification engine or its functionality as defined by instruction 104 to parse response 132 into Prolog format in this way. Continuing with the example above, the above response can be parsed to generate the following logical programming predicates: As shown above, resolving response 132 into logical programming predicates (as described) transforms response 132 into output that defines the logical programming statements associated with response 132, including conjunctions that construct relationships between the logical programming statements.

[0049] Referring to box 203 in Figure 2, processor 102 may perform an accuracy verification operation on response 132 based on a knowledge base or otherwise to determine whether response 132 is valid and / or should be modified in some form. In some examples, this may be achieved by ( Figure 3As shown in the diagram and developed according to the examples described herein, a Prolog knowledge base is used to verify (as resolved) the facts defined in response 132. In some examples, accuracy verification includes: (i) verifying and retaining the facts in response 132 that are indicated to be true via the knowledge base, and (ii) removing any other facts from the logic programming predicate response 132, whereby any other facts define negations that are indicated to be true in the knowledge base. More specifically, in some examples, if processor 102 determines, based on the knowledge base, that the negation of a predicate in a logic programming predicate is true, processor 102 executes logic to remove that predicate from the logic programming predicates associated with response 132. If the predicate is determined to be true in the knowledge base, processor 102 can execute logic to accept that predicate in the logic programming predicate. If both the fact of the logic programming predicate and its negation are considered false in the knowledge base, processor 102 can perform category verification, as further described herein. Furthermore, for action predicates of logic programming predicates, processor 102 can implement a multi-agent optimization solver to obtain further functionality, such as... Figure 5 As shown and further described herein.

[0050] Continuing with the example above associated with query 128 (“Logician Hao Wang Affiliations”), processor 102, which performs accuracy verification based on the knowledge base as described, determines the following: Specifically, as shown, the proposition defined by the predicate and the associated facts from the example in response 132, asserting that Hao Wang was from or affiliated with the University of Notre Dame and was a faculty member in 1950, is determined to be false by processor 102 in view of the knowledge base referencing the predicate. In some examples, this determination by processor 102 includes performing a "consultation" operation on the Prolog file of the knowledge base, which is at least partially created using Wikipedia or some other data source.

[0051] Referring to box 204 in Figure 2, processor 102 generates output based on an accuracy verification operation, evaluating the at least one response and returning a verified version of the at least one response. In some examples, when multiple facts are all determined to be true and valid through a reference knowledge base, the output may include simple verification confirmation. However, in other examples, such as the example above handling query 128 "Logician Hao Wang Affiliations" and associated response 132, the output may include... already ReviseA response or a modified version of response 132. For example, a modified version of response 132 presented back to the end-user device or otherwise may take the following form or a similar form: Note that the language indicated by strikethrough above can be omitted in the modified response because the processor 102 has determined, based on the knowledge base (by comparing the associated predicate with the data in the knowledge base), that such information is incorrect or unverifiable.

[0052] refer to Figure 2B The illustration shows example process 250, which provides a process that can be configured by processor 102 and... Figure 2A Further verification operations are implemented in accordance with the operations shown. In block 251 of process 250, processor 102 first determines from Figure 2A Multiple facts extracted from the parsing operation cannot be verified or confirmed by the knowledge base, for whatever reason.

[0053] Referring to reference boxes 252-255, processor 102 can perform category validation to capture inconsistencies associated with response 132 that may indicate that response 132 consists at least partially of fabricated information. In other words, the category theory component examines these responses by modeling them from LLM 130 as category objects mapped from input prompts / queries. Deviations in the mapping make inconsistencies detectable.

[0054] In some examples, the category verification operation that can be performed by processor 102 includes generating one or more hints that are modified from query 128 but are semantically equivalent to query 128, obtaining a corresponding response from large language model 130 for each of the one or more hints, establishing a structural relationship between the generated hints and the corresponding responses from large language model 130, and using category theory to detect any semantic inconsistencies, wherein the identification of semantic inconsistencies reflects a lack of indication of reliability associated with said at least one response.

[0055] Figure 3-5 The diagram can be Figure 2A-2B Further non-limiting steps (1-35) achieved through operation or other means are as follows: 1. ① User Interface. Processor 102 can extract user input in the form of English text or spoken English. When the user completes the input, the input can be translated or assembled into English. The input can then be sent from processor 102 to (Large Language Model) LLM 130, which may include API wrappers.

[0056] 2. ② A Prolog validation engine can be implemented to parse responses from LLMs (such as ChatGPT responses) into Prolog format (defined logic programming predicates). This step demonstrates the use of a Prolog knowledge base to validate correctness. There are two different types of Prolog statements in the response: facts and actions. For facts, validation is performed using the Prolog knowledge base. If the negation of a predicate is true in the knowledge base, the predicate is removed. If the predicate is true in the knowledge base, it is accepted. For facts and their negations that are both false in the knowledge base, they are sent to the category validation engine for further processing. For action predicates, they are sent to the multi-agent optimization solver for detailed actions.

[0057] 3. ③ The category validation engine can be used for category filtering and detection of potential illusions. If an illusion is detected, this step includes notifying the Prolog validation engine to perform query realignment.

[0058] 4. ④ Multi-agent optimization solver: This feature allows for the use of a Prolog knowledge base to store and reference domain-specific topics, and further enables the maintenance of corresponding problem formulation templates to support automatic solution generation (e.g., a multi-agent solver for supplementing response 132). Available solvers provided by the Matlab toolbox can be installed. The underlying problem can be decomposed into modal problems solvable by agents. Joint multi-agent solutions can be obtained by sharing the constraints of each agent's solution. Each agent's solution can be a solution to an optimization problem.

[0059] 5. ⑤ Prolog Knowledge Integrator, consisting of two parts: 1. A Prolog DB built using Instruction-Level Parallelism (ILP). 2. Dynamic querying of Wikipedia and other enterprise databases.

[0060] 6. ⑥ Interface to Wikipedia: This allows for the combination of HTML operations to obtain requests from Wikipedia for the underlying topic. The response can be parsed into Prolog format by a parser. Subsequent steps may include updating the Prolog knowledge base using the parsed Wikipedia response.

[0061] 7. ⑦ Interface to the enterprise business database: Query the enterprise database to obtain facts. For example, retrieve the CEO's name.

[0062] 8. ⑧ Interface to Enterprise Utility GIS: Manage geographic queries to obtain the location and geometric attributes of assets. Parse the results into Prolog format for loading into the Prolog knowledge base.

[0063] 9. ⑨ Interface to the Enterprise Ontology Database: Query the ontology database to obtain domain-specific definitions of the underlying terms and their equivalents. Parse the results into Prolog format and accept the file as Prolog facts and rules.

[0064] 10. ⑩ LLM (Large Language Model) wrapper. A package that maintains the same interface as GPTProX. It can be customized for all available LLMs.

[0065] 11. ⑳ A domain-specific engine based on Prolog. It is responsible for two things: fact verification and action completion.

[0066] 12. ㉑ Multi-agent optimization solver.

[0067] 13. 29. LLM packager.

[0068] 14. 23 User input service.

[0069] 15. Initial User Prompt.

[0070] 16. ㉛ Obtain the user prompt to ㉒.

[0071] 17. ㉜LLM responded to ⑳.

[0072] 18. The prompts adjusted based on the process of step ⑳ are sent to step ≒ for iteration.

[0073] 19. The LLM response of ㉞ to ㉒ in the iteration.

[0074] 20. Finally, the response is sent to the user.

[0075] Relationships ① and ⑩: ① extracts user input to prepare it as an LLM (Large Language Model) query. ① sends the query to ⑩. The message travels unidirectionally from ① to ⑩.

[0076] Relationships ② and ③: ② sends query and response pairs to ③ for category-based exploration. ③ sends a "passed" or "failed" category validation result to ②.

[0077] Relationships ② and ④: ② sends the attributes and specifications of the solution to be obtained. ④ sends the solution of the underlying topic to ②.

[0078] Relationships ② and ⑤: ② sends facts to ⑤ for verification and to obtain additional information about those facts. ⑤ sends a conclusion that the facts are true or false. If the facts are true, any additional information the system can obtain will also be sent back to ②.

[0079] Relations ⑤ and ⑥: ⑤ sends a query to ⑥ for fact retrieval. ⑤ parses the response into Prolog format and loads it as a true fact into the Prolog knowledge base.

[0080] Relationships ⑤ and ⑦: ⑤ sends a query to ⑦ to retrieve facts and rules for the enterprise. ⑤ parses the response into Prolog format and loads it into Prolog as facts and rules.

[0081] Relationships ⑤ and ⑧: ⑤ sends a geospatial query to ⑧ to obtain asset location, asset connectivity, optimal routing, etc. ⑧ parses the response into Prolog format and loads the results into the Prolog knowledge base.

[0082] Relationships ⑤ and ⑨: ⑤ sends a query to ⑨ to retrieve domain-specific definitions of the term and its equivalents. ⑤ parses the results into Prolog format to load the Prolog knowledge base.

[0083] Relationships ① and ⑩: ① sends language prompts (queries) to ⑩.

[0084] Relationship ② and ⑩: ② sends a language prompt (query) to ⑩. ② receives a prompt response from ⑩ (queried by ① or ②).

[0085] Relationships ③ and ⑩: ③ sends a language prompt and receives a response to ① for category validation run.

[0086] In ②: If the response message is acknowledged by the Prolog knowledge base in terms of facts and is coordinated with ④ in terms of action, then the message is sent to ①. If a predicate parsed from the response message requires verification, and the Prolog knowledge base cannot accept or refute it, then the predicate is sent to ⑤ for verification or fact retrieval. If the action predicate is an action, then send it to ④ for optimization. If a fact predicate cannot be accepted or refuted by the Prolog knowledge base and ⑤, it is sent to ③ for category validation.

[0087] In ③: If the category verification iteration (between ③ and ④) has not yet been completed, then continue the iteration between ③ and ④. If the iteration is complete, send the result to ②.

[0088] In ⑤: If the predicate set involves individuals or organizations, then send a query to ⑥ to complete it. If the predicate refers to an individual within this enterprise, then send a query to ⑦ to complete the knowledge. If the predicate involves an asset with a location, then send a query to ⑧ to complete it. If the predicate is an unknown concept, send a query to ⑨ to obtain its definition and its equivalent name.

[0089] In ③: If it is in the category validation loop, the message is sent to 10. If validation is complete, the result is sent to ②.

[0090] Building a Prolog knowledge base The following non-restrictive / example methods can be used for the initial construction and continuous enhancement of the knowledge base described earlier. The knowledge base can incorporate manually coded and automatically extracted knowledge via web scraping, APIs, etc.

[0091] Domain experts codify key regulations, policies, and principles into logical programming rules in Prolog. This establishes a foundational set of verified facts covering key governance constraints.

[0092] Then, inductive logic programming techniques are used to learn new rules and facts from structured enterprise data sources. The system actively interfaces with databases such as geographic information systems and financial systems to extract knowledge through inductive learning. The interface code enables bidirectional communication and supports robust fact checking.

[0093] Furthermore, Monte Carlo simulations model the behavior of the ChatGPT response for the target domain. By analyzing response patterns and characteristics across numerous simulation iterations, the system derives supplementary rules to further enhance the Prolog knowledge base.

[0094] This hybrid approach, combining manual coding, inductive machine learning, and simulation-based discovery, allows for the efficient development of comprehensive and validated knowledge bases. Continuous integration with on-site enterprise data and the logic extracted from observations of conversational models ensure that knowledge remains relevant and complete as the application environment evolves.

[0095] An English-to-Prolog parser built using SpaCy and ontology Using SpaCy (an open-source natural language processing library) along with an ontology database, it's possible to build an English text-to-Prolog parser. This combination can help resolve ambiguities and domain-specific nuances. The general steps include: • Develop an ontology database with structured knowledge about the domain (including concepts, relations, and attributes).

[0096] • Use SpaCy to parse text into tokens, named entities, part-of-speech tags, etc. Customize it based on ontology to identify domain entities.

[0097] • Based on the ontology structure and the target Prolog knowledge base format, the parsed text is converted into Prolog predicates.

[0098] • When translating into Prolog, handle negation, modifiers, and other complex language structures.

[0099] • Iteratively improve the parser by analyzing ChatGPT interactions and expanding the ontology.

[0100] • Implement a user feedback loop to improve the parser and knowledge base over time.

[0101] The combination of ontology and SpaCy provides a solid foundation. Focused domain knowledge and continuous improvement will allow the system to parse English into the target Prolog representation and continuously improve accuracy.

[0102] Building a Fact-Checking System for ChatGPT Using NLP and Logic Programming To develop a system that cross-references ChatGPT answers with reliable knowledge sources, the following process can be implemented: 1. Using SpaCy for text parsing: See the section “Building an English-to-Prolog Parser Using SpaCy and Ontologies” for details.

[0103] 2. Convert to Prolog structure: • Convert the parsed data into Prolog fact and rule format.

[0104] • Convert subject-verb-object conjunctions into Prolog predicates with specified arguments.

[0105] • Convert the identified named entities into Prolog constants.

[0106] 3. Building a knowledge base in Prolog: • Draft rules for encapsulating ontology insights.

[0107] • Construct a Prolog database containing predetermined facts and regulations that represent credible information.

[0108] • Incorporate authoritative sources to establish facts about entities and their interrelationships.

[0109] • Collaborate with experts to develop domain-specific rules and principles.

[0110] • Regularly expand this database to achieve comprehensiveness.

[0111] 4. Verify ChatGPT assertions: • Probe the Prolog database to verify the ChatGPT statements.

[0112] • Design a query for each assertion to determine its validity.

[0113] • If a claim is refuted, it is classified as incorrect.

[0114] 5. TextBF Enhancement Measures: The hint is: the branch of the IEEE 13 circuit. .

[0115] Local knowledge base content This is a Prolog program that describes the connections between nodes: The steps include: Store frequently used queries and their results to speed up processing.

[0116] • Where feasible, evaluate multiple assertions uniformly.

[0117] • Ensure robust error management during the parsing, translation, and querying phases.

[0118] • Catalog and carefully analyze the scenarios that lead to the discrepancies.

[0119] 6. TextBF is continuously being improved: • Strengthen each component to improve system resilience.

[0120] • Involve domain experts in the evaluation.

[0121] • Allow experts to verify the claims expressed after analysis.

[0122] • Utilize expert evaluation to improve the parser and knowledge base.

[0123] By following this guide, you can build a resilient fact-checking system for ChatGPT based on NLP and logic programming techniques. The essence lies in transforming linguistic content into logical entities to facilitate effective cross-referencing with authoritative data.

[0124] Analysis and verification example: IEEE 13 test feeder refer to Figure 6 To demonstrate how to integrate LLMs such as ChatGPT with engineering applications, the IEEE 13-node test feeder circuit, originally published in a report by the IEEE Distribution Systems Analysis Subcommittee, can be used.

[0125] ChatGPT's response to the IEEE 13 test feeder ChatGPT Tip: IEEE 13 test feeder, including connectivity using tributaries. ChatGPT Response: The IEEE 13-bus test feeder is a highly unbalanced network commonly used to test new power system algorithms. Its topology is typical of urban residential areas, and it is known for being simple yet complex enough to test various aspects of power system research.

[0126] The topology of the IEEE 13-node test feeder is shown below (using branches between nodes to illustrate): This branch list represents the power distribution lines connecting nodes in the IEEE 13-node test feeder. The system topology belongs to... In a typical urban residential area, some nodes have more than one power distribution line connected to them. .

[0127] The "consultation" directly uses the Prolog file created from the IEEE 13-node diagram, and the proposition... Category-based illusion detection 1. Scope (a) P (hint) i. Object: ChatGPT hint. ii. Morphology: A function that transforms one cue into another while preserving its semantics. (b) R (Response) i. Object: ChatGPT response. ii. Morphology: A function that transforms one response into another, such as paraphrasing or refining it, without losing the intended meaning.

[0128] 2. Functor F (a) Mapping the transformation suggested in category P to the resulting transformation in the response in category R. i. Object mapping: For each hint in P A F in R ( A ) is ChatGPT A The response. ii. Morphological mapping: for a transformation in P f : A→B, R F ( f ): F ( A → F ( B ) indicates the correlation transformation between responses.

[0129] 3. Monostructures in R (a) The “computational effect” on the encapsulation response: i. Self-functional T: Represents a computational process on the response, such as refinement or extension. ii. Natural transformation: A. Unit ( ηEmbed the response T In the context of the single. B. Multiplication ( µ ): Combining or simplifying the calculation process T Two consecutive applications.

[0130] 4. Operation process Semantic cue generation: Use morphisms in P to generate various semantically equivalent cues.

[0131] (b) Response collection: For each generated prompt, obtain the ChatGPT response.

[0132] (c) Mapping via functor F: Establishing a structural relationship between the generated prompts and the corresponding responses.

[0133] (d) Monomer processing: Using T, η and µ to introduce and manage computational effects on the response.

[0134] (e) Illusion Detection: After multiple iterations of the monad processing, check whether the categorical relations are still maintained in the response. If any relation deviates or breaks, the corresponding response is marked as a potential illusion.

[0135] (f) Human verification: Submit the tagged responses to human review to determine whether they are genuine hallucinations.

[0136] 5. Results (a) The system will provide a refined set of ChatGPT responses, in which potential illusions are either flagged or filtered out, ensuring a more reliable and trustworthy user experience.

[0137] Multi-agent solver This section explains how to formulate fundamental mathematical problems. It also describes three common templates for formulating the applied systems: linear programming, dynamic programming, and optimal control. In addition to the theoretical descriptions, example formulations using the Prolog programming language are provided. Once the Prolog verification engine determines that ChatGPT's response has proposed a feasible solution to the problem that can be solved by the GPTProX multi-agent solver, GPTProX will activate its own computational methods to compute and provide the actual solution. It is important to note that ChatGPT's proposed solutions to mathematical problems are based on language likelihood, not direct computation; therefore, only solutions computed by GPTProX can be trusted as correct.

[0138] We're using Matlab as the solver for this demonstration. Other packages can be used similarly.

[0139] Basic Mathematical Calculations This section provides an overview of the Prolog setup designed to identify mathematical operations and subsequently call MATLAB to solve them.

[0140] Problem description in Prolog We first set up Prolog settings to recognize basic mathematical operations: ·addition ·multiplication · derivative ·integral Recognition is based on parsing and recognizing operations using the constant clause grammar (DCG) in Prolog.

[0141] Sample code: Prolog calling MATLAB Using Matlab solvers as examples is based on experience that they meet the necessary criteria. However, alternative solvers, including open-source solvers, can be employed to potentially reduce system costs.

[0142] Linear Programming (LP) Linear programming is used to solve problems characterized by linear objective functions and linear constraints. Common applications include resource allocation, scheduling, and many other optimization tasks.

[0143] MATLAB solver calls Given the Prolog representation of an LP problem, we need to convert it to MATLAB format and then call the 'linprog' function to solve it.

[0144] Dynamic Programming (DP) Dynamic programming operates by breaking down a problem into smaller subproblems. Each subproblem is solved only once, and its result is stored, thus avoiding redundant computation. DP is very useful for optimization problems whose solutions can be recursively decomposed.

[0145] MATLAB solver calls for dynamic programming Given a recursive approach in Prolog, translating this into a bottom-up iterative approach in MATLAB requires solving the basic cases first and then using their solutions to progressively solve the main problem.

[0146] Optimal control Optimal control aims to find control strategies that optimize specific performance metrics. It is particularly beneficial for systems spanning domains such as robotics and economics, as well as many physical systems.

[0147] MATLAB solver calls for optimal control Given a Prolog expression, converting this to a MATLAB nonlinear optimization problem requires setting up the problem. The state dynamics, constraints, and objective function must be properly captured in order to apply MATLAB's solver.

[0148] Multi-agent The GPTProX system utilizes a modular, agent-based architecture to solve complex optimization problems that require the integration of multiple technologies.

[0149] Each mathematical programming template (such as linear programming or optimal control) is modeled as an independent agent. These agents are capable of solving independent instances of their respective problem categories.

[0150] For situations requiring the combination of multiple optimization formulations, GPTProX coordinates these agents through constraint sharing. State and decision variables from one agent are encoded as constraints for other agents.

[0151] This allows modular agents to collaborate in order to find an integrated solution. These agents exchange constraints in multiple interactions, iteratively improving each local solution to converge to a unified global optimization.

[0152] By breaking down multifaceted problems into specialized agents and orchestrating their activities through constraints, GPTProX can address complex real-world optimization challenges. This agent-based approach extends the system's mathematical problem-solving capabilities by providing flexibility and scalability through modular design.

[0153] Example: GPTProX in post-hurricane aid In the aftermath of a hurricane, utility companies face numerous unforeseen challenges. Their usual relief plans are often inadequate because each disaster presents unique challenges. Using traditional methods, residents in affected areas may not be able to quickly access the specific assistance they need.

[0154] GPTProX is a solution to this problem. It combines ChatGPT's knowledge with specific data from utility companies. When residents request help, GPTProX consults databases such as utility asset databases, GIS, and ontology databases to provide accurate and location-specific advice.

[0155] ChatGPT suggests possible responses based on residents' queries. These suggestions are then checked against a Prolog knowledge base to ensure relevance. If any questions arise regarding the suggestion's content, category theory is used to eliminate any inconsistencies.

[0156] This system doesn't just provide general information. It offers clear, actionable steps. For example, it can guide residents to safer areas, tell them how to obtain emergency supplies, or help them report specific damage so that repairs can be completed more quickly.

[0157] GPTProX's best feature is its multi-agent optimization solver. After a hurricane, it considers all the various challenges to provide the most effective recommendations. This means residents quickly receive intelligent and practical advice.

[0158] Example: Personalized health assistance GPTProX can be highly valuable in providing personalized health and wellness support by connecting its conversational capabilities to an individual's health data. For example, consider a user named John who regularly interacts with the system for dietary and lifestyle advice.

[0159] John built a personal health knowledge base in GPTProX, which includes the following facts: John is allergic to peanuts. John suffers from high blood pressure and takes medication for it. John's favorite foods are sushi, pasta, and salad. John's goal is to exercise 3 times a week. When John requests dietary recommendations from GPTProX, its conversation with ChatGPT generates responses with dietary tips and recipes. The logic programming engine parses these responses into predicate logic for verification against John's personal knowledge base. Any contradictory information, such as recipes containing peanuts, is filtered out.

[0160] GPTProX also expands John's health knowledge by interfaceing with his fitness tracker data and the latest lab tests from his doctor. It transforms this data into logically programmed facts to keep the knowledge up-to-date. Over time, GPTProX accumulates extensive knowledge about John's health condition.

[0161] The category theory component further examines these responses by modeling them as category objects mapped from John's input prompts. Discrepancies in the mapping allow for the detection of inconsistencies. For example, dinner recipes containing nuts, mapped from the "no nuts" prompt, are flagged.

[0162] For suggested actions (such as adopting exercise and health regimens), the multi-agent optimization solver customizes the plan based on John's health level, past injuries, exercise preferences, and weekly schedule.

[0163] In this way, GPTProX delivered personalized health guidance to John, tailored to his unique medical needs and constraints. The integration of LLM with the health knowledge base makes the system more robust, accurate, and reliable.

[0164] Example: Software project bidding GPTProX helps software companies develop optimal bid proposals for large projects. The company's knowledge base includes: • Developer resources categorized by skill set (front-end, back-end, DevOps, etc.) • Resource availability schedule • Detailed information on the company's flagship products and previous projects When high-level functional requirements for a tender are input into GPTProX, it engages in a dialogue with ChatGPT to suggest implementation methods and required developer resources. A logic programming validator parses these suggestions to query the knowledge base. It verifies feasibility based on the company's current staffing and constraints. Suggestions that fail validation are discarded.

[0165] The category theory module further examines the implementation plan by modeling it against the requirements to capture inconsistencies. This enhances the robustness of the final proposal.

[0166] For tasks such as assembling technical teams and estimating costs, GPTProX's optimization solver and knowledge base interface provide the optimal allocation and scheduling for developers based on skill sets, availability windows, and dependencies.

[0167] These optimized outputs are combined to generate a comprehensive project execution plan. GPTProX performs financial calculations to provide a detailed cost estimate for this plan.

[0168] The final output is an accurate bid proposal that aligns staffing, timelines, and budgets with the project's technical and functional requirements. Knowledge base integration automates, speeds up, and reliably generates this feasible bid.

[0169] Example: Personalized holiday planning GPTProX can suggest customized vacation itineraries by combining users' personal interests and preferences.

[0170] Consider Jack, who maintains a knowledge base with the following facts: Jack enjoys hiking, art galleries, and wine tasting. Jack's favorite vacation spots used to be Barcelona and Florence. Jack's budget for the 7-day trip is approximately $3,000. When Jack asked GPTProX to plan his vacation in Paris, it built suggestions that incorporated Jack's interests and past vacation patterns. Chat-GPT then generated Paris sightseeing recommendations based on these personalized suggestions.

[0171] The logic programming module parses Chat-GPT responses into Prolog format to query Jack's knowledge base. This filters out unfeasible suggestions, such as expensive Michelin-starred restaurants.

[0172] GPTProX also interfaces with the Paris GIS system and tourism ontology to extract relevant geographic facts and semantic relationships. These expand the contextual basis for vacation planning.

[0173] The category-theoretic verifier models the vacation itinerary over multiple iterations, correcting any inconsistencies that violate Jack's constraints.

[0174] Finally, the multi-agent optimization solver customizes the schedule based on opening time, travel time between locations, pedestrian traffic, and Jack's budget.

[0175] Example: Rapid adjustment of the supply chain introduction: In today's dynamic enterprise environment, unexpected disruptions to the supply chain—whether due to natural disasters, geopolitical issues, or pandemics—can severely impact a company's operations. Rapid response to such disruptions is crucial for maintaining operational efficiency. ChatGPT, enhanced with GPTProX functionality, can help businesses quickly identify and implement alternative strategies.

[0176] Participants: 1. Supply Chain Manager 2. ChatGPT with GPTProX 3. Local knowledge base Prerequisites: The company has established a supply chain management system.

[0177] • GPTProX with ChatGPT integrates with a local knowledge base containing data on suppliers, logistics, inventory levels, and customer orders.

[0178] Scene: 1. Supply chain disruptions: Unexpected events (such as severe storms) disrupt supply chains, halting shipments from major suppliers.

[0179] 2. Consult GPTProx: The supply chain manager consulted GPTProX on this issue, providing all the necessary details.

[0180] 3. Local knowledge base integration: GPTProx sends a query to ChatGPT to seek a solution.

[0181] ChatGPT recommends a list of potential alternative suppliers and logistics partners that may fill the gaps.

[0182] GPTProX accesses the local knowledge base to understand current inventory status, pending orders, alternative suppliers, shipping routes, and associated costs.

[0183] • The knowledge base filters out inappropriate options based on current data (such as blacklisted suppliers or routes that have recently experienced disruptions).

[0184] 4. Solution generation: For each recommendation, provide a brief analysis of its advantages and disadvantages, along with the expected delivery timeline.

[0185] GPTProX uses its built-in optimizer to rank solutions based on parameters such as cost-effectiveness, delivery speed, and reliability, ensuring that recommendations are not only feasible but also optimal.

[0186] GPTProX presents the supply chain manager with an optimized list of alternatives, along with actionable steps.

[0187] 5. accomplish: The company took immediate action based on the GPTProX recommendation.

[0188] 6. Feedback loop: The results are fed back to the local knowledge base.

[0189] Postconditions: • Address supply chain disruptions promptly.

[0190] Maintain customer satisfaction.

[0191] Advantages: 1. Rapid response: ChatGPT and GPTProX enable instant analytics and response.

[0192] 2. Informed decision-making: Data-driven recommendations ensure relevant and current decisions.

[0193] 3. Continuous learning: The system continuously learns and adapts through feedback.

[0194] Example: Fraud detection in phishing emails Advances in AI generation have enabled hackers to create seemingly authentic, customized phishing emails. Traditional spam filters that rely on keywords are insufficient because generative AI can generate contextually coherent text. GTProX's hybrid inference helps detect such sophisticated fraud.

[0195] Consider Sarah, who maintains a personal knowledge base of the following facts: Sarah has bank accounts in Chase and Wells Fargo. Her salary was deposited into her Chase checking account. She has a car loan in Wells Fargo. When Sarah received an email claiming her Wells Fargo account was locked, GPTProX parsed it as a logical predicate to check against her knowledge base. Statements contradicting Sarah's facts, such as "Your Bank of America account has a pending wire transfer," were identified as hallucinations, revealing the message to be fraudulent.

[0196] For executable statements such as "Click here to unlock your account," GPTProX parses them into Prolog and sends them as prompts to ChatGPT. A risk assessment from ChatGPT determines that actions such as clicking links or downloading attachments are harmful.

[0197] By combining language understanding with logical reasoning, GPTProX provides robust protection against sophisticated phishing attempts. Knowledge base integration allows detection to be based on a user's personal financial profile, rather than solely relying on language patterns. As AI generation becomes increasingly sophisticated, this hybrid approach enhances fraud defense.

[0198] discuss This example demonstrates how GPTProX can leverage logical and semantic inconsistencies to identify fraudulent emails generated by AI systems. Key aspects include: • A personal knowledge base containing facts about real accounts, assets, etc. • Parse emails into Prolog predicates and check them against a knowledge base. • The detection of a factual contradiction revealed that the message was invalid. • Identify risks by analyzing actions via ChatGPT Customized protection based on individual profiles (not just language) Extend this approach across the organization by enabling robust defenses through detection based on employees' collective factual knowledge. With the proliferation of creative phishing attacks, reasoning-based technologies will become increasingly critical.

[0199] Example Implementation Systems for validating and enhancing responses 1. Systems for validating and enhancing responses from large language models, including: 1. The logic programming module is configured as follows: (a) Parse the response from the large language model into logical programming predicates; The logic programming module is configured as follows: 2. (a) Query the logic programming knowledge base to verify the predicate; 3. The logic programming module is configured as follows: (a) Filter out contradictory information from the response.

[0200] Methods for enhancing response 2a. A computer-implemented method for enhancing responses from large language models, comprising: 1. Receive user input queries; 2. Retrieve one or more responses to the query from the large language model; 3. Parse the response into logical programming predicates; 4. Verify the predicates by referring to the logic programming knowledge base; 5. Model queries and validated responses as category objects and morphisms; 6. Use category theory to identify semantic inconsistencies; 7. Optimize any suggested actions in the response using mathematical programming techniques; and 8. Output a verified and optimized response.

[0201] Computer systems for enhancing response 2b. Computer systems for enhancing responses from large language models, including: 1. Processor; 2. Memory, which stores instructions that can be executed by the processor to: (a) Receive user input queries; (b) Obtain responses from large language models; (c) Parse the response into logical programming predicates; (d) Verify the predicate against the knowledge base; (e) Perform category modeling on the response to identify inconsistencies; (f) Optimize the suggested actions mathematically; (g) Output the enhanced response.

[0202] Glossary ChatGPT ChatGPT is a variant of the OpenAI GPT (Generative Pretrained Transformer) model, specifically designed for conversational AI tasks.

[0203] GPT (Generative Pretrained Transformers) is a series of AI models developed by OpenAI for natural language processing and understanding.

[0204] Transformer architecture is a deep learning model architecture introduced by Vaswani et al. in the paper "Attention Is All You Need", which forms the basis of models such as GPT and BERT.

[0205] Fine-tuning a pre-trained model trained on a specific dataset to adapt it to a specific task such as dialogue interaction.

[0206] LLM (Language Model) LLM is an abbreviation for "Language Model," which is an artificial intelligence model for understanding and generating human-like text.

[0207] The lexical language model reads text units that can be as short as a single character or as long as a word.

[0208] Zero-shot, one-shot, and few-shot learning are terms that describe how a model performs a task with no examples, one example, or a small number of examples, respectively.

[0209] Prolog Prolog (Prolog) is a high-level programming language primarily associated with artificial intelligence and symbolic reasoning.

[0210] In Prolog, facts represent the basic statements of knowledge.

[0211] A statement in a rule Prolog that expresses the relationship between facts.

[0212] A method for backtracking Prolog searches for possible solutions to find a solution that satisfies a given query.

[0213] Predicates are functions that return a Boolean value (true or false) and are used in Prolog to define relationships between objects.

[0214] Horn clauses are logical expressions used in logic programming and formal logic, consisting of disjunctions of literals with at most one positive literal. They are commonly used in Prolog to represent rules and facts, enabling efficient reasoning and inference.

[0215] MATLAB MATLAB (Matrix Labs) is a high-level programming language and environment designed for numerical and matrix computation, data analysis, and visualization.

[0216] Simulink is an add-on to MATLAB that provides a graphical environment for modeling, simulating, and analyzing dynamic systems.

[0217] M-files are scripts or function files written in MATLAB's native language.

[0218] A rectangular array of matrix numbers is commonly used in MATLAB for computation and data representation.

[0219] SpaCy SpaCy is an open-source library for natural language processing in Python.

[0220] Lexicalization is the process of segmenting text into words, phrases, symbols, or other meaningful elements (called lexes).

[0221] NER (Named Entity Recognition) is the process of identifying and classifying named entities (such as people's names, organization names, and place names) in text.

[0222] Dependency parsing analyzes the grammatical structure of a sentence to determine the relationships between words.

[0223] A pipeline is a series of processing steps, often used in SpaCy to process text through various stages such as lexicalization, annotation, and parsing.

[0224] Exemplary computing device: Reference Figure 7 The illustration shows a computing device 1200, which can be configured to perform the functionality described herein via one or more of an application 1211 or computer-executable instructions. More specifically, in some embodiments, aspects of the methods described herein can be translated into software or machine-level code that can be installed on and / or executed by the computing device 1200, such that the computing device 1200 is configured to perform the functionality described herein. It is contemplated that the computing device 1200 may include any number of devices, such as personal computers, server computers, handheld or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, digital signal processors, state machines, logic circuits, distributed computing environments, and the like.

[0225] The computing device 1200 may include various hardware components, such as a processor 1202, a main memory 1204 (e.g., system memory), and a system bus 1201 that couples the various components of the computing device 1200 to the processor 1202. The system bus 1201 may be any of a variety of bus architectures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus (also known as a Mezzanine bus).

[0226] The computing device 1200 may also include a variety of memory devices and computer-readable media 1207, which includes removable / non-removable media and volatile / non-volatile media and / or tangible media, but excludes transient propagation signals. The computer-readable media 1207 may also include computer storage media and communication media. Computer storage media includes removable / non-removable media and volatile / non-volatile media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data), such as RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, cassette tape, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information / data and is accessible by the computing device 1200. Communication media includes computer-readable instructions, data structures, program modules, or other data in modulated data signals (such as carrier waves or other transmission mechanisms), and includes any information transmission medium. The term "modulated data signal" means a signal whose one or more characteristics are set or altered in a manner that encodes information in a signal. For example, communication media may include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and / or other wireless media), or some combination thereof. Computer-readable media may be implemented as computer program products, such as software stored on computer storage media.

[0227] Main memory 1204 includes computer storage media in the form of volatile / non-volatile memory, such as read-only memory (ROM) and random access memory (RAM). The Basic Input / Output System (BIOS) is typically stored in ROM and contains basic routines that facilitate (e.g., during startup) the transfer of information between elements within computing device 1200. RAM typically contains data and / or program modules that are immediately accessible and / or currently operated by processor 1202. Furthermore, data storage 1206, in the form of read-only memory (ROM) or other forms, may store the operating system, application programs, and other program modules and program data.

[0228] Data storage 1206 may also include other removable / non-removable, volatile / non-volatile computer storage media. For example, data storage 1206 may be: a hard disk drive that reads from or writes to a non-removable, non-volatile magnetic medium; a disk drive that reads from or writes to a removable, non-volatile magnetic disk; a solid-state drive; and / or an optical disk drive that reads from or writes to a removable, non-volatile optical disk (such as a CD-ROM or other optical media). Other removable / non-removable, volatile / non-volatile computer storage media may include magnetic tape cassettes, flash memory cards, digital multifunction disks, digital videotapes, solid-state RAM, solid-state ROM, etc. The drives and their associated computer storage media provide storage for computer-readable instructions, data structures, program modules, and other data for the computing device 1200.

[0229] Through the user interface 1240 (displayed via monitor 1260), a user can input commands and information using input devices 1245 such as tablets, electronic digitizers, microphones, keyboards, and / or pointing devices (commonly referred to as mice, trackballs, or touchpads). Other input devices 1245 may include joysticks, game controllers, satellite antennas, scanners, and so on. Furthermore, voice input, gesture input (e.g., via hand or finger), or other natural user input methods may also be used with appropriate input devices such as microphones, cameras, tablets, touchpads, gloves, or other sensors. These and other input devices 1245 are operatively connected to processor 1202 and may be coupled to system bus 1201, but may also be connected via other interfaces and bus structures such as parallel ports, game ports, or universal serial bus (USB) . Monitor 1260 or other types of display devices may also be connected to system bus 1201. Monitor 1260 may also be integrated with touchscreen panels, etc.

[0230] The computing device 1200 can be implemented in a networked or cloud computing environment, using a network interface 1203 to make a logical connection to one or more remote devices, such as remote computers. The remote computer can be a personal computer, server, router, network PC, peer-to-peer device, or other public network node, and typically includes many or all of the elements described above with respect to the computing device 1200. The logical connection may include one or more local area networks (LANs) and one or more wide area networks (WANs), but may also include other networks. Such networking environments are common in offices, corporate WANs, intranets, and the Internet.

[0231] When used in a networked or cloud computing environment, computing device 1200 can be connected to public and / or private networks via network interface 1203. In such embodiments, modems or other components for establishing communication on the network are connected to system bus 1201 via network interface 1203 or other suitable mechanisms. Wireless networking components, including interfaces and antennas, can be coupled to the network via suitable means such as access points or peer computers. In a networked environment, program modules or portions thereof described with respect to computing device 1200 can be stored in a remote memory storage device.

[0232] Some embodiments are described herein as comprising one or more modules. Such modules are hardware-implemented and therefore include at least one tangible unit capable of performing certain operations, and can be configured or arranged in a certain manner. For example, a hardware-implemented module may include dedicated circuitry permanently configured (e.g., as a dedicated processor, such as a field-programmable gate array (FPGA) or application-specific integrated circuit (ASIC)) to perform certain operations. A hardware-implemented module may also include programmable circuitry (e.g., contained within a general-purpose processor or other programmable processor) temporarily configured by software or firmware to perform certain operations. In some example embodiments, one or more computer systems (e.g., standalone systems, client and / or server computer systems, or peer-to-peer computer systems) or one or more processors may be configured by software (e.g., an application or application portion) as a hardware-implemented module that operates to perform certain operations as described herein.

[0233] Therefore, the term "hardware-implemented module" encompasses tangible entities, whether physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmable), that operate and / or perform certain operations described herein in a certain way. Consider embodiments of hardware-implemented modules that are temporarily configured (e.g., programmable), each of which does not need to be configured or instantiated at any given time. For example, in the case where the hardware-implemented modules include a general-purpose processor configured using software, the general-purpose processor can be configured as correspondingly different hardware-implemented modules at different times. Software can accordingly configure processor 1202, for example, to constitute a particular hardware-implemented module at one time and different hardware-implemented modules at different times.

[0234] Hardware-implemented modules can provide information to and / or receive information from other hardware-implemented modules. Therefore, the described hardware-implemented modules can be considered as communicatively coupled. When multiple such hardware-implemented modules exist simultaneously, communication can be achieved through signal transmission connecting the hardware-implemented modules (e.g., via appropriate circuitry and buses). In embodiments where multiple hardware-implemented modules are configured or instantiated at different times, communication between such hardware-implemented modules can be achieved, for example, by storing and retrieving information in a memory structure accessible to the multiple hardware-implemented modules. For example, one hardware-implemented module can perform an operation and store the output of that operation in a communicatively coupled memory device. Another hardware-implemented module can then access the memory device at a later time to retrieve and process the stored output. Hardware-implemented modules can also initiate communication with input or output devices.

[0235] The computing systems or devices mentioned herein may include desktop computers, laptops, tablets, e-readers, personal digital assistants, smartphones, gaming devices, servers, and so on. The computing devices can access computer-readable media, including computer-readable storage media and data transmission media. In some embodiments, a computer-readable storage medium is a tangible storage device that does not include transient propagation signals. Examples include memories such as main memory, cache memory, and secondary storage (e.g., DVDs), as well as other storage devices. Instructions may be recorded on or encoded with computer-executable instructions or logic that implement the functional aspects described herein. Data transmission media can be used to transmit data via wired or wireless connections, via transient propagation signals, or via a carrier wave (e.g., electromagnetic waves).

[0236] The described methods, processes, operations, and associated actions can also be performed in various orders, in parallel, and / or simultaneously, other than the order described herein. The described systems are exemplary in nature and may include additional elements and / or omit elements. Furthermore, references to “one example” in this disclosure are not intended to be construed as excluding the existence of additional embodiments also incorporated into the described features. It should be understood that when a part or process “includes” a component or operation, that part or process does not exclude another component or operation. While illustrative examples (including systems, apparatuses, etc.) of name filtering techniques using phoneme embedding have been described herein, it is to be understood that various other adjustments and modifications may be made within the spirit and scope of the examples herein. Furthermore, it is to be appreciated that while specific figures are shown and described, such figures are illustrative and exemplary and are not intended to limit the scope of this disclosure.

[0237] The foregoing description has been directed to specific examples. However, it will be apparent that other changes and modifications can be made to the described examples, and some or all of their advantages can be obtained. For example, it is explicitly contemplated that the components and / or elements described herein can be implemented as software stored on tangible (non-transitory) computer-readable media, devices, and memories (e.g., disks / CDs / RAM / EEPROMs, etc.) having program instructions that execute on a computer, hardware, firmware, or a combination thereof. Furthermore, the methods describing the various functions and techniques described herein can be implemented using computer-executable instructions stored on or otherwise available from computer-readable media. Such instructions may include, for example, instructions and data that cause or otherwise configure a general-purpose computer, special-purpose computer, or special-purpose processing apparatus to perform a function or group of functions. Part of the computer resources used may be accessible via a network. Computer-executable instructions may be, for example, binary files, intermediate format instructions (such as assembly language), firmware, or source code. Examples of computer-readable media that can be used to store instructions, information used, and / or information created during the methods according to the described examples include disks or optical discs, flash memory, USB devices equipped with non-volatile memory, networked storage devices, and the like. Furthermore, apparatuses implementing these disclosed methods may include hardware, firmware, and / or software, and may take any of a variety of form factors. Typical examples of such form factors include laptops, smartphones, minicomputers, personal digital assistants, and the like. The functionality described herein may also be implemented in peripheral devices or add-in cards. As a further example, such functionality may also be implemented on a circuit board within a different chip or in different processes executed within a single device. Instructions, media for transmitting such instructions, computing resources for executing them, and other structures for supporting such computing resources are means of providing the functionality described in these disclosures. Therefore, this description is to be regarded as illustrative only and not in any other way to limit the scope of the examples herein. Accordingly, the appended claims are intended to cover all such changes and modifications that fall within the true spirit and scope of the examples herein.

[0238] Additional examples This invention details GPTProX, an artificial intelligence system that integrates large language models with logic programming and mathematical optimization to deliver accurate, customized enterprise solutions. GPTProX overcomes the unreliable phantom responses from systems like ChatGPT through a multi-pronged approach. The logic programming engine parses responses into verifiable logical predicates to detect contradictions with a real-world knowledge base. Furthermore, a category theory module models responses as mathematical morphisms, identifying semantic inconsistencies that may indicate misinformation.

[0239] For executable queries, GPTProX utilizes problem templates that incorporate techniques including linear programming, dynamic programming, and optimal control. It decomposes the objective into modular components that can be solved via multi-agent optimization. These solutions are combined to generate optimized, executable instructions tailored to the user.

[0240] The knowledge representation is enriched by interfacing with various structured data sources, such as ontology, geographic information systems, and financial databases. The logic programming knowledge base is also expanded through inductive learning.

[0241] This fusion of language modeling, logical reasoning, category theory, and mathematical optimization results in an enterprise-grade AI assistant. GPTProX overcomes the accuracy limitations of large language models, delivering cross-industry, customizable, logically valid, and real-world optimized responses. The system's architecture brings accuracy and robust configurability to conversational AI.

[0242] In some examples, GPTProX uses multi-agent solving to complement LLM responses. In a manufacturing scenario, one agent can use linear programming to optimize the allocation of raw materials to different production lines, ensuring the most efficient use of resources. Another agent can employ dynamic programming to manage sequential decisions regarding production scheduling, while a third agent might use optimal control to determine the best strategy that minimizes energy consumption during the production process over time. Each agent collaborates by sharing relevant constraints (e.g., machine availability, delivery deadlines) and processes their combined solutions to achieve the optimal global outcome.

[0243] Other non-limiting examples are expected, including the following.

[0244] • API Integration: This supplement specifies how to format the constructed input messages into API calls, a common method for interacting with enterprise systems.

[0245] • Practical applicability: By explicitly mentioning API calls, this statement enhances clarity by reflecting standard communication between generative AI systems, such as GPTProX, and existing enterprise systems.

[0246] It should be understood from the foregoing that, although specific embodiments have been illustrated and described, various modifications can be made thereto without departing from the spirit and scope of the invention, as will be apparent to those skilled in the art. These variations and modifications are within the scope and teachings of the invention as defined in the appended claims.

Claims

1. A method for enhancing responses from a large language model (LLM), comprising: (a) Access at least one response to the query from the large language model; (b) Parse the at least one response into a logical programming predicate, the logical programming predicate defining a plurality of facts and actions associated with the at least one response; (c) Based on the knowledge base used for logic programming, an accuracy (correctness) verification operation is performed on the at least one response, including: (c)(i) Verify and retain the facts in the at least one response that are indicated to be true via the knowledge base, and (c)(ii) Remove any other fact from the plurality of facts from the at least one response, wherein any other fact is defined as a negation of true in the knowledge base; as well as (d) Based on the accuracy verification operation, generate an output that evaluates the at least one response and returns a verified version of the at least one response.

2. The method according to claim 1, further comprising: It has been determined that the knowledge base is unable to verify the aforementioned facts; as well as Perform category verification, including: Generate one or more suggestions that are modified from the query but semantically equivalent to the query. For each of the one or more prompts, obtain the corresponding response from the large language model. Establish the structural relationship between the generated prompts and the corresponding responses from the large language model, and Category theory is used to detect any semantic inconsistencies, where the identification of semantic inconsistencies reflects a lack of indication of the reliability associated with the at least one response.

3. The method according to claim 1, wherein, The logical programming predicate is defined from logical programming sentences associated with the at least one response, and includes conjunctions to construct relationships between the logical programming sentences.

4. The method according to claim 1, further comprising: Perform multi-agent optimization to supplement the at least one response with additional functionality or information associated with the query.

5. The method according to claim 4, further comprising: Iterate over the large language model to help develop at least one mathematical model to customize a modified version of the response, at least in part, based on the query.

6. The method of claim 4, further comprising supplementing the at least one response by the following steps: Multiple agents are generated, each configured to solve a corresponding optimization problem associated with the query using linear programming, dynamic programming, or optimal control. Assign an objective function and constraints to each agent.

7. The method according to claim 6, wherein, Each of the multiple agents, guided by the LLM, exchanges constraints and decisions with other agents in multiple iterations until it converges to the global optimum.

8. The method according to claim 1, further comprising: The at least one response is parsed into the logical programming predicate to construct an input message for an enterprise computer system; The input messages are integrated with existing enterprise computer systems, including information systems and control systems, by formatting the constructed input messages into application programming interface (API) calls for communication with existing enterprise computer systems. as well as Distributing the verified version of the at least one response to the existing enterprise computer system via API enables automated data querying and control actions based on generative artificial intelligence output.

Images