Generative AI using logical reasoning

Abstract

The invention discloses a generative AI adopting logical reasoning. Systems and methods related to generative AIs utilizing logical reasoning are disclosed herein. For example, the LLM may be used to convert statements, such as natural language statements or software code lines, to logical statements in a logical specification language, the logical reasoning engine may be used to evaluate the logical statements, and the LLM may be used to interpret the output of the logical reasoning engine in a natural language (e.g., using a system that employs retrieval enhanced generation (RAG) and / or trim the LLM). Thus, the present technology can be used to provide a generative AI system that can logically infer a set of statements (e.g., natural language statements or software code), infer new facts therefrom, calculate their logical outcomes, and / or check their consistency, and provide a logical interpretation of how to compute.

Description

Background Technology

[0001] Existing large language models (LLMs), such as Generative Pre-trained Transformers (GPTs) and other generative artificial intelligence (AI) systems, primarily utilize deep learning architectures to process and generate responses based on patterns identified from massive datasets. Generative models have become adept at recognizing and replicating patterns, enabling them to generate human-like text by predicting the most likely next word or phrase in a sequence. This approach effectively incorporates analogy-based reasoning, where decisions are made based on similarity scores that quantify the similarity between the vector embeddings of words in a query and the corresponding embeddings of example words learned from the training data. This type of reasoning allows these models to excel in tasks that model language understanding based on patterns encountered in the training data. However, using analogy-based reasoning can lead to illusions, for example, when similarity scores do not adequately represent logic, or when similar words appear in different contexts (e.g., river bank vs. private bank vs. cache bank vs. race-track bank).

[0002] In contrast, logical reasoning relies on formal logic rules and structured thinking to infer new information or solve problems. This form of reasoning requires understanding causality, applying abstract rules in new situations, and arriving at conclusions logically consistent with given premises. Existing LLMs struggle with true logical reasoning because they do not understand or model causality, and the similarity scores they rely on often do not represent logical outcomes or equivalence. Therefore, conventional LLMs cannot accurately perform deductive reasoning, cannot infer generated content or facts not explicitly stated in natural language input, and cannot infer logical outcomes from natural language input. For example, conventional LLMs struggle to solve logical puzzles and often fail to recognize facts that can be inferred from input prompts. Some existing research products have performed logical reasoning on mathematics, geometry, or lemmas, but these products cannot perform logical reasoning on statements such as those appearing in natural language or software. For example, some existing technologies use LLMs to call theorem solvers in mathematics or geometry, but these technologies do not attempt (and cannot) understand or reason about natural language. Furthermore, existing theorem solvers often require specialized knowledge (e.g., providing axioms when prompted), which is beyond the skill range of most users.

[0003] The limitations of generative AI in logical reasoning present significant challenges, especially in contexts where risk is typically high and / or decision-making requires precision and interpretability, such as in law, healthcare, financial services, and code generation (e.g., safety-critical software). More specifically, existing generative AI systems (such as conventional LLMs that rely on pattern recognition) cannot easily trace and explain the principles behind their decisions. This lack of transparency is problematic in fields such as law and medicine, regulated fields, and / or any field where professionals need to understand the basis of decisions to ensure compliance with ethical standards and / or regulatory requirements. For example, in the context of medical diagnosis, clinicians must be able to understand why an LLM recommends a particular treatment to assess its suitability for a patient. In a legal context, where decisions must be justified by legal precedent or regulations, the lack of logical reasoning ability in LLMs can lead to outputs that appear to fit well with observed patterns but fail to adequately consider applicable laws, resulting in inappropriate or even illegal recommendations. The code generation implemented by LLM is difficult to interpret, which can cause problems, especially in applications where safety requirements are critical, such as avionics or medical devices. This is because it hinders the ability to verify and validate safety and operational requirements and complicates troubleshooting and maintenance. In another example, this opacity raises issues in the context of financial services, where financial decisions typically require transparency and justification. The inability to explain why a trading algorithm might execute a particular transaction could lead to regulatory compliance issues and prevent the identification and correction of errors. In summary, existing generative AI systems cannot perform logical reasoning and explain their decision-making processes, complicating their deployment in specialized environments where the integrity, interpretability, reproducibility, and adherence to specific rules and ethical standards are paramount. Therefore, there is a need for generative AI systems capable of logical reasoning from statements such as those appearing in natural language or software code. Summary of the Invention

[0004] Embodiments of this disclosure relate to generative AI with logical reasoning capabilities. For example, an LLM (Logical Reasoning Model) can be used to convert statements, such as natural language statements or lines of software code, into logical statements in a logical canonical language. A logical reasoning engine can be used to evaluate the logical statements, and the LLM can be used to interpret the output of the logical reasoning engine in natural language. Compared to conventional systems such as those described above, this technology can be used to provide a generative AI system capable of performing logical reasoning on a set of statements (e.g., natural language statements or software code), inferring new facts from them, calculating their logical consequences, and / or checking their consistency, and providing a logical explanation of how the calculations were performed.

[0005] Therefore, generative AI systems can use logical reasoning capabilities to evaluate natural language, software, and / or other types of input, and can formulate and return logical evaluations in natural language. Thus, unlike conventional techniques, this technology not only provides LLM and other generative AI systems with a way to logically reason about statements (such as natural language input statements or lines of code), but also provides interpretability (e.g., reasoning, fundamental principles, etc.) for logical evaluations. Therefore, this technology can be used to provide transparency in decision-making in contexts such as law, healthcare, financial services, and code generation, where functional safety standards and / or requirements are high, or where decisions require precision and interpretability. Attached Figure Description

[0006] The system and method for generative AI used for logical reasoning in natural language are described in detail below with reference to the accompanying drawings, wherein:

[0007] Figure 1 These are illustrations of example logical reasoning systems according to some embodiments of the present disclosure;

[0008] Figure 2 This is a data flow diagram according to some embodiments of the present disclosure, illustrating a method for generating a natural language response based on one or more logical evaluations of one or more statements;

[0009] Figure 3 This is a data flow diagram according to some embodiments of the present disclosure, illustrating a method for generating a natural language response based on one or more logical evaluations of time series data;

[0010] Figure 4A This is a block diagram of an example generative LLM system suitable for implementing some embodiments of the present disclosure;

[0011] Figure 4B This is a block diagram of an example generative LLM including a converter encoder-decoder suitable for implementing some embodiments of this disclosure;

[0012] Figure 4C This is a block diagram of an example generative LLM including a decoder-only converter architecture suitable for implementing some embodiments of this disclosure;

[0013] Figure 5 This is a block diagram of an example computing device suitable for implementing some embodiments of the present disclosure; and

[0014] Figure 6 This is a block diagram of an example data center suitable for implementing some embodiments of this disclosure. Detailed Implementation

[0015] Systems and methods related to generative AI with logical reasoning are disclosed. For example, LLM can be used to convert statements such as natural language statements or lines of software code into logical statements in a logical canonical language, a logical reasoning engine can be used to evaluate the logical statements, and LLM can be used to interpret the output of the logical reasoning engine in natural language (e.g., using systems employing Retrieval Augmented Generation (RAG) and / or fine-tuned LLM). Unlike prior art, this technology can be used to provide a generative AI system that can perform logical reasoning on a set of statements (e.g., natural language statements or software code), infer new facts from them, compute their logical consequences, and / or check their consistency, and provide a logical explanation of how the computation was performed.

[0016] In some embodiments, the logic inference engine uses mathematical logic to reason about statements in natural language, software, or other types. To convert statements (e.g., natural language, software, etc.) into a format that the logic inference engine can understand, the LLM can be prompted to convert one or more statements (e.g., natural language, software, etc.) into logical representations in a logic canonical language (e.g., Theorem Provers' Thousands of Problems (TPTP)). In some embodiments, the LLM (e.g., a pre-trained LLM or a base LLM) can be tuned (e.g., fine-tuned, prompted tuning, p-tuned, using few-shot learning, using transfer learning, using retrieval-enhanced prompts), adapted, or otherwise generated to convert from a first language (e.g., a natural language such as English, a programming language) into a format specified by the logic canonical language. For example, the LLM can be fine-tuned using a dataset that associates one or more sets of statements in the first language with a corresponding set or more sets of logic statements (e.g., first-order and / or higher-order) (e.g., in TPTP format). In this way, user-provided prompts (e.g., in a first language, such as natural language) can be inserted into one or more template prompts that instruct (e.g., fine-tuned) the LLM to convert the content of the prompts (e.g., natural language statements, a piece of software code) into corresponding (e.g., first-order and / or higher-order) logical statements in a format specified by the logic specification language.

[0017] Logic statements can be provided to a logic inference engine, such as a theorem prover or a satisfiability modular theory (SMT) solver, and the logic inference engine can use any known techniques to process the logic statements to generate one or more logic evaluations (e.g., inferring new facts not explicitly stated in natural language input, applying logic rules to prove or disprove conjectures derived from corresponding queries in natural language prompts, performing consistency checks on logic statements, etc.). The logic inference engine can output a representation of its logic evaluations in a logic specification language (e.g., a set of logic statements representing a proof or disprove), which can be the same as the logic specification language accepted as input.

[0018] In some embodiments, an LLM can be used to translate the logical evaluation of a logic inference engine into a first language (e.g., natural language). For example, logical statements representing the logical evaluation of the logic inference engine's output can be inserted into one or more template prompts that instruct the LLM to interpret the logical statements in natural language. In some embodiments, the LLM can be adapted or otherwise modified to translate the format used by the logic inference engine's output (e.g., expressed in a logical canonical language) into natural language. For example, explanations of logical canonical language or theorem proofs can be encoded into a knowledge database (e.g., a vector database such as VectorDB or Chroma), and then retrieval-enhanced generation can be used to enhance the prompts using information encoded in the knowledge database, providing the retrieval-enhanced prompts to the LLM to prompt it to interpret the logical statements generated by the logic inference engine. In this way, natural language explanations or answers to questions can be provided to the user.

[0019] This technology can be used in a variety of applications and use cases. For example, a chatbot can use this technology to perform logical reasoning on a set of natural language statements or software code provided in one or more input prompts or generated by the chatbot, infer new facts from them, calculate their logical consequences, and / or check their consistency (e.g., explain why they produce a particular output). In some embodiments, a chatbot can accept one or more statements and queries associated with those statements, and the chatbot can use this technology to answer the query based on those statements. In some embodiments, input of some modality other than natural language can be converted into natural language, and this technology can be used to perform logical reasoning on the input or answer questions about that input. For example, time series data (e.g., video including frame time series, time series of sensor data, time series of financial or stock performance, etc.) can be converted into a sequence of natural language statements in various ways, depending on the data type involved. As a non-limiting example, any known technology can be used to generate a text representation (e.g., a content summary) of each time series data point (e.g., each frame in a video, each sensor measurement, each financial data point, or stock price) to create a sequence of natural language statements, and this technology can be used to reason and / or answer questions about that sequence of statements. Therefore, time-series data can be converted into natural language statements, allowing questions to be asked about the data, and this technology can use logical reasoning to answer these questions. For example, with video (e.g., surveillance video or surgical video), this technology can be used to answer questions about the video content or generate logical explanations of the video. Similarly, with tabular data (e.g., financial or stock performance presented in tabular form), the data can be converted into logical statements, and this technology can be used to answer questions about the data or generate logical explanations of the data (e.g., explaining outliers, the consequences of increased debt, or decreased assets). These are just a few examples; other applications can be implemented within the scope of this disclosure.

[0020] Therefore, generative AI systems can use logical reasoning capabilities to evaluate natural language, software, and / or other types of input, and can formulate and return logical evaluations in natural language. Thus, unlike conventional techniques, this technology not only provides LLM, VLM, and other generative AI systems with a method for logical reasoning on statements such as natural language input sentences or lines of code, but also provides interpretability for logical evaluations (e.g., reasoning, fundamental principles, etc.). Therefore, this technology can be used to provide decision transparency in contexts such as legal, healthcare, financial services, and code generation fields; regulated fields; and / or any high-risk or decision-making contexts requiring precision and interpretability.

[0021] refer to Figure 1 , Figure 1This is an example logic reasoning system 100 according to some embodiments of this disclosure. It should be understood that such and other arrangements described herein are presented by way of example only. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groups, etc.) may be used in addition to or in place of the arrangements and elements shown, and certain elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in combination with other components, and can be implemented in any suitable combination and location. The various functions performed by the entities described herein can be implemented by hardware, firmware, and / or software. For example, various functions can be implemented by a processor executing instructions stored in memory.

[0022] In passing Figure 1 In an example data stream of the logic reasoning system 100, the logic reasoning system 100 receives input text 110 (e.g., via input interface 105, such as a chatbot interface), and prompt generator 120 can use the input text 110 to prompt LLM 130 to convert the input text 110 from a first language (e.g., natural language, programming language) into one or more logic statements 135. The format of these logic statements is specified by a specified logic specification language that is compatible with logic reasoning engine 140 (e.g., can be digested, understood, used, etc. by logic reasoning engine 140). The logic statements 135 can be processed by logic reasoning engine 140 to generate one or more logic evaluations 145, and prompt generator 150 can use the logic evaluations 145 to prompt LLM 180 to generate a natural language response that interprets the logic evaluations 145, answers questions about the logic evaluations 145, or provides other information associated with the logic evaluations 145. In some embodiments, the retrieval augmented prompt generator 160 may use information encoded in the knowledge database 170 about logic, theorem proofs and / or the logic specification language used by the logic reasoning engine 140 to augment the prompts provided to the LLM 130 and / or LLM 180.

[0023] Input text 110 may represent one or more statements and / or queries (or questions) (e.g., natural language, software), which may be received in various ways. In some embodiments, input text 110 is input to input interface 105 (e.g., a graphical user interface (GUI) such as a form or chat interface). For example, a chatbot provided by a website or mobile app may accept one or more user queries and provide them as input text 110 to prompt generator 120. In some embodiments, spoken input received through an audio interface in input interface 105 is provided to text generation component 107, which may use speech recognition and / or natural language processing to convert the spoken input into text, and the resulting text may be used as input text 110. In some embodiments, visual input received through a visual interface in input interface 105 (e.g., sign language, such as American Sign Language) is provided to text generation component 107, which may use visual recognition and / or natural language processing to convert the visual input into text, and the resulting text may be used as input text 110. Input text 110 can be received via one or more application programming interfaces (APIs) that connect to certain upstream applications or functions, can be recognized by text generation component 107 using optical character recognition (OCR) from one or more document scans or images, can be generated by text generation component 107 using any known detection or summarization techniques (e.g., summarizing the visual content of one or more image or video frames, summarizing some other data such as text, numbers, or audio data) and / or other methods. For example, image captioning techniques (e.g., contrastive language-image pre-training (CLIP)) can rely on one or more neural networks to analyze images and generate descriptive text that interprets the image content in natural language. In some embodiments, data summarization techniques can convert time series or other types of data into easily understandable summaries, for example, using any known data analysis, feature extraction, and / or natural language generation techniques. These are merely examples, and other sources of input text 110 are also included within the scope of this disclosure.

[0024] Therefore, the prompt generator 120 can use the input text 110 to prompt the LLM 130. For example, the prompt generator 120 can use one or more template prompts with one or more slots for input statements and / or queries. Template prompts may include system prompts that set, guide, or define the context for interacting with the LLM 130 (e.g., “You are a helpful logician and an expert in mathematical logic and theorem proving. You will help convert an English statement into a first-order logic formula in TPTP format.”), task- or user-focused prompts that instruct the LLM 130 to perform a task or ask a question for the LLM 130 to answer (e.g., “Convert the following English statement into a first-order formula in TPTP format: [Input text 110]”), and / or auxiliary prompts for rephrasing, expanding, or clarifying prompts or tasks. The prompt generator 120 can construct any number of prompts and apply them to the LLM 130. For example, the prompt generator 120 can construct one or more prompts that fill the input text 110 into a single slot. In some embodiments, the prompt generator 120 may use any known technique to parse and identify statements from the questions in the input text 110, and may construct prompts for each statement, a set of statements, or all statements, and for each question (or all questions). These are just a few examples, and other ways of generating prompts using the input text 110 are envisioned within the scope of this disclosure.

[0025] At a high level, LLM 130 can accept prompts including input text 110 (e.g., generated by prompt generator 120) and can use the prompts to generate logic statements 135 that represent the input text 110 in a logic specification language compatible with logic reasoning engine 140 (e.g., digestible, understandable, and usable by logic reasoning engine 140).

[0026] A logic reasoning engine is a computational system designed to apply the principles of logical reasoning, typically by manipulating symbols and following logical rules to reason about new information or make decisions based on given premises. Logic reasoning engines often rely on mathematical logic and formal methods (such as propositional logic, predicate logic, or other logical systems) to model and solve problems. There are various logic reasoning engines, such as theorem provers and SMT solvers, designed to accept input formatted in a specialized logic specification language (also known as a logic programming language) tailored to logical reasoning tasks. A logic specification language is typically characterized by a set of syntactic and semantic rules that define how expressions can be constructed and what their meaning is. Some example logic specification languages ​​that logic reasoning engines can use include: Thousands of Problems for Theorem Provers (TPTP), Prolog, Web Ontology Language (OWL), Satisfiability Modular Theory Library (SMT-LIB), and General Algebraic Specification Language (CASL). Taking TPTP as an example, TPTP provides an automated theorem proving framework and specifies the syntax for defining problems, axioms, and conjectures in a standard, machine-readable format. Such logic specification languages ​​can be used to standardize the input of any number of logic inference engines, thereby promoting interoperability and benchmarking between different systems. Therefore, logic inference engines that accept TPTP can handle a wide range of problems.

[0027] Continuing with TPTP as an example, a TPTP file can include a series of annotated logical statements or formulas (e.g., axioms, assumptions, conjectures, derived logical statements or formulas). Each logical statement or formula can be represented using a specified syntax as a first-order form (FOF), a typed higher-order form (THF), or other forms. In TPTP, the syntax can be represented as:

[0028] language(name,role,formula,source),[useful_info].

[0029] Wherein, “language” indicates the type of the form (e.g., fof for a first-order formula), “name” indicates the unique identifier of the formula, “role” indicates the role of the formula (e.g., as an axiom, assumption), “formula” indicates the actual logical statement or formula, “source” indicates optional information about the source of the formula, and “useful_info” indicates optional comments.

[0030] Furthermore, logic specification languages ​​like TPTP can specify standardized logical symbols, quantifiers, and / or other syntax, which can be used to represent logical formulas, logical relations, logical operations, or other logical statements. For example, TPTP uses "!" to represent a universal quantifier (for all) and "?" to represent an existential quantifier (exists). More specifically, a universal quantifier can be used as in "![X]:" to assert that the formula is true for all values ​​of X; an existential quantifier can be used as in "?[X]:" to assert that there exists at least one value of X for which the formula is true. Predicates representing attributes or relations between entities can be denoted by names beginning with uppercase letters (e.g., P(X,Y)), while functions based on their input return values ​​can be denoted by names beginning with lowercase letters (e.g., f(X,Y)). Different symbols can be used to represent different logical connectives, such as conjunction (e.g., &), disjunction (e.g., |), negation (e.g., ~), implication (e.g., =>), or equivalence (e.g., <=>). For example, a first-order formula in TPT can be represented as follows:

[0031] `fof(example_formula, axiom, ![X]:(?[Y]:(P(X)&Q(Y)=>R(X,Y)))))` declares: "For all X, there exists a Y such that if P(X) and Q(Y) are true, then R(X,Y) is also true." The above description is intended only as an example to illustrate some possible ways in which logic specification languages ​​define the format for specifying logical statements, and other logic specification languages ​​and / or syntaxes may be implemented within the scope of this disclosure.

[0032] The preceding text described an example format of a logic canonical language. Now, returning to LLM 130, LLM 130 can be used to convert input text 110 (e.g., provided in a prompt by prompt generator 120) from a first language (e.g., natural language, programming language) to a logic canonical language (e.g., TPTP). Typically, LLM 130 can be any suitable LLM, such as... Figure 4A Example Generative LLM System 400 or Figure 4A , Figure 4B or Figure 4CGenerative LLM 430. Depending on the specific implementation, LLM 130 can include pre-trained LLMs or base LLMs, such as NVIDIA's Megatron-Turing Natural Language Generation (MT-NLG) or Megatron-LM; OpenAI's GPT series; Google's T5 or Pathways Language Model (PaLM); Meta's BlenderBot or Meta AI (LLaMA) series of large language models; Anthropic's Claude series, or others.

[0033] In some embodiments, to improve conversion or translation, the LLM 130 can be tuned (e.g., using fine-tuning, hint tuning, p-tuning, few-shot learning, transfer learning, retrieval-enhanced generation) or otherwise adapted to convert from a first language (e.g., natural language, programming language) to a format specified by a logic specification language. By way of non-limiting example, a dataset can be used to fine-tune (e.g., pre-trained or base) an LLM that associates one or more sets of statements in the first language (e.g., merged into one or more template hints) with a corresponding set or more sets of logic statements (e.g., first-order and / or higher-order) in a target logic specification language supported by the logic inference engine 140. Thus, taking a first training data point as an example, a set of training statements in the first language can be inserted into one or more template hints that instruct the LLM to convert these statements into corresponding (e.g., first-order and / or higher-order) logic statements in the specified logic specification language. The resulting hints can then be applied to the LLM, and the generated output can be compared with truth logic statements in the target logic specification language and used to update the LLM. This process can be repeated on any number of training data points to fine-tune the LLM 130, thereby converting it into a logical canonical language.

[0034] Taking the translation from natural language to TPTP as an example, an input statement such as "Every boy who loves Alexis hates every other boy Alexis loves" can be inserted into a template prompt such as "You are a helpful logician and an expert in mathematical logic and theorem proving. You will help convert English statements into first-order logic formulas in TPTP format. Convert the following English statement into a first-order formula in TPTP format: [Input statement]". The resulting prompt can then be applied to LLM 130 to generate the corresponding TPTP format.

[0035] fof(ex6_179,axiom,![X]:boy(X)&loves(X,alexis))=>! [Y]:(boy(Y)&

[0036] loves(alexis,Y)&(Y!=X)=>hates(X,Y))).

[0037] In another example, LLM 130 can be prompted to convert the following English statement into the corresponding TPTP format:

[0038] Example input statement A city has two types of people: knights and villains. Everyone in a certain city is either a knight or a villain. Knights always tell the truth. Villains always lie. Mel and Zoe are two of these people. Zoe says Mel is a villain. Mel says neither Zoe nor Mel is a villain. Is Mel a villain?

[0039] Example TPTP output:

[0040] tff(person_type,type,person:$tType).

[0041] tff(knight_type,type,knight:person>$o).

[0042] tff(knave_type,type,knave:person>$o).

[0043] tff(knight_or_knave,axiom,![P:person]:(knight(P)<~>knave(P))).

[0044] tff(says_type,type,says:(person*$o)>$o).

[0045] tff(knights_tell_truth,axiom,![P:person,S:$o]:((knight(P)&says(P,S))=>

[0046] S)).

[0047] tff(knaves_lie,axiom,![P:person,S:$o]:((knave(P)&says(P,S))=>~S)).

[0048] tff(zoey_type,type,zoey:person).

[0049] tff(mel_type,type,mel:person).

[0050] tff(zoey_says,hypothesis,says(zoey,knave(mel))).

[0051] tff(mel_says,hypothesis,says(mel,(~knave(mel)&~knave(zoey)))).

[0052] tff(mel_is_knave,conjecture,knave(mel)).

[0053] In this example, the LLM 130 identifies the question at the end of the input statement and converts it into the corresponding guess in TPTP. Note that by including TPTP type declarations in the truth-training formula, the LLM 130 will learn to include type declarations defining the types of each variable at the beginning of logic statement 135. Additionally or alternatively, depending on the implementation, the output of the LLM 130 can be processed to extract logic statement 135 and / or identify and add any necessary type declarations.

[0054] The foregoing example involves a logic puzzle, but other types of input statements can also be applied, such as statements derived from articles, speeches, books, conversations, lectures, letters, emails, research papers, software code, manuals, a set of requirements, and / or other content. For example, if Company A is acquired by Company B, a question can be posed to the governing law to apply the legal requirement to the corresponding set of facts (e.g., "Company A was acquired by Company B. After the acquisition, can Company A's open-source software be closed-source?"). In this example, LLM 130 can be used to transform the provided legal requirement and declarative statements about the acquisition into corresponding axioms, and the question into corresponding conjectures.

[0055] These examples are only intended to illustrate some possible implementations and applications; other variations (e.g., input language, logic specification language, logical formulas used to encode input statements, and / or the types of variables, etc.) are also considered within the scope of this disclosure. For example, many English statements can be encoded as first-order formulas (e.g., using Boolean statements that specify variables as true or false). More complex statements (such as those appearing in software code) can be encoded using higher-order statements and / or other types of variables (e.g., integers, decimals). In a simple example, months can be encoded using twelve Boolean variables or one integer variable that can take values ​​between 1 and 12. In some domains, such as software (e.g., control systems involving differential equations), certain statements can be encoded using higher-order logic. Typically, the LLM 130 can learn the required encoding behavior for an application by including appropriate examples in its training dataset.

[0056] Therefore, LLM 130 can be used to transform or translate input text 110 into logical statements 135 in a logical canonical language compatible with the logic reasoning engine 140 (e.g., digestible, understandable, usable, etc. by the logic reasoning engine 140). Thus, logical statements 135 can be applied to the logic reasoning engine 140 to derive one or more logical evaluations 145 of the logical statements 135 (e.g., inferred facts, proofs or refutations of conjectures, results of consistency checks, etc.).

[0057] The logic reasoning engine 140 can be any known logic reasoning engine and can use any known techniques to generate one or more logic evaluations 145 based on the logic statement 135. For example, the logic reasoning engine 140 can parse the logic statement 135, check the syntax and semantics of each statement (e.g., verify that the statement conforms to the rules of the supported logic specification language, check whether predicates and functions use the correct number of arguments), encode the logic statement 135 using data structures such as trees or graphs, and / or apply logic reasoning rules to derive new information or make decisions based on the logic statement 135. The logic reasoning engine 140 can perform various types of logic evaluations, such as consistency checks (e.g., checking whether the logic statement 135 is internally consistent and without contradictions), validity tests (e.g., evaluating whether certain conclusions necessarily follow given premises), satisfiability (e.g., checking whether there is at least one interpretation that makes a formula true), implication (e.g., determining whether a particular fact logically follows the logic statement 135), model or property checks (e.g., checking the properties of a model, such as a software model represented by the logic statement 135), and / or others. Some examples of logic reasoning engines include Prolog, answer set programming (ASP) tools (such as Clingo and DataLog (DLV) with disjunction), automatic theorem provers (such as Z3 and Vampire), SMS or other satisfiability (SAT) solvers (such as MiniSAT), etc.

[0058] Typically, the logic reasoning engine 140 can output its logic evaluation 145 in various forms (e.g., binary or Boolean values ​​representing the results of checks such as satisfiability or validity, proofs or derivations showing how to draw conclusions from logic statement 135 using logic rules, one or more counterexamples indicating where the model represented by logic statement 135 is not true, etc.). The logic evaluation 145 can take the form of a set of logic statements in a logic specification language, which can be the same as the logic specification language accepted as input.

[0059] In some embodiments, LLM 180 can be used to translate the logical evaluation 145 of logic reasoning engine 140 into natural language. For example, prompt generator 150 can insert one or more of the logical evaluations 145 into one or more template prompts that instruct LLM 180 to interpret the logical evaluations 145 in natural language. Template prompts may include system prompts that set, guide, or define the context for interaction with LLM 180 (e.g., “You are a helpful logician and an expert in mathematical logic and theorem proving. You will use simple English to help explain the proofs given in the first-order logic formulas in TPTP format.”), task- or user-focused prompts that instruct LLM 180 to perform tasks or ask questions for LLM 180 to answer (e.g., “Explain the following set of proof derivations given in TPTP format: [Logical Evaluation 145]” using simple English), and / or auxiliary prompts that paraphrase, expand, or clarify prompts or tasks. Prompt generator 150 can construct any number of prompts and apply them to LLM 180. For example, the prompt generator 150 can construct one or more prompts that populate a single slot with the logic evaluation 145. In some embodiments, the prompt generator 120 and / or 150 can extract a question from the input text 110 or the logic statement 135, and the prompt generator 150 can insert that question into the corresponding slot in one or more template prompts (e.g., “Answer the following question in simple English based on the following set of proof derivations given in TPTP format. Proof derivations: [Logic evaluation 145]. Question: [Question]”). These are just a few examples, and other methods for generating prompts using the logic evaluation 145 can be implemented within the scope of this disclosure.

[0060] Therefore, the prompt generator 150 can generate one or more prompts and apply them to the LLM 180, which can use these prompts to generate responses (e.g., answers to questions about logic evaluation 145 and / or natural language explanations of logic evaluation 145). Typically, the LLM 180 can be any suitable LLM, such as... Figure 4A Example Generative LLM System 400 or Figure 4A picture, Figure 4B or Figure 4CThe generative LLM 430. Depending on the implementation, LLM 180 may include a pre-trained LLM or a base LLM, such as NVIDIA’s Megatron-Turing Natural Language Generation (MT-NLG) or Megatron-LM; OpenAI’s GPT series; Google’s T5 or Pathways Language Model (PaLM); Meta’s BlenderBot or the Meta AI (LLaMA) series of large language models, Anthropic’s Claude series, or others. In some embodiments, to improve the natural language response, LLM 180 may be tuned (e.g., using fine-tuning, cue tuning, p-tuning, few-shot learning, transfer learning, retrieval-enhanced generation) or otherwise adapted to generate natural language explanations or answer questions about logical evaluation 145 (e.g., using a dataset that associates logical evaluations (e.g., proofs) with corresponding summaries, explanations of new facts, etc.).

[0061] In some embodiments, prompt generator 150 may provide generated prompts to retrieval enhancement prompt generator 160, which may use retrieval enhancement to enhance the prompts using logical or theorem proof content 165 stored in knowledge database 170, and prompt generator 150 (or retrieval enhancement prompt generator 160) may provide enhanced prompts to LLM 180. Additionally or alternatively, prompt generator 120 may provide generated prompts to retrieval enhancement prompt generator 160, which may use retrieval enhancement to enhance the prompts using logical or theorem proof content 165, and prompt generator 120 (or retrieval enhancement prompt generator 160) may provide enhanced prompts to LLM 130.

[0062] More specifically, any suitable logic or theorem proof content 165 can be encoded and stored in the knowledge database 170 using any known technique. For example, the logic or theorem proof content 165 may include text content (e.g., one or more text files (e.g., .txt), document files (e.g., .docx, .pdf), or web page files (e.g., .html) representing one or more explanations, tutorials, or other educational materials that provide information about the logic canonical language or theorem proving technique used by the logic reasoning engine 140. Continuing with the example implementation of the TPTP format, the logic or theorem proof content 165 may take the form of one or more research papers and / or other documents explaining the TPTP format. Generally, any known technique can be used to encode the logic or theorem proof content 165. For example, the retrieval enhancement prompt generator 160 can use OCR to identify representations of logic or theorem proofs. The text in each applicable file of the theorem proof content 165 (e.g., tables and / or equations removed using regular expressions, custom scripts, or other document parsing tools) can be converted into corresponding embeddings using OpenAI's Text Embedding API. In some embodiments, the retrieval enhancement hint generator 160 can combine multiple embeddings into a single unified embedding that encapsulates the collective semantic information of all logical or theorem proof content 165 (e.g., through some type of averaging, such as weighted averaging, using dimensionality reduction, etc.). Thus, one or more embeddings representing the logical or theorem proof content 165 can be stored in a knowledge database 170, which can take the form of a vector database, such as VectorDB or Chroma.

[0063] Therefore, the retrieval enhancement prompt generator 160 can use any known retrieval enhancement generation technique to enhance the received prompt based on one or more embeddings representing the logical or theorem proof content 165 in the knowledge database 170. Taking a single unified database embedding as an example, when the retrieval enhancement prompt generator 160 receives a prompt, it can convert the prompt into an embedding using the same model used for database embeddings, and can calculate a measure of similarity (e.g., cosine, Euclidean) between the prompt's embedding and a single embedding in the database (e.g., using mean reciprocal rank (MRR) or mean precision (mAP)). If the similarity measure meets a certain threshold similarity, the retrieval enhancement prompt generator 160 can enhance the prompt based on the logical or theorem proof content 165 in the knowledge database 170 (e.g., by attaching an overall summary, identifying a text fragment that best represents the overall content, identifying target content consistent with the prompt, and / or extracting or otherwise deriving other explanatory content from the logical or theorem proof content 165). In some embodiments, if the similarity measure does not meet the threshold, the retrieval enhancement prompt generator 160 can determine not to enhance the prompt. In some embodiments, the knowledge database 170 may store multiple embeddings, the retrieval enhancement prompt generator 160 may query the knowledge database 170 to retrieve the most relevant documents or text fragments, and the retrieval enhancement prompt generator 160 may append the retrieved content to the prompt. These are merely examples, and other types of retrieval enhancement generation may be implemented within the scope of this disclosure.

[0064] Therefore, returning to LLM 180, prompt generator 150 (and / or retrieval enhancement prompt generator 160) can prompt LLM 180 to explain or answer questions about logic evaluation 145, and LLM 180 or some other component of logic reasoning system 100 can provide the generated text as output text 190 (e.g., via a chatbot interface). In some embodiments, output text 190 can be further processed for visual and / or auditory presentation (e.g., via a chatbot interface).

[0065] For example, consider an example where input text 110 includes the known logic puzzle “Who killed Aunt Agatha?”, which consists of a series of statements about possible suspects in Aunt Agatha’s death and ends with the question “Who killed Aunt Agatha?”. Hint generator 120 can prompt LLM 130 to convert the statements into corresponding logic statements (e.g., axioms and conjectures), which a logic reasoning engine (e.g., Vampire) can evaluate and use to generate a logical evaluation that proves or refutes the conjecture given the axioms. By prompting LLM 180 to interpret this logical evaluation, LLM 180 can generate a natural language interpretation, such as:

[0066] The given set of proofs is a first-order logic proof designed to refute the disproven conjecture "killed (Agatha, Agatha)". The proof uses various input formulas and applies logical reasoning rules to derive a contradiction, ultimately showing that the disproven conjecture is incorrect.

[0067] The following is a step-by-step explanation of the proof:

[0068] The input formulas state various relations and properties involving the predicates “killed,” “lives,” “hates,” “richer,” and the constants “Agatha,” “Charles,” and “butler.” These formulas provide the initial information for the proof. The negation conjecture “kill (Agatha, Agatha)(killed (agatha, agatha))” is introduced as the target to be refuted.

[0069] The proof is made by applying logical reasoning rules (such as resolution, superposition, back-demodulation, and inclusion resolution) to derive new formulas.

[0070] The derived formula is expressed in conjunctive normal form (CNF), which is the standard form of logical formulas.

[0071] The proof continues to apply various transformations and simplifications to the formula, such as existence instantiation, flattening, and ENF transformation.

[0072] The proof also uses the axiom of choice and Scholes to introduce new variables and simplify the formula.

[0073] Each step of the proof aims to derive contradictions or inconsistencies in the form of clauses containing complementary literals (e.g., “p” and “~p”).

[0074] A contradiction is found when the proof derives the clause "$false" which represents a logical contradiction.

[0075] The proof terminates with the result "refutation," indicating that the negative conjecture has been successfully refuted. In conclusion, the given set of proof derivations shows that the negative conjecture "killed(agatha,agatha)" is false, thus providing evidence that Agatha did indeed commit suicide.

[0076] In another example, input text 110 may include or identify one or more generated statements (e.g., generated software code) generated by a first LLM (e.g., LLM 130, LLM 180, or some other LLM) in response to a current or previous prompt, and may include instructions to explain why the first LLM generated these statements. Thus, prompt generator 120 may prompt LLM 130 to translate the generated statements into corresponding logical statements, and logic reasoning engine 140 may evaluate these logical statements and use them to generate logical evaluations. In some embodiments, the first LLM may use the logical evaluation (and / or its natural language interpretation) to verify, interpret, and / or regenerate some or all of the generated statements to ensure logical consistency. As a non-limiting example, any known generative technique can be used to generate software code, and logic reasoning system 100 may be prompted to interpret or verify the generated code. Additionally or alternatively, the LLM may invoke any known generative technique to generate software code, use the logic reasoning system 100 to validate the software code during generation, and / or rewrite the software code to satisfy the logic representing the generated software code and the logical formulas generated by the logic reasoning system 100. These are merely a few examples, and other variations are contemplated within the scope of this disclosure.

[0077] Now for reference Figure 2 and Figure 3 Each block of methods 200 and 300 described herein includes a computational process that can be executed using any combination of hardware, firmware, and / or software. For example, various functions can be implemented by a processor executing instructions stored in memory. These methods can also be embodied as computer-usable instructions stored on a computer storage medium. These methods can be provided by standalone applications, services, or managed services (independently or in combination with other managed services), or plug-ins to another product, etc. Furthermore, methods 200 and 300 are combined... Figure 1 The logical reasoning system 100 described herein is illustrated by way of example. However, the method may also be performed additionally or alternatively by any system or combination of systems, including but not limited to the system described herein.

[0078] Figure 2 This is a flowchart of a method 200 for generating a natural language response based on one or more logical evaluations of one or more statements, according to some embodiments of this disclosure. Method 200 includes, at block B202, generating one or more logical statements representing the one or more statements in a logical canonical language, based at least on representations of one or more statements (e.g., natural language, software) processed using one or more Large Language Models (LLMs). For example, refer to... Figure 1The prompt generator 120 can use the input text 110 to prompt the LLM 130 to convert the input text 110 from a first language (e.g., a natural language, a programming language) into a logical statement 135 in a format specified by a specified logical specification language.

[0079] Method 200, at box B204, includes: generating a representation of one or more logical evaluations of one or more logical statements. For example, refer to... Figure 1 The logical statement 135 can be applied to the logical reasoning engine 140 to derive one or more logical evaluations 145 of the logical statement 135 (e.g., inferred facts, proofs or refutations of conjectures, results of consistency checks, etc.).

[0080] Method 200, at box B206, includes: generating a natural language response based at least on one or more logical evaluations, based at least on a representation processed using one or more LLMs to perform one or more logical evaluations. For example, refer to... Figure 1 The prompt generator 150 can use the logic evaluation 145 generated by the logic reasoning engine 140 to prompt the LLM 180 to generate a natural language response, which interprets the logic evaluation 145, answers questions about the logic evaluation 145, or provides some other information associated with the logic evaluation 145.

[0081] Figure 3 This is a data flow diagram illustrating a method 300 for generating a natural language response based on one or more logical evaluations of time-series data, according to some embodiments of the present disclosure. Method 300 includes, at block B302: receiving time-series data. For example, regarding... Figure 1 Input interface 105 may include a GUI and / or API for receiving or otherwise accessing specified (e.g., uploaded or otherwise identified) time-series data, such as univariate time-series data involving a single variable measured over time (e.g., daily temperature readings), multivariate time-series data involving multiple variables recorded over time (e.g., stock prices and trading volumes), continuous time-series data (e.g., continuous heart rate monitoring using medical devices), discrete time-series data recorded at set intervals (e.g., hourly electricity consumption measurements or videos consisting of sequences of images separated by fixed time intervals), and / or event-based time-series data that captures information only when certain events occur (e.g., transaction logs in a database at each purchase). Various implementations may involve different types of time-series data.

[0082] Method 300, at box B304, includes: converting time-series data into a sequence of natural language statements. For example, refer to... Figure 1The text generation component 107 can use any known detection or summarization technique (e.g., summarizing the visual content of one or more image or video frames, summarizing other data such as text, numerical, or audio data) to convert time-series data into natural language, depending on the data type and / or implementation scheme. For univariate time-series data, the text generation component 107 can use any known technique to identify trends or changes and describe them in natural language (e.g., "Temperature steadily increased in the first week of March."). For multivariate time-series data, the text generation component 107 can use any known technique to identify and describe relationships between multiple variables (e.g., "Trading volume increased in the first quarter of 2023, and Company A's stock price also rose sharply."). For continuous time-series data, the text generation component 107 can use any known technique to identify and summarize periods of change or stability (e.g., "Patients' heart rates exhibited irregular patterns, often peaking in the afternoon."). For discrete time-series data, the text generation component 107 can use any known technique to identify and summarize changes (e.g., "Electricity consumption peaked in the evening."). For event-based time-series data, the text generation component 107 can use any known technique to identify and summarize events and / or trends (e.g., "A large number of transactions were recorded immediately after the promotional email was sent out.").

[0083] Method 300, at box B306, includes: converting a sequence of natural language statements into logical statements. For example, refer to... Figure 1 The prompt generator 120 can use the input text 110 to prompt the LLM 130 to convert the input text 110 from a first language (e.g., a natural language, a programming language) into a logical statement 135 in a format specified by a specified logical specification language.

[0084] Method 300 includes, at box B308: generating a logical evaluation of the logical statement. For example, refer to... Figure 1 Logical statement 135 can be applied to logical reasoning engine 140 to derive one or more logical evaluations 145 of logical statement 135 (e.g., inferred facts, proofs or refutations of conjectures, results of consistency checks, etc.).

[0085] Method 300, at box B310, includes: generating a natural language response based at least on logical evaluation. For example, refer to... Figure 1 The prompt generator 150 can use a logical evaluation 145 generated by the logical reasoning engine 140 to prompt the LLM 180 to generate a natural language response that interprets the logical evaluation 145, answers questions about the logical evaluation 145, or provides certain other information associated with the logical evaluation 145. In some embodiments, the natural language response may be further processed for visual and / or auditory presentation.

[0086] The systems and methods described herein can be used for a variety of purposes, including but not limited to: machine control, machine motion, machine driving, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and supervision, simulation and digital twins, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation and / or digital twins, data center processing, conversational AI, optical transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation of 3D assets, cloud computing, generative AI, and / or any other suitable application.

[0087] The disclosed embodiments can be included in a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots, aviation systems, medical systems, marine systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using edge devices, systems containing one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems implemented at least partially in a data center, systems for performing conversational AI operations, systems implementing one or more language models (e.g., one or more large language models (LLMs) or one or more visual language models (VLMs)), systems for performing optical transmission simulation, systems for performing collaborative content creation of 3D assets, systems implemented at least partially using cloud computing resources, and / or other types of systems.

[0088] Example Large Language Model

[0089] Large Language Models (LLMs) are a type of generative artificial intelligence (AI) that can understand, summarize, translate, or otherwise generate human-like text based on context provided in input prompts or queries. These language models are often considered "large" because they are trained on massive datasets and have architectures containing a large number of learnable network parameters (weights and biases), with popular LLMs boasting millions or even billions of parameters. LLMs have become adept at summarizing textual data, analyzing and extracting data insights, and generating new text in user-specified styles, tones, or formats. Some LLMs (such as early versions of chatbots like ChatGPT) focus on text processing, while some multimodal LLMs can accept, understand, and / or generate text as well as other types of content, such as images, audio, and / or video. For example, Visual Language Models (VLMs) are a type of LLM that can accept visual and text input and / or generate visual and text output.

[0090] Different types of LLM architectures exist, employing various techniques to understand and generate human-like text. Some early LLM architectures used recurrent neural networks (RNNs) or long short-term memory networks (LSTMs), while many modern LLMs utilize transformer architectures that rely on self-attention mechanisms to understand and recognize relationships between words or tokens. LLMs can include encoder and / or decoder blocks. Discriminative or encoder-only LLMs (e.g., BERT, Bidirectional Encoder Representations from Transformers) are well-suited for tasks involving language understanding, such as classification, sentiment analysis, question answering, and named entity recognition. Generative or decoder-only LLMs (e.g., GPT, Generative Pretrained Transformer) are well-suited for tasks involving both language and content generation, such as text completion, story generation, and dialogue generation. LLMs that include both encoder and decoder components (e.g., T5 (Text-to-Text Transformer)) can understand and generate content, making these models well-suited for tasks such as translation and summarization.

[0091] LLMs primarily utilize unsupervised learning for training, where they learn patterns from large amounts of unlabeled text data. Due to their extensive training, LLMs typically do not require task-specific or domain-specific training. These types of LLMs are already extensively pre-trained on large amounts of unlabeled text data, often referred to as base models, and are suitable for a variety of tasks such as question answering, summarizing, filling in missing information, and translation. Some LLMs can be customized for specific use cases using techniques such as cue tuning, fine-tuning, and / or adding regulators.

[0092] Figure 4A This is a block diagram of an example generative LLM system 400 suitable for implementing certain embodiments of this disclosure. Figure 4A In the example shown, the generative LLM system 400 includes an input processor 405, a tokenizer 410, an embedding component 420, and a generative LLM 430.

[0093] At a high level, input processor 405 can receive input 401, which includes text and other types of input data, depending on the architecture of the generative LLM 430. Typically, input 401 comprises plain text in the form of one or more sentences, paragraphs, or documents. Additionally or alternatively, input 401 may include numerical sequences, pre-computed embeddings (e.g., word or sentence embeddings), and / or structured data (e.g., tabular formats, JSON, or XML). In some implementations of the generative LLM 430 capable of handling multimodal input, input 401 may combine text with image data, audio data, and / or other types of input data. Taking raw input text as an example, input processor 405 can prepare the raw input text in various ways. For example, input processor 405 can perform various types of text cleaning to remove noise (e.g., special characters, punctuation marks, HTML tags, stop words) from the relevant text content. In examples involving stop words (often common words with less semantic meaning), input processor 405 can remove stop words to reduce noise and allow the generative LLM 430 to focus on more meaningful content. Input processor 405 can apply text normalization, for example, by converting all characters to lowercase, removing accent marks, and / or handling special cases such as abbreviations or acronyms to ensure consistency. These are just some examples; other types of input processing can also be applied.

[0094] The tokenizer 410 can segment (e.g., processed) text into smaller units (tags) for subsequent analysis and processing. Depending on the implementation, the tags can represent individual words, sub-words, or characters. Word-based tokenization divides the text into individual words, treating each word as a separate tag. Sub-word tokenization breaks words down into smaller, meaningful units (e.g., prefixes, suffixes, stems), enabling the generative LLM 430 to understand morphological changes and process words outside the vocabulary more effectively. Character-based tokenization represents each character as a separate tag, enabling the generative LLM 430 to process text at a fine-grained level. The choice of tokenization strategy can depend on factors such as the language being processed, the task at hand, and / or the characteristics of the training dataset. Thus, the tokenizer 410 can transform (e.g., processed) text into a structured format.

[0095] Embedding component 420 can use any known embedding technique to transform discrete tokens into semantically meaningful (e.g., dense, continuous vector) representations. For example, embedding component 420 can use pre-trained word embeddings (e.g., Word2Vec, GloVe, or FastText), one-hot encoding, Term Frequency-Inverse Document Frequency (TF-IDF) encoding, one or more embedding layers of a neural network, and / or others.

[0096] In some implementations where input 401 includes image data, input processor 401 may resize the image data to a standard size compatible with the format of the corresponding input channel and / or normalize pixel values ​​to a common range (e.g., 0 to 1) to ensure consistent representation, and embedding component 420 may encode the image data using any known techniques (e.g., using one or more convolutional neural networks (CNNs) to extract visual features). In some implementations where input 401 includes audio data, input processor 405 may resample the audio file to a consistent sampling rate for uniform processing, and embedding component 420 may use any known techniques to extract and encode audio features. In some implementations where input 401 includes video data, input processor 405 may extract frames or apply resizing to extracted frames, and embedding component 420 may extract features such as optical flow embedding or video embedding and / or encode temporal information or frame sequences. In some implementations where input 401 includes multimodal data, the embedded component 420 can use techniques such as early fusion (concatenation), late fusion (sequential processing), and attention-based fusion to fuse representations of different types of data (e.g., text, images, audio).

[0097] The generative LLM 430 and / or other components of the generative LLM system 400 may use different types of neural network architectures depending on the implementation scheme. Transformer-based architectures (such as those used in models like GPT) typically include a self-attention mechanism that weights the importance of different words or tokens in the input sequence and a feedforward network that processes the output of the self-attention layer, applying a nonlinear transformation to the input representation and extracting higher-level features. Some non-limiting example architectures include transformers (e.g., encoder-decoder, decoder-only, multimodal), RNNs, LSTMs, fusion models, cross-modal embedding models that learn a joint embedding space, graph neural networks (GNNs), hybrid architectures that combine different types of adversarial networks (such as generative adversarial networks or GANs or adversarial autoencoders (AAEs) for joint distribution learning), etc. Thus, depending on the implementation scheme and architecture, the embedding component 420 may apply the encoded representation of input 401 to the generative LLM 430, and the generative LLM 430 may process the encoded representation of input 401 to generate output 490, which may include response text and / or other types of data.

[0098] Figure 4B This is a block diagram of an example implementation scheme where the generative LLM 430 includes a converter encoder-decoder. For example, suppose the input text (e.g., “Who discovered gravity”) is tokenized (e.g., by...) Figure 4A The tokenizer 410) is used for tokens such as words, and each token is encoded (e.g., by...). Figure 4A The embedding component 420) is a corresponding embedding (e.g., of size 512). Since these token embeddings do not typically represent the position of the tokens in the input sequence, positional encoding can be added to each token embedding using any known technique to encode the order relation and context of the tokens in the input sequence. Thus, (e.g., the resulting) embeddings can be applied to one or more encoders 435 of the generative LLM 430.

[0099] In the example implementation, encoder 435 forms an encoder stack, where each encoder includes a self-attention layer and a feedforward network. In the example converter architecture, each token (e.g., a word) flows through a separate path. Therefore, each encoder can accept a sequence of vectors, pass each vector through the self-attention layer, then through the feedforward network, and then up to the next encoder in the stack. Any known self-attention technique can be used. For example, to compute a self-attention score for each token (word), a query vector, a key vector, and a value vector can be created for each token. The self-attention score for a token pair can be computed by taking the dot product of the query vector and the corresponding key vector, normalizing the resulting score, multiplying by the corresponding value vector, and summing the weighted value vectors. The encoder can apply multi-head attention, where the attention mechanism is applied multiple times in parallel with different learned weight matrices. Any number of encoders can be cascaded to generate context vectors that encode the input. Attention projection layer 440 can transform the context vectors into attention vectors (keys and values) for decoder 445.

[0100] In the example implementation, decoder 445 forms a decoder stack, where each decoder includes a self-attention layer, an encoder-decoder self-attention layer that uses attention vectors (keys and values) from the encoder to focus on relevant parts of the input sequence, and a feedforward network. Similar to encoder 435, in the example converter architecture, each token (e.g., a word) flows through a separate path in decoder 445. During the first pass, decoder 445, classifier 450, and generation mechanism 455 can generate a first token, and generation mechanism 455 can apply the generated token as input during a second pass. This process can be repeated cyclically, generating tokens (e.g., words) and adding them to the output of the previous pass, and in subsequent passes applying token embeddings of positionally encoded composite sequences as input to decoder 445, generating one token at a time (called autoregression) until a symbol or token indicating the end of the response is predicted. In each decoder, the self-attention layer is typically restricted to focusing only on preceding positions in the output sequence by applying a masking technique (e.g., setting future positions to negative infinity) before the softmax operation. In the example implementation, the encoder-decoder attention layer operates similarly to the (e.g., multi-head) self-attention operation in encoder 435, except that it creates its query from the layer below it and obtains keys and values ​​(e.g., matrices) from the output of encoder 435.

[0101] Therefore, decoder 445 can output some decoded (e.g., vector) representation of the input applied during a particular pass. Classifier 450 can include a multi-class classifier comprising one or more neural network layers and a softmax operation that transforms logit probabilities into probabilities, the neural network layers projecting the decoded (e.g., vector) representation onto corresponding dimensions (e.g., one dimension for each supported word or token in the output vocabulary). Thus, generation mechanism 455 can select or sample words or tokens based on corresponding predicted probabilities (e.g., selecting the word with the highest predicted probability) and append it to the output of the previous pass, thereby generating each word or token sequentially. Generation mechanism 455 can repeat this process, triggering successive decoder inputs and corresponding predictions until a symbol or token representing the end of the response is selected or sampled, at which point generation mechanism 455 can output the generated response.

[0102] Figure 4C This is a block diagram of an example implementation where the generative LLM 430 includes a decoder-only converter architecture. For example, Figure 4C The decoder 460 can be used with Figure 4B The decoder 445 operates similarly, except... Figure 4C Each decoder 460 omits the encoder-decoder self-attention layer (because there is no encoder in this implementation). Therefore, decoders 460 can form a decoder stack, where each decoder includes a self-attention layer and a feedforward network. Furthermore, instead of encoding the input sequence, a symbol or tag indicating the end of the input sequence (or the beginning of the output sequence) can be appended to the input sequence, and the resulting sequence (e.g., a corresponding embedding with positional encoding) can be applied to decoder 460. Figure 4B Similar to decoder 445, each tag (e.g., a word) can flow through a separate path in decoder 460, and decoder 460, classifier 465, and generation mechanism 470 can use autoregression to generate one tag at a time sequentially until a symbol or tag indicating the end of the response is predicted. Classifier 465 and generation mechanism 470 can be combined with... Figure 4B The classifier 450 and generation mechanism 455 operate similarly, wherein generation mechanism 470 selects or samples each consecutive output label based on the corresponding predicted probability and appends it to the output of the previous iteration, generating each label sequentially until a symbol or label representing the end of the response is selected or sampled. The architectures described herein, and others, are merely examples, and other suitable architectures may be implemented within the scope of this disclosure.

[0103] Example computing device

[0104] Figure 5This is a block diagram of an example computing device 500 suitable for implementing some embodiments of the present disclosure. The computing device 500 may include an interconnect system 502 directly or indirectly coupled to: a memory 504, one or more central processing units (CPUs) 506, one or more graphics processing units (GPUs) 508, a communication interface 510, input / output (I / O) ports 512, input / output components 514, a power supply 516, one or more presentation components 518 (e.g., one or more displays), and one or more logic units 520. In at least one embodiment, one or more computing devices 500 may include one or more virtual machines (VMs), and / or any of its components may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 508 may include one or more vGPUs, one or more CPUs 506 may include one or more vCPUs, and / or one or more logic units 520 may include one or more virtual logic units. Accordingly, one or more computing devices 500 may include discrete components (e.g., a full GPU dedicated to computing device 500), virtual components (e.g., a portion of the GPU dedicated to computing device 500), or a combination thereof.

[0105] although Figure 5 The various blocks are shown as being connected to lines via interconnect system 502, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, presentation component 518 (such as a display device) may be considered I / O component 514 (e.g., if the display is a touchscreen). As another example, CPU 506 and / or GPU 508 may include memory (e.g., memory 504 may represent a storage device in addition to the memory of GPU 508, CPU 506, and / or other components). Therefore, Figure 5 The computing devices described are merely illustrative. No distinction is made between categories such as "workstation," "server," "laptop computer," "desktop computer," "tablet computer," "client device," "mobile device," "handheld device," "game console," "electronic control unit (ECU)," "virtual reality system," and / or other device or system types, as all are conceived in… Figure 5 Within the scope of computing devices.

[0106] Interconnect system 502 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 502 may include one or more bus or link types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Fast Peripheral Component Interconnect (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. For example, CPU 506 may be directly connected to memory 504. Further, CPU 506 may be directly connected to GPU 508. In cases where there is a direct connection or point-to-point connection between components, interconnect system 502 may include a PCIe link to perform that connection. In these examples, a PCI bus is not required in computing device 500.

[0107] The memory 504 may include any of a variety of computer-readable media. The computer-readable media may be any available medium accessible by the computing device 500. The computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, the computer-readable media may include computer storage media and communication media.

[0108] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 504 may store computer-readable instructions (e.g., representing programs and / or program elements, such as operating systems). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Universal Disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by computing device 500. As used herein, computer storage media does not include the signal itself.

[0109] Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals (such as carrier waves or other transmission mechanisms) and include any information transmission medium. The term "modulated data signal" can refer to a signal whose characteristics are set or altered in a manner that encodes information in the signal. By way of example and not limitation, computer storage media can include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media). Any combination of the foregoing should also be included within the scope of computer-readable media.

[0110] CPU 506 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 500 to perform one or more of the methods and / or processes described herein. Each CPU 506 may include one or more cores (e.g., 1, 2, 4, 8, 28, 72, etc.) capable of processing multiple software threads simultaneously. CPU 506 may include any type of processor and may include different types of processors depending on the type of computing device 500 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 500, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplemental coprocessors such as math coprocessors, computing device 500 may also include one or more CPUs 506.

[0111] In addition to or replacing CPU 506, one or more GPUs 508 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 500 to perform one or more of the methods and / or processes described herein. One or more GPUs 508 may be integrated GPUs (e.g., having one or more CPUs 506) and / or one or more GPUs 508 may be discrete GPUs. In embodiments, one or more GPUs 508 may be a coprocessor of one or more CPUs 506. GPUs 508 may be used by computing device 500 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, GPUs 508 may be used for general-purpose computing on a GPU (GPGPU). GPUs 508 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. GPUs 508 may generate pixel data for outputting an image in response to rendering commands (e.g., rendering commands received from CPU 506 via a host interface). GPU 508 may include graphics memory, such as display memory, for storing pixel data or any other suitable data, such as GPGPU data. Display memory may be included as part of memory 504. GPU 508 may include two or more GPUs operating in parallel (e.g., via a link). The link may be directly connected to the GPUs (e.g., using NVLINK) or may connect the GPUs via a switch (e.g., using NVSwitch). When combined, each GPU 508 may generate pixel data or GPGPU data for different portions of the output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU may include its own memory or may share memory with other GPUs.

[0112] In addition to or replacing CPU 506 and / or GPU 508, one or more logic units 520 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 500 to perform one or more of the methods and / or processes described herein. In embodiments, one or more CPUs 506, one or more GPUs 508, and / or one or more logic units 520 may perform any combination of methods, processes, and / or portions thereof, discretely or jointly. One or more logic units 520 may be part of and / or integrated into one or more of CPUs 506 and / or GPUs 508, and / or one or more logic units 520 may be discrete components or otherwise external to CPUs 506 and / or GPUs 508. In embodiments, one or more logic units 520 may be coprocessors of one or more CPUs 506 and / or one or more GPUs 508.

[0113] Examples of logic unit 520 include one or more processing cores and / or components thereof, such as a data processing unit (DPU), tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree traversal unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), arithmetic logic unit (ALU), application-specific integrated circuit (ASIC), floating-point unit (FPU), input / output (I / O) element, peripheral component interconnect (PCI) or peripheral component interconnect fast (PCIe) element, etc.

[0114] Communication interface 510 may include one or more receivers, transmitters, and / or transceivers that allow computing device 500 to communicate with other computing devices via electronic communication networks (including wired and / or wireless communications). Communication interface 510 may include components and functions that allow communication over any of a plurality of different networks (e.g., wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication over Ethernet or wireless bandwidth), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet). In one or more embodiments, logic unit 520 and / or communication interface 510 may include one or more data processing units (DPUs) for directly transmitting data received via a network and / or via interconnect system 502 to one or more GPUs 508 (e.g., the memory of one or more GPUs 508).

[0115] I / O port 512 allows computing device 500 to be logically coupled to other devices including I / O component 514, presentation component 518, and / or other components, some of which may be built into (e.g., integrated into) computing device 500. Illustrative I / O component 514 includes a microphone, mouse, keyboard, joystick, gamepad, game controller, disc satellite dish, scanner, printer, wireless device, etc. I / O component 514 provides a natural user interface (NUI) that handles air gestures, voice, or other physiological input generated by the user. In some instances, the input may be passed to appropriate network elements for further processing. NUI can implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, on-screen and near-screen gesture recognition, air gestures, head and eye tracking, and touch recognition associated with the display of computing device 500 (as described in more detail below). The computing device 500 may include a depth camera, such as a stereo camera system, an infrared camera system, an RGB camera system, touchscreen technology, and combinations thereof, for attitude detection and recognition. Additionally, the computing device 500 may include an accelerometer or gyroscope that allows motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the computing device 500 may use the output of the accelerometer or gyroscope to render immersive augmented reality or virtual reality.

[0116] Power supply 516 may include a hardwired power supply, a battery power supply, or a combination thereof. Power supply 516 may provide power to computing device 500 so that the components of computing device 500 can operate.

[0117] One or more presentation components 518 may include displays (e.g., monitors, touchscreens, television screens, head-up displays (HUDs), other display types, or combinations thereof), speakers, and / or other presentation components. Presentation component 518 may receive data from other components (e.g., GPU 508, CPU 506, DPU, etc.) and output data (e.g., as images, videos, sounds, etc.).

[0118] Example Data Center

[0119] Figure 6 An example data center 600 that can be used in at least one embodiment of this disclosure is shown. The data center 600 may include a data center infrastructure layer 610, a framework layer 620, a software layer 630, and / or an application layer 640.

[0120] like Figure 6As shown, the data center infrastructure layer 610 may include a resource coordinator 612, grouped computing resources 614, and node computing resources (“nodes CR”) 616(1)-616(N), where “N” represents any integer, a positive integer. In at least one embodiment, the nodes CR 616(1)-616(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including DPUs, accelerators, field-programmable gate arrays (FPGAs), graphics processors or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (“NW”) devices, and network network interfaces (“NW”). I / O devices, network switches, virtual machines ("VMs"), power modules and / or cooling modules, etc. In some embodiments, one or more nodes CR616(1)-616(N) may correspond to a server having one or more of the aforementioned computing resources. Furthermore, in some embodiments, nodes CR616(1)-616(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more nodes CR616(1)-616(N) may correspond to virtual machines (VMs).

[0121] In at least one embodiment, the grouped computing resources 614 may include separate groups of nodes CR616 housed within one or more racks (not shown) or within a plurality of racks in data centers (also not shown) located in different geographical locations. The separate groups of nodes CR616 within the grouped computing resources 614 may include grouped computing, networking, memory, or storage resources that may be configured or allocated to support one or more workloads. In at least one embodiment, a plurality of nodes CR616, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0122] Resource coordinator 612 may be configured or otherwise controlled to control one or more nodes CR616(1)-616(N) and / or grouped computing resources 614. In at least one embodiment, resource coordinator 612 may include a Software Design Infrastructure (“SDI”) management entity for data center 600. Resource coordinator 612 may include hardware, software, or some combination thereof.

[0123] In at least one embodiment, such as Figure 6As shown, framework layer 620 may include a job scheduler 628, a configuration manager 634, a resource manager 636, and / or a distributed file system 638. Framework layer 620 may include a framework of software 632 supporting software layer 630 and / or one or more applications 642 of application layer 640. Software 632 or application 642 may respectively include web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 620 may be, but is not limited to, a type of free and open-source software web application framework that can utilize the distributed file system 638 for large-scale data processing (e.g., "big data"), such as Apache Spark. TM (Hereinafter referred to as "Spark"). In at least one embodiment, job scheduler 628 may include Spark drivers to facilitate the scheduling of workloads supported by various layers of data center 600. Configuration manager 634 may be able to configure different layers, such as software layer 630 and framework layer 620 including Spark and distributed file system 638 for supporting large-scale data processing. Resource manager 636 may be able to manage cluster or group computing resources mapped to or allocated for supporting distributed file system 638 and job scheduler 628. In at least one embodiment, cluster or group computing resources may include group computing resources 614 at data center infrastructure layer 610. Resource manager 636 may coordinate with resource coordinator 612 to manage these mapped or allocated computing resources.

[0124] In at least one embodiment, the software 632 included in software layer 630 may include software used by at least a plurality of portions of nodes CR616(1)-616(N), grouped computing resources 614, and / or the distributed file system 638 of framework layer 620. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0125] In at least one embodiment, the application 642 included in the application layer 640 may include one or more types of applications used by at least a plurality of portions of nodes CR616(1)-616(N), grouped computing resources 614, and / or the distributed file system 638 of the framework layer 620. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing and machine learning applications (including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.) and / or other machine learning applications used in combination with one or more embodiments).

[0126] In at least one embodiment, any of the configuration manager 634, resource manager 636, and resource coordinator 612 can implement any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. Self-modification actions can free the data center operator of data center 600 from making potentially undesirable configuration decisions and potentially avoid underutilized and / or poorly performing portions of the data center.

[0127] According to one or more embodiments described herein, data center 600 may include tools, services, software, or other resources for training one or more machine learning models or using one or more machine learning models to predict or infer information. For example, one or more machine learning models may be trained by calculating weight parameters according to a neural network architecture using the software and / or computing resources described above with respect to data center 600. In at least one embodiment, a trained or deployed machine learning model corresponding to one or more neural networks may be used to infer or predict information using the resources described above with respect to data center 600 by using weight parameters calculated through one or more training techniques (such as, but not limited to, those described herein).

[0128] In at least one embodiment, the data center 600 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the aforementioned resources. Furthermore, one or more of the aforementioned software and / or hardware resources may be configured to allow users to train or execute information inference services, such as image recognition, speech recognition, or other artificial intelligence services.

[0129] Example network environment

[0130] A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network attached storage (NAS), other back-end devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 5 The implementation is carried out on one or more instances of computing device 500, for example, each device may include similar components, features and / or functions of computing device 500. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of data center 600, examples of which are provided in this document. Figure 6 To describe in more detail.

[0131] Components of a network environment can communicate with each other via one or more networks, which may be wired, wireless, or both. A network can include multiple networks or a network of networks. For example, a network may include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (such as the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. Where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0132] A compatible network environment may include one or more peer-to-peer network environments—in which case the network environment may not include a server—and one or more client-server network environments—in which case the network environment may include one or more servers. In a peer-to-peer network environment, the functionality described herein with respect to one or more servers can be implemented on any number of client devices.

[0133] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. The cloud-based network environment may include a framework layer, job scheduler, resource manager, and distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework for software supporting the software layer and / or one or more applications supporting the application layer. The software or application may respectively include web-based service software or applications. In embodiments, one or more client devices may use web-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, free and open-source software web application frameworks, such as those used for large-scale data processing (e.g., "big data") using distributed file systems.

[0134] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions may be distributed across multiple locations from a central or core server (e.g., across one or more data centers distributed across states, regions, countries, globally, etc.). If the connection to the user (e.g., client device) is relatively close to the edge server, the core server may assign at least a portion of the functionality to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0135] One or more client devices may include the information described in this article. Figure 5 At least some of the components, features, and functions of one or more example computing devices 500 described. By way of example and not limitation, a client device may be embodied as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming device or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these described devices, or any other suitable device.

[0136] This disclosure can be described in the general context of computer code or machine-usable instructions (including computer-executable instructions, such as program modules) that are executed by a computer or other machine (such as a personal data assistant or other handheld device). Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be implemented in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, and more specialized computing devices. This disclosure can also be implemented in a distributed computing environment where tasks are performed by remote processing devices linked via a communication network.

[0137] As used herein, the phrase “and / or” relating to two or more elements should be interpreted as meaning only one element, or a combination of elements. For example, “element A, element B, and / or element C” can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Furthermore, “at least one of element A or element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Additionally, “at least one of element A and element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0138] This document provides a detailed description of the subject matter of this disclosure to meet statutory requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the inventors have anticipated that the claimed subject matter may also be embodied in other ways in combination with other current or future techniques to include combinations of different steps or steps similar to those described in this document. Furthermore, although the terms “step” and / or “box” may be used herein to refer to different elements of the method employed, such terms should not be construed as implying any particular order among or between the various steps disclosed herein, unless and only if the order of individual steps is explicitly described.

[0139] Example text support

[0140] The disclosure of this application also includes the following numbered clauses:

[0141] Clause 1. One or more processors, including processing circuitry, the processing circuitry being configured to: generate one or more logical statements expressed in a logical canonical language and representing the one or more input statements, based at least on representations of the one or more input statements processed using one or more Large Language Models (LLMs).

[0142] Clause 2. One or more processors according to Clause 1, wherein the processing circuitry is further configured to: generate a representation of one or more logical evaluations of the one or more logical statements using a logical reasoning engine corresponding to the logical specification language.

[0143] Clause 3. One or more processors according to Clause 1 or 2, wherein the processing circuitry is further configured to: generate a natural language response for the one or more input statements based at least on the representations of the one or more logic evaluations processed using the one or more LLMs.

[0144] Clause 4. One or more processors according to Clause 1, 2 or 3, wherein the processing circuitry is further configured to: provide the natural language response as output, wherein the one or more input statements include a representation of a query, and the natural language response includes at least one of: an explanation of at least a portion of the query or an answer to a question about at least a portion of the query.

[0145] Clause 5. One or more processors according to Clauses 1, 2, 3 or 4, wherein the processing circuitry is further configured to: generate the representation of the one or more input statements at least based on inserting the one or more input statements into one or more template prompts, the one or more template prompts instructing the one or more LLMs to convert the one or more input statements into the logic specification language.

[0146] Clause 6. One or more processors according to Clause 1, 2, 3 or 4, wherein the processing circuitry is further configured to: generate one or more logic statements using a first large language model of the one or more LLMs, wherein the first large language model is adapted to fit the logic specification language.

[0147] Clause 7. One or more processors as described in Clauses 1, 2, 3 or 4, wherein the logic reasoning engine implements one or more solvers.

[0148] Clause 8. One or more processors according to Clause 1, 2, 3 or 4, wherein the one or more logical evaluations of the one or more logical statements represent at least one of the following: a proof or refutation of at least one of the one or more input statements, a fact inferred based on at least one of the one or more input statements, or a consistency check of at least one of the one or more input statements.

[0149] Clause 9. One or more processors according to Clause 1, 2, 3 or 4, wherein the processing circuitry is further configured to: generate the representation of the one or more logic evaluations using one or more hints, said one or more hints being generated using retrieval enhancement generation and enhanced with a representation that interprets the content of the logic specification language.

[0150] Clause 10. One or more processors as described in Clauses 1, 2, 3 or 4, wherein the natural language response includes an interpretation of the one or more logical evaluations, or an answer to a question associated with the one or more input statements based at least on the one or more logical evaluations.

[0151] Clause 11. One or more processors according to Clause 1, 2, 3 or 4, wherein the one or more input statements represent a sequence of image data of a video, wherein the natural language response includes at least one of: an interpretation of at least a portion of the video or an answer to a question about at least a portion of the video.

[0152] Clause 12. One or more processors according to Clauses 1, 2, 3 or 4, wherein the one or more input statements represent a sequence of time series data, wherein the natural language response includes at least one of: an interpretation of at least a portion of the time series data or an answer to a question about at least a portion of the time series data.

[0153] Clause 13. One or more processors as described in Clauses 1, 2, 3, or 4, wherein said processing circuitry is included in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing analog operations; a system for performing digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation of 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system for generating synthetic data using AI; a system comprising one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

[0154] Clause 14. A system comprising one or more processors, said one or more processors being configured to: generate a representation of one or more logical evaluations expressed in a logical canonical language of said one or more natural language statements, based at least on a translated representation of said one or more natural language statements processed using a logical reasoning engine.

[0155] Clause 15. The system according to Clause 14, wherein the one or more processors are further configured to: generate the translated representation of the one or more natural language statements at least based on inserting the one or more natural language statements into one or more template prompts, the one or more template prompts instructing one or more large language models to convert the one or more natural language statements into the logical canonical language.

[0156] Clause 16. The system according to Clause 15, wherein the one or more processors are further configured to: generate the translated representation of the one or more natural language statements using one or more large language models adapted to be applicable to the logical canonical language.

[0157] Clause 17. The system according to Clause 15, wherein the one or more processors are further configured to: generate the representation of the one or more logical evaluations based at least on the translated representation of the one or more natural language statements processed using the logical reasoning engine.

[0158] Clause 18. The system according to Clause 15, wherein the one or more logical evaluations of the one or more natural language statements represent at least one of the following: a proof or refutation of at least one of the one or more natural language statements, a fact inferred based on at least one of the one or more natural language statements, or a consistency check of at least one of the one or more natural language statements.

[0159] Clause 19. The system according to Clause 15, wherein the one or more processors are further configured to: generate the representation of the one or more logical evaluations using one or more hints, the one or more hints being generated using retrieval enhancement generation and enhanced with a representation that interprets the content of the logical canonical language used by the logical reasoning engine.

[0160] Clause 20. The system according to Clause 15, wherein the one or more processors are further configured to: generate a natural language response based at least on the one or more logical evaluations, based at least on the representation processed using one or more large language models.

[0161] Clause 21. A system pursuant to Clause 15, wherein said system comprises at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulated operations; a system for performing digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation of 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system implementing one or more visual language models (VLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system containing one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

[0162] Clause 22. A method comprising: obtaining a representation of one or more logical evaluations of the one or more statements based at least on a translated representation of the one or more statements processed using a logic reasoning engine in a particular language.

[0163] Clause 23. The method according to Clause 22 further comprises: obtaining a response for the one or more statements using the particular language or at least one other language, based at least on the representation of the one or more logical evaluations processed using one or more large language models.

[0164] Clause 24. The method according to Clause 22 or 23, wherein said method is performed by at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulated operations; a system for performing digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation of 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system for generating synthetic data; a system for generating synthetic data using AI; a system comprising one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources.

Claims

1. One or more processors, including processing circuitry, said processing circuitry being used to: Based at least on the representation of one or more input statements processed using one or more large language model LLMs, generate one or more logical statements expressed and represented in a logical canonical language; Using a logic reasoning engine corresponding to the logic specification language, generate a representation of one or more logical evaluations of the one or more logical statements; A natural language response is generated for the one or more input statements, based at least on the representations processed by the one or more LLMs for the one or more logical evaluations. as well as The natural language response is provided as output, wherein the one or more input statements include a representation of the query, and the natural language response includes at least one of the following: an explanation of at least a portion of the query or an answer to a question about at least a portion of the query.

2. The processor of claim 1, wherein the processing circuitry is further configured to: generate the representation of the one or more input statements at least based on inserting the one or more input statements into one or more template prompts, the one or more template prompts instructing the one or more LLMs to convert the one or more input statements into the logic specification language.

3. The processor of claim 1, wherein the processing circuitry is further configured to: generate the one or more logic statements using a first large language model of the one or more LLMs, wherein the first large language model is adapted to fit the logic specification language.

4. The processor of claim 1 or more, wherein the logic reasoning engine implements one or more solvers.

5. The processor of claim 1, wherein the one or more logical evaluations of the one or more logical statements represent at least one of the following: a proof or refutation of at least one of the one or more input statements, a fact inferred based on at least one of the one or more input statements, or a consistency check of at least one of the one or more input statements.

6. The processor of claim 1 or more, wherein the processing circuitry is further configured to: generate the representation of the one or more logic evaluations using one or more hints, the one or more hints being generated using retrieval enhancement generation and enhanced with a representation that interprets the content of the logic specification language.

7. The processor of claim 1, wherein the natural language response includes an interpretation of the one or more logical evaluations, or an answer to a question associated with the one or more input statements based at least on the one or more logical evaluations.

8. The processor of claim 1 or more, wherein the one or more input statements represent a sequence of image data of a video, wherein the natural language response includes at least one of the following: an explanation of at least a portion of the video or an answer to a question about at least a portion of the video.

9. The processor of claim 1 or more, wherein the one or more input statements represent a sequence of time series data, wherein the natural language response includes at least one of the following: an explanation of at least a portion of the time series data or an answer to a question about at least a portion of the time series data.

10. One or more processors according to claim 1, wherein the processing circuitry is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system for generating synthetic data; Systems for generating synthetic data using AI; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

11. A system comprising one or more processors, said one or more processors being configured to: generate a representation of one or more logical evaluations expressed in a logical canonical language of said one or more natural language statements, based at least on a translated representation of said one or more natural language statements processed using a logical reasoning engine.

12. The system of claim 11, wherein the one or more processors are further configured to: generate the translated representation of the one or more natural language statements at least based on inserting the one or more natural language statements into one or more template prompts, the one or more template prompts instructing one or more large language models to convert the one or more natural language statements into the logical canonical language.

13. The system of claim 11, wherein the one or more processors are further configured to: generate the translated representation of the one or more natural language statements using one or more large language models adapted to fit the logical canonical language.

14. The system of claim 11, wherein the one or more processors are further configured to: generate the representation of the one or more logical evaluations based at least on the translated representation of the one or more natural language statements processed using the logical reasoning engine.

15. The system of claim 11, wherein the one or more logical evaluations of the one or more natural language statements represent at least one of the following: a proof or refutation of at least one of the one or more natural language statements, a fact inferred based on at least one of the one or more natural language statements, or a consistency check of at least one of the one or more natural language statements.

16. The system of claim 11, wherein the one or more processors are further configured to: generate the representation of the one or more logical evaluations using one or more hints, the one or more hints being generated using retrieval enhancement generation and enhanced with a representation that interprets the content of the logical canonical language used by the logical reasoning engine.

17. The system of claim 11, wherein the one or more processors are further configured to: generate a natural language response based at least on the one or more logical evaluations, based at least on the representations processed using one or more large language models.

18. The system of claim 11, wherein the system comprises at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more language models; A system that implements one or more large language model LLMs; A system that implements one or more Visual Language Models (VLMs); A system for generating synthetic data; Systems for generating synthetic data using AI; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

19. A method comprising: Based at least on the translated representation of one or more statements in a specific language processed using a logic reasoning engine, a representation of one or more logical evaluations of the one or more statements is obtained; as well as Based at least on the representation of the one or more logical evaluations processed using one or more large language models, a response in the specific language or at least one other language is obtained for the one or more statements.

20. The method of claim 19, wherein the method is performed by at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for collaborative content creation of 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system for generating synthetic data; Systems for generating synthetic data using AI; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

Images