generating a response to a query using one or more neural networks

Abstract

This disclosure relates to generating responses to queries using one or more neural networks. Apparatus, systems, and techniques for generating responses to one or more user queries using one or more neural networks are also provided. In at least one embodiment, a processor includes a processing circuitry system configured to generate computer program code using one or more neural networks to obtain performance information of one or more programs, at least in part, based on user queries instructing one or more programs executed by one or more graphics processing units (GPUs), and to execute the computer program code to generate one or more responses to the one or more user queries.

Description

[0001] Cross-references to related applications This application claims the benefit of Indian Provisional Application No. 202511023275, filed on March 16, 2025, entitled “GENERATING RESPONSES TOQUERIES USING ONE OR MORE NEURAL NETWORKS”, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0002] At least one embodiment relates to processing resources for using one or more neural networks to generate one or more responses to one or more queries. Background Technology

[0003] In computing systems with graphics processing units (GPUs), optimizing GPU performance is crucial for maintaining high performance and efficiency. Optimizing GPU performance can be a daunting task for users because GPU programs can be very complex and it can be difficult to pinpoint the optimal points. Improvements can be made to the amount of memory, time, or computing resources used to optimize GPU performance. Attached Figure Description

[0004] Figure 1 This is a block diagram of an example system that uses one or more neural networks to generate one or more responses according to at least one embodiment; Figure 2 This is a block diagram of an example system that generates one or more answers using a generic agent and / or a reporting agent according to at least one embodiment; Figure 3 This is an example of a user interface according to at least one embodiment; Figure 4 It is a block diagram of a system that performs information retrieval using one or more neural networks according to at least one embodiment; Figure 5 This is a flowchart illustrating a method for providing one or more answers to a prompt, according to at least one embodiment; Figure 6 This is a flowchart illustrating a method of providing one or more answers using a generic agent, according to at least one embodiment; Figure 7 This is a flowchart illustrating a method of providing one or more answers using a reporting agent, according to at least one embodiment; Figure 8A This is a block diagram of an example generative language model system applicable to implementing at least some embodiments of the present disclosure; Figure 8B This is a block diagram of an example generative language model including a converter encoder-decoder, applicable to implementing at least some embodiments of this disclosure; Figure 8C This is a block diagram of an example generative language model including a decoder-only converter architecture suitable for implementing at least some embodiments of this disclosure; Figure 9 This is a block diagram of an example computing device applicable to implementing at least some embodiments of the present disclosure; and Figure 10 This is a block diagram of an example data center applicable to implementing at least some embodiments of the present disclosure. Detailed Implementation

[0005] In one example, the technique described herein improves the performance of a graphics processing unit (GPU) in a computing system by generating code using one or more large language models (LLMs) based on a user query about GPU performance to obtain information, thereby generating a response to the user query; enabling the implementation or otherwise execution of that code to obtain the information; and using that information to generate a response. This addresses a challenge that users may face when optimizing GPU performance, as they may want to improve the performance of programs that use the GPU and may also have access to GPU performance data but may lack appropriate guidance for optimization. The technique described herein improves the process by which users optimize the performance of their programs that run on the GPU by: obtaining a question from the user about the performance of their program; generating specific code that can be used to obtain information to generate a response to that question, for example by extracting information from GPU performance data; executing that specific code to obtain the information; and using that information to generate specific guidance or analysis relevant to the user and their program. In this way, the user can obtain guidance on how to optimize the performance specific to their program and the GPU.

[0006] The techniques described herein may involve computing systems comprising one or more LLMs. Such computing systems may include one or more agents that implement one or more LLMs and provide a user interface that allows users to provide queries about the performance of their programs. These queries may include general questions about usage and metrics; general questions about GPU architecture and programming models; program-specific questions about performance bottlenecks; program-specific questions about optimization suggestions to resolve bottlenecks; general or program-specific questions about occupancy; performance comparison queries (e.g., given two cores and their reports, calculate the speedup and determine the root cause of the performance difference); and / or any appropriate queries related to one or more GPUs and one or more programs.

[0007] The computational system can enable one or more agents to process queries using one or more LLMs to generate responses to those queries. Agents can generate scripts to extract performance information from performance reports to answer queries. Agents can leverage various tools (such as those described herein) and perform a Retrieval Enhancement Generation (RAG) process to retrieve context and reduce illusions, thereby generating responses. The computational system can implement or otherwise utilize a tool that generates versions of source programs tagged with profile information to provide more source-level context, thus answering more general questions about program performance, including performance bottlenecks, root causes of performance problems, and optimization recommendations that specify which changes to be made to the kernel in question to improve its performance. The computational system can enable scripts to be executed to obtain performance information to answer queries. In some examples, performance information can be analyzed by other agents or neural networks, and further analysis can be provided as part of the response. These agents can also answer general user queries about GPU hardware and software by searching GPU documentation and the internet, and can also provide optimization recommendations based on performance information.

[0008] In the foregoing and following description, numerous specific details have been set forth in order to provide a more complete understanding of at least one embodiment. However, those skilled in the art will understand that the inventive concept can be practiced even without one or more of these specific details.

[0009] refer to Figure 1 , Figure 1 This is an example system 100 that uses LLM to generate one or more responses according to some embodiments of this disclosure. It should be understood that this and other arrangements described herein are merely examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groupings, etc.) may be used to supplement or replace the arrangements and elements shown, and certain elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in combination with other components, and can be implemented in any suitable combination and in any suitable location. The various functions performed by the entities described herein can be implemented by hardware, firmware, and / or software. For example, various functions can be implemented using one or more processors executing instructions stored in one or more memories. For example, in some embodiments, the systems and methods described herein may use one or more generative language models (e.g., such as...). Figures 8A-8C The above), one or more computing devices or components thereof (e.g., such as Figure 9 (as described above) and / or one or more data centers or components thereof (e.g., such as...) Figure 10 This is achieved through (as described above).

[0010] Figure 1 This is a block diagram of an example system 100 that generates one or more responses using a large language model (LLM) according to at least one embodiment. System 100 may include one or more prompts 104 as user input 102, a chat agent 106, one or more analytics tools (e.g., a profiler 108, a occupancy calculator 112, a language processor 114, a math tool 116, and / or one or more tools described herein), one or more kernels 110, one or more outputs to the user 118 (e.g., responses 120), one or more components described herein, and / or one or more combinations thereof.

[0011] System 100 may include an LLM-based chat interface for one or more computational profiling tools. For example, chat agent 106 (also referred to as an agent, chatbot, chatbot agent, and / or variations thereof) generates one or more answers as responses 120 to one or more user inputs 102 (e.g., prompts 104 and / or queries). Chat agent 106 may receive one or more user inputs 102, which may include one or more prompts 104. Chat agent 106 may implement or otherwise associate with one or more user interfaces, which users can use to provide user input 102, such as by entering text, audio, and / or variations thereof. In some embodiments, chat agent 106 is implemented as part of a software program that is executed using computer hardware.

[0012] Tip 104 may include one or more queries, such as tool usage-related queries (e.g., queries related to analyzer tools), GPU-related questions, kernel-specific tips, and / or any appropriate queries regarding the performance of one or more GPUs, kernels, programs, and / or their variants. As an example, performance information may include information from one or more performance reports. For instance, tool usage-related queries may include queries related to tool support for specific programming model features, configuration options, and / or error handling. As an example, GPU-related questions may include questions about metric interpretation, architectural details, and / or the capabilities of one or more parallel computing platforms. Kernel-specific tips (e.g., queries) may include questions related to profiling reports, such as occupancy limiters, one or more program hotspots, GPU utilization, and / or per-instruction metrics. For instance, kernel-specific tips may include queries related to requests for optimization recommendations tailored to a specific kernel and / or application.

[0013] Chat agent 106 may include or otherwise implement one or more agents. System 100 may utilize an agent workflow that may include one or more agents, each of which includes or otherwise implements an LLM, system prompts, and / or one or more tools to assist in answering user queries. Chat agent 106 may include: one or more tools, such as: Retrieval Enhancement Generation (RAG) for potentially reducing illusions by obtaining context relevant to analyzer documentation, programming model guides, reporting interface examples and API documentation, and / or GPU architecture documentation; web search tools for obtaining relevant context from blogs and / or developer forums (e.g., web search tool 122); and / or a occupancy calculator; and a REPL for a reporting interface based on a computational language (e.g., Python). Chat agent 106 may receive one or more user inputs 102, such as prompts 104 (e.g., queries). The tools described herein may be implemented as part of a software program, module, service, and / or variations thereof.

[0014] Users can use one or more GPUs to execute one or more programs (e.g., kernel 110) and obtain profiling reports related to the performance of one or more programs from profiler 108. In some examples, profiler 108 may be automatically invoked as part of the use of system 100, and profiling reports can be obtained from profiler 108. Profiling reports can be any suitable information related to the performance of one or more programs executed via one or more GPUs, such as one or more performance reports, and can be implemented using any suitable data or file format. To extract information from profiling reports, chat agent 106 (e.g., LLM) can write and execute computational language scripts using an interface called a reporting interface based on a computational language. Profiling reports can be generated by GPU performance monitoring / optimization software or through any suitable process. Since this may be a resource-constrained API, chat agent 106 (e.g., one or more LLMs) may not have been trained on its use. Therefore, system 100 may rely on contextual learning techniques, such as few-shot hints, and provide API usage guidelines as part of the context. In some embodiments, the chat agent 106 is trained using sample scripts to generate code that obtains information from a profile report based on user input 102.

[0015] To provide optimization suggestions, chat agent 106 can analyze the kernel and profile to identify bottlenecks and recommend solutions, including one or more suggestions for using library functions and / or certain existing programming models / GPU capabilities. System 100 can employ a workflow that involves using one or more agents in a supervisor-based network, where each agent focuses on a different task, and the supervisor routes queries to the appropriate expert. For example, different agents can be set up for general queries and kernel-specific queries. In this scenario, the router or supervisor agent can first classify the query and then route it to the appropriate agent, which invokes the necessary tools to generate a response. Furthermore, the supervisor agent can provide constraints by filtering out one or more queries that are irrelevant to the analyzer. Agents can respond to one or more user queries following a ReAct-style iterative "think-act-observe" process, where user queries can be tailored to the user's expertise (e.g., novice users might prefer a more detailed response including background, metrics, and terminology definitions, while expert users might prefer a concise response).

[0016] In at least one embodiment, the analyzer 108 receives one or more core and / or telemetry data (e.g., date and / or timestamps, system performance metrics, user interactions, error reports, and other relevant operational data) as input. As an example, the analyzer 108 may receive performance information to include one or more resource usage measurements of one or more GPUs obtained from executing one or more programs. In at least one embodiment, the analyzer 108 is one or more tools that may incorporate or otherwise implement an LLM and may be implemented as part of a software program. The chat agent 106 may generate one or more responses 120, allowing the quality of one or more responses 120 to be evaluated using a custom evaluation set constructed from expert knowledge of tools (e.g., usage calculator 112, language processor 114, mathematical tools 116, and / or analyzer 108), and to predict one or more user queries. Truth-valued answers can be manually collected using one or more results from the analyzer 108 and information from available documents.

[0017] System 100 can invoke analyzer 108 to obtain a profiling report related to specific kernel and GPU performance based on prompt 104 indicating a query about kernel and GPU performance, and generate code to analyze this report. Analyzer 108 can serve as a tool to analyze and optimize one or more programs by providing detailed insights into GPU resource usage and / or performance bottlenecks, such as information on GPU resource utilization and / or pause causes for each instruction. Chat agent 106 can use this data to generate output that identifies one or more hotspots in the program and suggests one or more optimizations.

[0018] Chat agent 106 may also receive one or more outputs from occupancy calculator 112. In at least one embodiment, occupancy calculator 112 is a tool that may include or otherwise implement an LLM and may be implemented as part of a software program. Occupancy calculator 112 can serve as a tool to assist in calculating occupancy information for one or more programs, such as a measure of the effective utilization of GPU resources, which GPU resources are used, and / or any suitable information related to the performance of one or more programs. Occupancy calculator 112 may receive one or more inputs, such as one or more block sizes, register usage, and / or shared memory usage, to perform one or more calculations. Occupancy calculator 112 can be accessed through an interface, such as a Python interface or any suitable interface, allowing chat agent 106 to combine information from occupancy calculator 112 with information from other analysis tools, such as language processor 114, mathematical tools 116, and / or analyzer 108. The occupancy calculator 112 may output one or more pieces of information to the chat agent 106, such as one or more detailed reports on occupancy levels, to help developers identify potential areas for optimization, thereby improving the performance of one or more programs.

[0019] Chat agent 106 may also receive one or more outputs from language processor 114. In at least one embodiment, language processor 114 is a tool that may include or otherwise implement an LLM and may be implemented as part of a software program. Language processor 114 may refer to a component in system 100 designed to execute one or more scripts (e.g., Python scripts or any suitable scripts in any suitable programming language), possibly as part of generating response 120. Language processor 114 may receive one or more inputs, such as scripts generated to extract information from data files or reports (e.g., data files or reports generated by analysis tools). Language processor 114 may execute or otherwise implement these scripts locally, enabling system 100 to retrieve and manipulate data without sharing sensitive information externally. One or more outputs from language processor 114 to chat agent 106 may include processed data or results obtained by executing one or more scripts, which can then be used to inform further actions or decisions in system 100. Chat agent 106 may utilize information from mathematical tool 116 (which may be a tool for performing one or more mathematical operations) and / or web search tool 122 (which may be a tool for performing one or more internet searches and / or retrieving information through one or more web searches).

[0020] In at least one embodiment, system 100 includes a collection of one or more hardware and / or software computing resources containing instructions that, when executed, implement the processes described herein. In at least one embodiment, system 100 is a software program running on computer hardware, an application program running on computer hardware, and / or variations thereof. In at least one embodiment, one or more processes of system 100 are executed by any suitable processing system or unit (e.g., graphics processing unit (GPU), general-purpose GPU (GPGPU), parallel processing unit (PPU), central processing unit (CPU)), data processing unit (DPU), as described below, in any suitable manner, including sequential execution, parallel execution, and / or variations thereof. In at least one embodiment, system 100 uses machine learning training frameworks, such as PYTORCH, TENSORFLOW, BOOST, CAFFE, MICROSOFT COGNITIVE TOOLKIT / CNTK, MXNET, CHAINER, KERAS, DEEPLEARNING4J, and / or other training frameworks, to implement and perform the operations described herein.

[0021] In at least one embodiment, one or more components of system 100 are implemented using any combination of any type of logic (e.g., software, hardware, firmware) and / or circuitry configured to perform the functions described herein. In at least one embodiment, the software includes software packages, code, programming languages, drivers, instructions, instruction sets, or some combination thereof. In at least one embodiment, the hardware includes hardwired circuitry, programmable circuitry, state machine circuitry, fixed-function circuitry, execution unit circuitry, firmware with stored instructions executed by programmable circuitry, or some combination thereof. Furthermore, it should be noted that while the various embodiments described herein may relate to or be specific to LLM, any suitable neural network, model, algorithm, machine learning model, process, and / or variations thereof may be used. Additionally, the LLM described herein may refer to any suitable model implementation for generating or otherwise processing text, such as transformer-based models, RNN-based models, Seq2seq models, hybrid models, fine-tuning models, multimodal models, and / or variations thereof, which may be implemented using any suitable data structures, instructions, and / or variations thereof.

[0022] System 100 may include one or more processors, such as one or more processors included in at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing analog operations; a system for performing digital twin operations; a system for performing optical transmission simulations; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using edge devices; a system implemented using robots; a system for performing conversational artificial intelligence operations; a system implementing one or more multimodal language models; a system implementing one or more large language models (LLMs); a system implementing one or more visual language models (VLMs); a system for generating synthetic data; a system for generating synthetic data using AI; a system containing one or more virtual machines (VMs); a system implemented at least partially in a data center; a system implemented at least partially using cloud computing resources; and / or combinations thereof.

[0023] In at least one embodiment, system 100 includes one or more processors for generating computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by one or more graphics processing units (GPUs); executing said computer program code to generate one or more responses to said one or more user queries; and / or otherwise performing the operations described herein. In at least one embodiment, system 100 is Figure 1-10 The systems shown herein, included in, and / or otherwise included in, are used to generate computer program code using one or more neural networks to obtain performance information of one or more programs based at least in part on one or more user queries instructing one or more programs to be executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to the one or more user queries; and / or otherwise perform the operations described herein.

[0024] In at least one embodiment, system 100 performs Figure 1-10The illustrated one or more processes are used, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, system 100 includes Figure 8A-10 The one or more hardware shown may be used, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by said one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein.

[0025] refer to Figure 2 , Figure 2 This is an example system 200 for generating one or more responses using one or more LLMs, according to some embodiments of this disclosure. It should be understood that this and other arrangements described herein are merely illustrated as examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groups, etc.) may be used to supplement or replace the illustrated arrangements and elements, and certain elements may be omitted. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in combination with other components, and can be implemented in any suitable combination and at any suitable location. The various functions performed by the entities described herein can be implemented in hardware, firmware, and / or software. For example, various functions can be implemented using one or more processors executing instructions stored in one or more memories. For example, in some embodiments, the systems and methods described herein may use one or more generative language models (e.g., such as...). Figures 8A-8C The above), one or more computing devices or components thereof (e.g., such as Figure 9 (as described above) and / or one or more data centers or components thereof (e.g., such as...) Figure 10 This is achieved through (as described above).

[0026] Figure 2This is a block diagram of an example system 200 that generates one or more answers using a generic agent and / or a reporting agent according to at least one embodiment. System 200 may include one or more user prompts 202, one or more routers 204, one or more generic agents 206, one or more vector repositories 208 and / or 216, one or more tools 210 and / or 218, one or more answer outputs 212 and / or 220, one or more reporting agents 214, one or more components described herein, and / or combinations thereof.

[0027] System 200 may provide or otherwise implement a user interface in which users can provide performance analysis-related queries, such as via text or voice. The system may utilize models (e.g., the GPT-4-Turbo model), agents (e.g., React agents), occupancy calculators, language processors, and / or mathematical tools, and / or any suitable model or agent. One or more inputs to System 200 may include a detailed report page containing GPU resource utilization and SOL information, performance counters, one or more throughput matrices, and / or a source page that provides the reason for the pause for each instruction.

[0028] The source page may be difficult for LLMs to process, so system 200 can utilize one or more neural networks (e.g., GPT-4-Turbo) to generate programs (e.g., computer program scripts) to extract relevant information, execute the program using language processor tools, and then use the information to answer one or more queries. Furthermore, system 200 can implement an optimized agent flow for profiling guidance, which can suggest optimizations.

[0029] System 200 may receive one or more user inputs 102, such as user prompts 202, which contain one or more queries (e.g., queries related to tool use and / or GPU-related questions). System 200 may include router 204, which may be a software program, module, or component that receives one or more user prompts 202, for example, through one or more user interfaces. For example, if router 204 determines that user prompt 202 is relevant to a specific profile result, router 204 may forward user prompt 202 to reporting agent 214. As another example, if router 204 determines that user prompt 202 is irrelevant to a specific profile result, router 204 may forward user prompt 202 to general agent 206. Router 204 may act as a decision component in system 200, for example, directing user prompts 202 (e.g., queries) to the appropriate processing agent. Router 204 may receive one or more inputs, such as one or more user prompts or profile results, which can be used to determine the nature of the query, whether it is report-related or general. Router 204 can operate by first checking for the existence of a profile result and then routing the query to a report-specific or general agent based on that. One or more response outputs from router 204 may include the category of the query and the subsequent route to the relevant agent, thereby ensuring that system 200 can efficiently process and respond to one or more user queries.

[0030] System 200 may include one or more report agents 214, for example, in series and / or parallel. Report agents 214 may function as dedicated components within system 200, for example, for processing report-related queries (e.g., those involving performance analysis and / or optimization). Report agents 214 may receive one or more inputs, such as one or more user prompts and one or more profiling results, which may contain detailed data from GPU analysis reports. Report agents 214 may operate by extracting relevant information from profiling reports to obtain profiling results information using a programming language reporting interface (e.g., the Python Reporting Interface (PRI) or any suitable interface), and may generate and execute computer program scripts to do so. As an example, report agents 214 may access one or more vector repositories 216, for example, by using one or more retrievers and / or one or more tools 218, such as language processors, occupancy calculators, and / or other tools described herein. One or more answer outputs 220 from report agents 214 may include detailed answers to user queries, providing insights into GPU resource usage, bottlenecks, and / or optimization opportunities using the extracted data. Reporting agent 214 may be included in a larger agent network (e.g., general agents and / or reporting agents), with each agent undertaking a specific task in processing and responding to one or more user queries.

[0031] System 200 may include one or more general agents 206, such as those connected in series and / or parallel. General agents 206 may be components of system 200 used to process queries unrelated to reporting and may provide users with information about various programming models and general GPU concepts. General agents 206 may receive one or more inputs, such as one or more user hints that do not contain specific profile results, allowing them to focus on a broader range of issues. General agents 206 may operate by leveraging retrieved augmented generation (RAG) techniques, accessing resources such as documentation (e.g., programming model documentation, hardware documentation, technical blog posts, video tutorials, and / or variations thereof) to generate one or more responses. As an example, general agent 206 may access one or more vector repositories 208 (e.g., processor analyzer documentation and / or programming guides), for example, by using one or more retrievers and / or one or more tools 210 (e.g., developer forum searches and / or other tools described herein). One or more answer outputs 212 from the general agent 206 may include one or more answers to one or more user queries, such as those unrelated to a specific analytics report, providing insights into tool usage, GPU architecture, and / or programming models. The general agent 206 can work in conjunction with other agents in the network to ensure comprehensive coverage of user queries.

[0032] In at least one embodiment, system 200 includes one or more processors configured to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, system 200 is Figure 1-10 The system shown is included Figure 1-10 The system shown, and / or otherwise includes Figure 1-10 The system shown is used to generate computer program code using one or more neural networks to obtain performance information of one or more programs based at least in part on one or more user queries instructing one or more programs to be executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to the one or more user queries; and / or otherwise perform the operations described herein.

[0033] In at least one embodiment, system 200 performs Figure 1-10The processes shown herein, for example, are used to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, system 200 includes Figures 8A to 10 The one or more hardware shown may be used, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by said one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein.

[0034] refer to Figure 3 , Figure 3 This is an example of a user interface 300 according to some embodiments of this disclosure. It should be understood that this and other arrangements described herein are merely examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groups, etc.) may be used to supplement or replace the illustrated arrangements and elements, and certain elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in combination with other components, and can be implemented in any suitable combination and in any suitable location. The various functions performed by the entities described herein can be implemented using hardware, firmware, and / or software. For example, various functions can be implemented using one or more processors executing instructions stored in one or more memories. For example, in some embodiments, the systems and methods described herein may use one or more generative language models (e.g., such as...). Figures 8A-8C The above), one or more computing devices or components thereof (e.g., such as Figure 9 (as described above) and / or one or more data centers or components thereof (e.g., such as...) Figure 10 This is achieved through (as described above).

[0035] Figure 3This is an example of a user interface 300 according to at least one embodiment. The user interface 300 may be implemented by or otherwise provided by the systems described herein (e.g., systems 100, 200, and / or 400). The system may provide an interface for a user to interact with a chatbot via a chat tool window, which may be integrated into a user interface (e.g., a user interface for GPU performance monitoring / optimization software), possibly implemented in Qt using C++, as part of an application library, or implemented in any suitable manner. The system using user interface 300 may provide a generic API that allows the front-end user interface 300 to communicate with a back-end application, representing the client side of client / server communication. Execution of the system may be performed locally on the host side. One or more inputs to the system using the user interface may include one or more user queries and profile results, which can be processed to generate one or more responses that are returned to the user via user interface 300.

[0036] To integrate the chat window into the user interface 300, systems (e.g., systems 100, 200, and / or 400) can use a tool window with general search functionality by adding a "Chat" tab. This "Search" tool window of the user interface 300 provides access to general "Search" and "Chat" bots. The user interface 300 may include a chat window for displaying a history of user-provided queries and their AI-generated responses in chronological order, with each query visually separated from one or more responses. One or more inputs to the system using the user interface 300 may include human prompts and / or relevant context, such as one or more summary results. One or more outputs from the chat window of the user interface 300 may include one or more formatted responses, one or more options for retrying or voting on responses, and / or functionality for saving chat history.

[0037] Repeated user prompts in user interface 300 can provide context for one or more answers, such as when rereading the chat log at a later stage. On the other hand, relevant summary results can record what inputs were provided to the chatbot when generating answers. The generated responses may include formatted responses and / or instructions for processing displayed in a stream, as well as "Retry," "Agree," "Disagree," and / or "Save" buttons. This setup of user interface 300 enhances user interaction by allowing users to effectively interact with the chatbot and provide feedback on one or more responses.

[0038] Because generating answers using a large language model (LLM) can take a certain amount of time, some answers may be displayed in a streaming manner, allowing the user to cancel the request once satisfied with the response. LLMs typically also use Markdown syntax to produce output, which can be rendered using Qt's rich text processing capabilities or other suitable methods. User interface 300 can indicate whether processing is still in progress. Given the non-deterministic nature of LLM execution, a "Retry" button provides the ability to generate new, potentially more helpful answers. This button in user interface 300 can also be used to "rewind" the conversation to a previous question, deleting chat history of queries and responses that may have occurred during the process. User interface 300 can always prompt the user for confirmation when using the "Retry" button to ensure that potentially valuable answers are not accidentally deleted. User interface 300's vote-for / vote buttons can be used to track user satisfaction with one or more generated responses. The "Save" button allows the user to save the chat history to a specific point in a text format (e.g., .txt or .md).

[0039] One or more buttons in the user interface 300 can be extended. For example, an "Artifacts" button can be added, allowing users to save one or more scripts (e.g., Python scripts) generated by the LLM backend during response generation. In addition to the query and response lists, the chatbot tool window of the user interface 300 can also support multiple chats running in parallel, display a "Start Screen" for new conversations, report errors and connection status, and / or optionally retain chat history between restarts. This setup of the user interface 300 enhances user interaction by allowing users to manage multiple conversations and maintain continuity between sessions. In some examples, a document list or "Documents" button may be included to indicate or otherwise display documents or information sources (e.g., links or other citations or identifiers) used to generate answers.

[0040] User interface 300 supports multiple chats, each with its own chat history and context. This allows users to ask questions about different reports and topics without overwhelming the LLM with excessive contextual information. The "Start" or "Default" screen of user interface 300 can be used to provide users with basic information, such as links to the chatbot's documentation, basic instructions, and / or suggestions for initial user prompts. User interface 300 may also include information on how to select and / or change the reports of interest. Furthermore, users can have clear indications in user interface 300 regarding the availability of a connection to the LLM backend and can be informed if the backend has encountered errors or timeouts. Retaining the chat history of user interface 300 between restarts (e.g., reloads) is a feature when using the application in conjunction with one or more projects. This feature in user interface 300 allows users to retrieve the previous state of their analysis and find information they previously queried from the chatbot. In at least one embodiment, user interface 300 is and / or otherwise includes user interface 408. It should be noted that... Figure 3 It can be an illustrative example of one or more queries that can be provided through the user interface, and the query can be any suitable query, such as the query described in this article, such as queries related to overview reports, analysis, optimization and / or their variations.

[0041] In at least one embodiment, the system using user interface 300 includes one or more processors configured to: generate computer program code using one or more neural networks to obtain performance information of one or more programs based at least in part on one or more user queries instructing one or more programs executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to the one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, the system using user interface 300 is Figure 1-10 The system shown is included Figure 1-10 The system shown, and / or otherwise includes Figure 1-10 The system shown is configured to: generate computer program code using one or more neural networks to obtain performance information of one or more programs based at least in part on one or more user queries instructing one or more programs executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to the one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, the system uses user interface 300 for execution. Figure 1-10The illustrated one or more processes are used, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, the system using user interface 300 includes Figure 8A-10 The one or more hardware shown may be used, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by said one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein.

[0042] refer to Figure 4 , Figure 4 This is an example system 400 for generating one or more responses using LLM, according to some embodiments of this disclosure. It should be understood that this and other arrangements described herein are merely illustrated as examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groups, etc.) may be used to supplement or replace the illustrated arrangements and elements, and certain elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in combination with other components, and can be implemented in any suitable combination and in any suitable location. The various functions performed by the entities described herein can be implemented by hardware, firmware, and / or software. For example, various functions can be implemented using one or more processors executing instructions stored in one or more memories. For example, in some embodiments, the systems and methods described herein may use one or more generative language models (e.g., such as...). Figures 8A-8C The above), one or more computing devices or components thereof (e.g., such as Figure 9 (as described above) and / or one or more data centers or components thereof (e.g., such as...) Figure 10 This is achieved through (as described above).

[0043] Figure 4This is a block diagram of a system 400 that performs information retrieval using one or more neural networks according to at least one embodiment. System 400 may include one or more users 402, one or more user interfaces 408, one or more GPU clouds 404, one or more LLMs 404A stored in the GPU cloud (e.g., remote storage), one or more web resources 404B stored in the GPU cloud (e.g., remote storage), telemetry information 406 stored in remote storage, one or more user interfaces 408, and one or more chat system backends 410 (e.g., system 200, see also...). Figure 2 Local storage and / or disk 412 (e.g., one or more setup API keys 412A, one or more reports 412B, one or more documents 412C and / or one or more language processors and / or other tools 412D), local storage 414 (e.g., chat history 416), the components described herein, and / or combinations thereof. In at least one embodiment, the chat system backend 410 is or otherwise includes system 200 (see...) Figure 2 In at least one embodiment, user interface 408 is or otherwise includes user interface 300 (see [link to documentation]). Figure 3 ).

[0044] System 400 may include one or more backend applications for implementing, for example, but not limited to, business logic chatbots. System 400 may implement a network of one or more LLM agents that collaboratively provide answers to user questions by retrieving one or more reports and architecture-specific information from documentation and profile-specific results. System 400 may utilize various libraries, such as one or more libraries for natural language processing (e.g., LangChain and / or LangGraph), which can simplify LLM invocation, prompt building, tool invocation, and / or RAG implementation. System 400 may also include an LLM architecture for remotely communicating with inference microservices using libraries such as natural language processing libraries (e.g., the langchain-nvidia-ai-endpoints library or any suitable library) and locally communicating with a frontend via one or more interfaces (e.g., gRPC), representing the server side of the latter's communication.

[0045] User 402 may ask one or more questions related to a specific profiling result and / or general questions about computing units, architecture, and / or hardware, and provide said one or more questions through user interface 408, for example, via text or audio input. User 402 may include the profiling result as context in their prompts. A valid user 402 request may include questions or prompts, and optional profiling results. If no profiling result is included, the request may be treated as a general question or a “non-report-related” question and routed to a general reporting broker. However, if a profiling result is included, user 402 may still ask a non-report-related question. To handle this, the question may initially be treated as a “report-related” question, and if no answer is found in the first step, it will be steered to the non-report-related subgraph of the LLM network (e.g., chat system backend 410).

[0046] System 400 may include one or more local storage, disk 412, and / or local storage 414 in local (e.g., host) memory. In at least one embodiment, the local storage and / or disk 412 stores (e.g., one or more API setting API keys 412A, as described herein, for one or more APIs and libraries), one or more reports 412B (e.g., overview reports as described herein), one or more documents 412C (e.g., programming model documents), and / or one or more tools 412D (e.g., language processor 114, usage calculator 112, mathematical tools 116, developer forum search tool 210, and / or analyzer 108). In at least one embodiment, the local storage includes chat history. Remote storage may include a GPU cloud, for example, for storing one or more large language models (LLMs) and / or web resources, such as API keys or other information associated with one or more APIs or libraries. Remote storage may also include telemetry information. In at least one embodiment, system 400 executes one or more methods 500, 600, and / or 700 (see...). Figure 5-7 ).

[0047] Now for reference Figure 5 Each block of the method 500 described herein contains a computational process that can be executed using any combination of hardware, firmware, and / or software. For example, various functions can be performed using one or more processors executing instructions stored in one or more memories. The method can also be embodied as computer-usable instructions stored on a computer storage medium. The method can be provided as a microservice by a standalone application, service, or managed service (standalone or in combination with other managed services) via an application programming interface (API) or plug-in to other products. Furthermore, method 500 is illustrated by way of example regarding... Figure 1-4The method is described in this document. However, it may be additionally or alternatively implemented by any system or combination of systems, including but not limited to the systems described herein.

[0048] Figure 5 This is a flowchart illustrating a method 500 for providing one or more answers to a prompt, according to at least one embodiment. Method 500 includes obtaining one or more prompts (e.g., queries) at block 502. Method 500 may begin when invoked by one or more processors.

[0049] Block 502, used to obtain prompts, may include generating or receiving prompts from the user for the LLM-based agent to resolve one or more user queries. The prompts may include one or more user questions related to the computation analyzer, architecture programming, and / or specific profile results. To obtain one or more prompts in block 502, the system may receive one or more inputs via user interface 300. The user may provide one or more prompts via text or audio input. One or more prompts may indicate at least one or more user queries, such as queries related to the performance of one or more programs executed by one or more GPUs (e.g., queries related to specific performance metrics, measurements, optimizations, and / or variations thereof).

[0050] Method 500 may include decision block 504. At decision block 504, if a valid profile result exists, the decision is "yes"; otherwise, it is "no". If the decision at decision block 504 is "no", the system executing method 500 may continue execution to block 510. The system executing method 500 may determine the existence of a valid profile result based on a prompt. For example, if the prompt indicates a query related to one or more programs that have been executed by one or more GPUs, wherein the performance of said one or more programs has been monitored by GPU performance monitoring / optimization software or otherwise processed to generate a profile report, and said query specifically relates to certain aspects, measurements, or results of the profile report, then the system may determine that a valid profile result exists. If the decision at decision block 504 is "yes", the system executing method 500 may continue execution to block 506.

[0051] Method 500 at block 506 includes: causing report agent 214 to generate one or more answers (e.g., answer output 220, see...). Figure 2Then, at block 508, one or more of the generated answers can be provided to the user and / or one or more LLMs. Method 500 at block 508 includes providing one or more answers to the prompt. One or more of the generated answers can be provided to the user through the user interface described herein. Method 500 at block 510 includes causing the general agent 206 to generate one or more answers (e.g., answer output 212, see...). Figure 2 After executing block 510 to have the generic agent generate an answer, the system performing method 500 may continue execution to block 508. In at least one embodiment, method 500 may include one or more agents (e.g., reporting agent 214 and / or generic agent 206) for accessing one or more web-based resources to generate one or more responses. After providing an answer to the prompt at block 508, the system may continue execution of one or more operations and / or terminate.

[0052] In at least one embodiment, one or more processors use method 500, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing one or more programs executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, as an example, a set of instructions is stored on a machine-readable medium (e.g., a non-transitory medium) that, if executed by one or more processors, causes one or more processors to perform method 500, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing one or more programs executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein.

[0053] In at least one embodiment, method 500 is Figure 1-10 The process shown is included Figure 1-10 The process shown, and / or otherwise includes Figure 1-10The process illustrated is used, for example, to: generate computer program code using one or more neural networks to obtain performance information of one or more programs based at least in part on one or more user queries instructing one or more programs to be executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to the one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, Figure 1-10 The one or more system execution methods 500 shown are, for example, used to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, Figure 8A-10 The one or more hardware methods 500 shown are used, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by said one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein.

[0054] Now for reference Figure 6 Each block of the method 600 described herein contains a computational process that can be executed using any combination of hardware, firmware, and / or software. For example, various functions can be performed using one or more processors executing instructions stored in one or more memories. The method can also be embodied as computer-usable instructions stored on a computer storage medium. The method can be provided as a microservice by a standalone application, service, or managed service (standalone or in combination with other managed services) via an application programming interface (API) or plug-in to other products. Furthermore, method 600 is illustrated by way of example regarding... Figure 1-5 The system described herein is described in this document. However, this method may be additionally or alternatively performed by any single system or any combination of systems, including but not limited to the system described herein.

[0055] Figure 6 The flowchart illustrates a method 600 for providing one or more answers using a generic agent, according to at least one embodiment. Method 600 includes obtaining one or more prompts (e.g., queries) at block 602. Method 600 may begin when invoked by one or more processors.

[0056] Block 602, used to obtain a prompt, may include generating or receiving from the user a prompt for the LLM-based agent to resolve one or more user queries. The prompt may include one or more user questions related to the computation analyzer, architecture programming, and / or specific profiling results. To obtain one or more prompts in block 602, the system may receive one or more inputs through user interface 300. The prompt may guide the LLM to generate a response and may integrate insights from various analysis tools 210, such as developer forum searches. For example, at block 602, the obtained prompt may not contain profiling results and (e.g., via router 204) be routed to general agent 206, for example, using method 500.

[0057] Method 600 at block 604 includes identifying any resources needed to answer one or more questions, such as one or more categories and / or embeddings using user input. As an example, at block 604, the generic agent can adjust one or more prompts and / or obtain any API information needed to access the identified information. The system executing method 600 can identify any documentation, guides, resources, or information that might be needed to answer one or more questions. The system executing method 600 can then proceed to obtain information using one or more prompts at block 606.

[0058] Method 600, at block 606, includes obtaining information using one or more prompts. The information may include information retrieved from one or more vector repositories 208, such as processor analysis documents and / or programming guides. The system executing method 600 may perform one or more RAG processes against the identified documents, guides, resources, or information to obtain information for answering one or more questions. The system executing method 600 may then continue execution to generate an answer at block 608, at least in part, based on the obtained information.

[0059] Method 600 includes generating an answer at least in part based on the obtained information at block 608. To generate an answer at least in part based on the obtained information, at block 608, the system may utilize a network of LLM agents collaboratively processing the user query. This process at block 608 may include having one or more LLMs retrieve, process, and / or integrate relevant data to construct a coherent response. The output at block 608 may include a detailed answer to the user query and may include links to other resources, such as scripts or relevant forum posts, providing comprehensive insights into performance analysis and / or optimization. The system executing method 600 can then proceed to provide an answer to the prompt at block 610.

[0060] Method 600 includes providing an answer to the prompt at block 610. In at least one embodiment, at block 610, the answer is provided to one or more LLMs, users, and / or user interfaces 300. After providing an answer to the prompt at block 610, the system performing method 600 may continue performing one or more operations described herein and / or terminate.

[0061] In at least one embodiment, one or more processors use method 600, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing one or more programs executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, as an example, a set of instructions is stored on a machine-readable medium (e.g., a non-transitory medium) that, when executed by one or more processors, causes one or more processors to perform method 600, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing one or more programs executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein.

[0062] In at least one embodiment, method 600 is Figure 1-10 The process shown, or included Figure 1-10 The process shown and / or otherwise includes Figure 1-10 The process illustrated is used, for example, to: generate computer program code using one or more neural networks to obtain performance information of one or more programs based at least in part on one or more user queries instructing one or more programs to be executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to the one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, Figure 1-10The one or more systems shown in the method 600 are, for example, used to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, Figure 8A-10 The one or more hardware methods 600 shown are used, for example, to: generate computer program code using one or more neural networks to obtain performance information of one or more programs based at least in part on one or more user queries instructing one or more programs to be executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to the one or more user queries; and / or otherwise perform the operations described herein.

[0063] Now for reference Figure 7 Each block of the method 700 described herein contains a computational process, which can be implemented using any combination of hardware, firmware, and / or software. For example, various functions can be performed by executing instructions stored in one or more memories using one or more processors. The method can also be implemented as computer-usable instructions stored on a computer storage medium. The method can be provided as a microservice by a standalone application, service, or managed service (standalone or in combination with other managed services) via an application programming interface (API) or plug-in to other products. Furthermore, method 700 is illustrated by way of example regarding... Figure 1-6 The system described herein. However, this method may be additionally or alternatively implemented by any system or combination of systems, including but not limited to the system described herein.

[0064] Figure 7 A flowchart illustrating a method 700 for providing one or more answers using a report agent, according to at least one embodiment, is shown. Methods 700 and 600 may be used in combination or otherwise combined to generate one or more answers to one or more user queries. The system may execute one or more of methods 600 and 700 in any suitable order to generate one or more answers to one or more user queries. Method 700 includes obtaining one or more prompts (e.g., queries) at block 702. Method 700 may begin when invoked by one or more processors.

[0065] Block 702, used to obtain prompts, may include generating or receiving prompts from the user for an LLM-based agent to handle one or more user queries. The prompts may include one or more user questions or queries related to a computation analyzer, architecture programming, and / or specific profiling results. To obtain one or more prompts, in block 702, the system may receive one or more inputs through user interface 300. The prompts may guide the LLM to generate responses and may integrate insights from various analysis tools 218 (e.g., language processors and / or occupancy calculators). The prompts may be requests for or analyses of information related to GPU resource usage information of one or more programs executed by one or more GPUs. As an example, at block 702, the obtained prompts may include profiling results and (e.g., via router 204) be routed to reporting agent 214, for example, using method 500.

[0066] Method 700 includes, at block 704, the use of prompts to generate scripts. To use prompts to generate scripts, at block 704, the system may employ an LLM-based agent that interprets the user's query and constructs scripts (e.g., Python scripts and / or other programming languages) to extract relevant data from a profiling report. This process may begin with a user-provided prompt containing specific questions related to one or more programs (e.g., kernels) executed by one or more GPUs, such as questions related to the performance of one or more programs, hardware and / or software (e.g., resource usage) measurements of one or more GPUs during the execution of one or more programs, and / or any suitable questions related to the performance of one or more programs. The system can identify specific performance information required to answer a specific question. As an illustrative example, if the user provides a question related to latency measurement, the system can determine that latency measurements must be obtained to answer the question. The system can access a profiling report generated by software that monitors the performance of one or more programs executed by one or more GPUs.

[0067] Then, at block 704, the system can use the Python Reporting Interface (PRI) or any suitable interface to write and execute a script that collects specific performance information from the reports to answer a specific question. The system can use one or more neural networks (such as the neural networks described herein) to generate computer program code that, when executed, obtains GPU resource usage information (e.g., profile reports or information from performance reports) from one or more performance reports generated by the GPU performance software. The script can be any suitable computer program code, such as a Python script, that, when executed, extracts and processes specific information indicated by the user query based on the profile report, enabling the system to provide a detailed response or optimization suggestion based on the extracted information. The system executing method 700 can then continue execution to allow the script to be executed. The system can generate the script so that it can be used to extract or otherwise obtain performance information, such as performance information from one or more profile reports.

[0068] Method 700 at block 706 includes determining whether the script is sufficient, which may refer to determining whether to generate the script such that no errors are encountered when executing the script, or that correct information is obtained during the execution of the script. If any runtime errors or script execution errors exist, the system (e.g., an agent) may modify, repair, or otherwise regenerate the script. If any errors exist in the script, or the script is insufficient for other reasons to obtain information to generate an answer to a prompt, the system executing method 700 may continue execution to block 704 to regenerate the script. One or more processors may execute the script or otherwise make the script be executed, wherein the script may enable one or more processors to analyze or otherwise extract performance information from one or more profile reports. In one embodiment, the agent may continuously generate and execute the script until the script can be executed without any runtime errors, or otherwise can be executed to obtain indicated information. Information may also be retrieved from one or more vector repositories or any suitable database or information source. The system executing method 700 may make the script be executed at block 708 to obtain information. The system may enable one or more processors to execute or otherwise run the script. Scripts can enable one or more processors to analyze or otherwise extract information from one or more performance reports (e.g., generated by GPU performance software) to obtain GPU resource usage information for one or more programs executed by one or more GPUs.

[0069] Method 700 at block 710 includes determining whether more information is needed. The system executing method 700 can determine whether the information obtained is sufficient to generate an answer to the prompt. The system can determine whether an answer to the prompt can be generated based on the obtained information, or whether more information is needed to generate an answer to the prompt. The system can determine whether additional scripts are needed to further process the obtained information, or to use the obtained information to obtain other information. If more information is needed, the system can continue execution to block 704 to generate another script that can be used to obtain other information, or otherwise utilize the obtained information to obtain or otherwise generate other information. Method 700 at block 712 includes generating an answer at least partially based on the obtained information. To generate an answer at least partially based on the obtained information at block 712, the system can utilize a network of LLM agents that work together to process user queries. This process at block 712 may include employing Retrieval Enhanced Generation (RAG) techniques, where one or more LLMs acquire and integrate relevant data to construct a coherent response. The output at block 712 may include a detailed answer to the user's query and may include additional resources, such as scripts or links to relevant forum posts, providing comprehensive insights into performance analysis and / or optimization. The system executing method 700 can then proceed to provide the answer to the prompt at block 714.

[0070] Method 700 includes providing an answer to the prompt at block 714. In at least one embodiment, at block 714, the answer is provided to one or more LLMs, users, and / or user interfaces 300. The answer can be provided to one or more users through the user interfaces described herein. After providing an answer to the prompt at block 714, the system performing method 700 can continue performing one or more operations described herein and / or terminate.

[0071] In at least one embodiment, one or more processors use method 700, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing one or more programs executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein. In at least one embodiment, as an example, a set of instructions is stored on a machine-readable medium (e.g., a non-transitory medium) that, if executed by one or more processors, causes one or more processors to perform method 700, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing one or more programs executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein.

[0072] In at least one embodiment, method 700 is Figure 1-10 The process shown is included Figure 1-10 The process shown or otherwise includes Figure 1-10 The process shown herein is used, for example, to: generate computer program code using one or more neural networks to obtain performance information of one or more programs based at least in part on one or more user queries instructing one or more programs to be executed by one or more graphics processing units (GPUs); execute the computer program code to generate one or more responses to the one or more user queries; and / or otherwise perform the operations described herein.

[0073] In at least one embodiment, Figure 1-10 The one or more systems shown in the diagram execute method 700, for example, for: generating computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by one or more graphics processing units (GPUs); executing said computer program code to generate one or more responses to said one or more user queries; and / or otherwise performing the operations described herein. In at least one embodiment, Figures 8A to 10The one or more hardware methods 700 shown are used, for example, to: generate computer program code using one or more neural networks to obtain performance information of said one or more programs based at least in part on one or more user queries instructing said one or more programs to be executed by said one or more graphics processing units (GPUs); execute said computer program code to generate one or more responses to said one or more user queries; and / or otherwise perform the operations described herein.

[0074] The systems and methods described herein can be used for a variety of purposes, as examples but not limited to machine (e.g., robots, vehicles, construction machinery, warehouse vehicles / machines, autonomous, semi-autonomous and / or other machine types) control, machine motion, machine driving, synthetic data generation, model training (e.g., using real, augmented and / or synthetic data, such as synthetic data generated using simulation platforms or systems, synthetic data generation techniques (such as, but not limited to, those described herein), perception, augmented reality (AR), virtual reality (VR), mixed reality (MR), robotics, security and supervision (e.g., in smart city implementations), autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation and / or digital twins, data center processing, conversational AI, optical transport simulation (e.g., ray tracing, path tracing, etc.), distributed or collaborative content creation for 3D assets (e.g., using generic scene descriptor (USD) data, such as OpenUSD and / or other data types), cloud computing, generative artificial intelligence (e.g., using one or more diffusion models, transformer models, etc.) and / or any other suitable application.

[0075] The disclosed embodiments can comprise a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots or robotic platforms, aviation systems, medical systems, rowing systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations (e.g., in driving or vehicle simulations, in robot simulations, in smart city or supervised simulations, etc.), systems for performing digital twin operations (e.g., in conjunction with collaborative content creation platforms or systems, such as, but not limited to, NVIDIA's OMNIVERSE and / or other platforms, systems, or services using USD or OpenUSD data types), systems implemented using edge devices, systems containing one or more virtual machines (VMs), and systems for (e.g.) For example, systems that perform synthetic data generation operations using one or more neural rendering fields (NERF), Gaussian sputtering techniques, diffusion models, converter models, etc.; systems implemented at least partially in data centers; systems for performing conversational AI operations; systems that implement one or more language models (such as one or more large language models (LLM), one or more visual language models (VLM), one or more multimodal language models, etc.); systems for performing optical transport simulations; systems for performing collaborative content creation for 3D assets (e.g., using generic scene descriptor (USD) data, such as OpenUSD, computer-aided design (CAD) data, 2D and / or 3D graphics or design data and / or other data types); systems implemented at least partially using cloud computing resources; and / or other types of systems.

[0076] Example language model In at least some embodiments, language models, such as Large Language Models (LLM), Visual Language Models (VLM), Multimodal Language Models (MMLM), and / or other types of generative artificial intelligence (AI), can be implemented. As an illustrative example, systems 100, 200, and / or 400 may implement language models as part of one or more agents. These models may be able to understand, summarize, translate, and / or otherwise generate text (e.g., natural language text, code, etc.), images, videos, computer-aided design (CAD) assets, OMNIVERSE and / or METAVERSE file information (e.g., USD formats such as OpenUSD), and / or the like, based on context provided in input prompts or queries. In embodiments, these language models may be considered “large” because they are trained on massive datasets and have architectures with a large number of learnable network parameters (weights and biases)—e.g., millions or billions of parameters. This disclosure allows for the implementation of LLM / VLM / MMLM, etc., for summarizing textual data, analyzing data (e.g., text, images, videos, etc.), extracting insights from data (e.g., text, images, videos, etc.), and generating new text / images / videos / , etc., in a user-specified style, tone, and / or format. In some embodiments, the LLM / VLM / MMLM, etc., of this disclosure may be specifically designed for text processing, while in other embodiments, a multimodal LLM may be implemented to accept, understand, and / or generate text and / or other types of content, such as images, audio, 2D and / or 3D data (e.g., USD format) and / or video. For example, a Visual Language Model (VLM) or more specifically a Multimodal Language Model (MMLM) may be implemented to accept images, videos, audio, text, 3D designs (e.g., CAD), and / or other input data types and / or generate or output images, videos, audio, text, 3D designs, and / or other output data types.

[0077] Various types of LLM / VLM / MMLM / etc. architectures can be implemented in various embodiments. For example, different architectures can be implemented using different techniques to understand and generate outputs (e.g., text, audio, video, images, 2D and / or 3D design or asset data, etc.). In some embodiments, LLM / VLM / MMLM / etc. architectures (e.g., recurrent neural networks (RNNs) or long short-term memory networks (LSTMs)) can be used, while in other embodiments, converter architectures (e.g., architectures relying on self-attention and / or cross-attention (e.g., between contextual data and textual data) mechanisms) can be used to understand and recognize relationships between words or tokens and / or contextual data (e.g., other text, video, images, design data, USD, etc.). One or more generative processing pipelines including LLM / VLM / MMLM / etc. may also include one or more diffusion blocks (e.g., noise reduction blocks). The LLM / VLM / MMLM / etc. of this disclosure may include encoder and / or decoder blocks. For example, discriminative or encoder-only models (e.g., BERT (Bidirectional Encoder Representations from Transformers)) can be implemented for tasks involving language understanding (e.g., classification, sentiment analysis, question answering, and named entity recognition). As another example, generative or decoder-only models (e.g., GPT (Generative Pretrained Transformer)) can be implemented for tasks involving language and content generation (e.g., text completion, story generation, and dialogue generation). LLM / VLM / MMLM / etc., including encoder and decoder components (e.g., T5 (Text-to-Text Transformer)), can be implemented to understand and generate content, such as for translation and summarization. These examples are not intended to be limiting and any architecture type (including, but not limited to, those described herein) can be implemented depending on the specific implementation and the task performed using LLM / VLM / MMLM / etc.

[0078] In various embodiments, LLM / VLM / MMLM / etc. can be trained using unsupervised learning, where LLM / VLM / MMLM / etc. learns patterns from a large amount of unlabeled text / audio / video / image / design / USD / etc. data. Due to the extensive training, in these embodiments, the model may not require task-specific or domain-specific training. An LLM / VLM / MMLM / etc. extensively pre-trained on a large amount of unlabeled data can be referred to as a base model and can excel at various tasks, such as question answering, summarizing, filling in missing information, translation, and image / video / design / USD / data generation. Some LLM / VLM / MMLM / etc. can be customized for specific use cases using techniques such as cue tuning, fine-tuning, retrieval augmentation generation (RAG), adding adapters (e.g., custom neural networks and / or neural network layers to tune or adjust cues or labels to bias the language model towards a specific task or domain), and / or using optimization models for specific tasks and / or other fine-tuning or customization techniques for specific domains.

[0079] In some embodiments, the LLM / VLM / MMLM / etc. disclosed herein can be implemented using various model alignment techniques. For example, in some embodiments, guardrails can be implemented to identify incorrect or unwanted inputs (e.g., prompts) and / or outputs of the model. In this process, the system can use guardrails and / or other model alignment techniques to prevent the processing of specific unwanted inputs using LLM / VLM / MMLM / etc., and / or to prevent the output or presentation of information generated by LLM / VLM / MMLM / etc. (e.g., displays, audio outputs, etc.). In some embodiments, one or more additional models (or layers thereof) can be implemented to identify problems with the model's inputs and / or outputs. For example, these "protective" models can be trained to identify "safe" or otherwise okay or desired inputs and / or outputs and / or "unsafe" or otherwise unwanted inputs and / or outputs for a particular application / implementation. Therefore, the LLM / VLM / MMLM / etc. disclosed herein are unlikely to output language / text / audio / video / design data / USD data / etc. that may be offensive, vulgar, inappropriate, insecure, out of scope, and / or unwanted for a particular application / implementation.

[0080] In some embodiments, an LLM / VLM / etc. can be configured or able to access or use one or more plugins, application programming interfaces (APIs), databases, data stores, repositories, etc. For example, for certain tasks or operations where the model is not ideally suited, the model may have instructions for accessing one or more plugins (e.g., third-party plugins) to help process the current input (e.g., as a result of training, and / or based on instructions in a given prompt). In such an example, when at least part of the prompt relates to restaurants or weather, the model can access one or more restaurant or weather plugins (e.g., via one or more APIs) to retrieve relevant information. Another example is that if at least part of the response requires mathematical computation, the model can access one or more mathematical plugins or APIs to help solve the problem, and then the response from the plugins and / or APIs can be used in the model's output. This process can be repeated (e.g., recursively) an arbitrary number of iterations, using any number of plugins and / or APIs, until a response to each query / question / request / process / operation / etc. can be generated in response to the input prompt. Therefore, models can rely not only on their own knowledge gained from training on large datasets, but also on the expertise or optimized properties of one or more external resources (such as APIs, plugins, etc.).

[0081] In some embodiments, multiple language models (e.g., LLM / VLM / MMLM / etc., multiple instances of the same language model, and / or multiple hints provided to the same language model or instances of the same language model) can be implemented, executed, or accessed (e.g., using one or more plugins, user interfaces, APIs, databases, data stores, repositories, etc.) to provide output in response to the same query or in response to separate parts of a query. In at least one embodiment, the same input query and hints (e.g., a set of constraints, condition generators, etc.) can be provided to multiple language models (e.g., language models with different architectures, language models trained on different (e.g., updated) data corpora). In one or more embodiments, the language models can be different versions of the same base model. In one or more embodiments, at least one language model can be instantiated as multiple agents, for example, providing more than one hint to constrain, guide, or otherwise influence the style, content, or character of the provided output. In one or more exemplary non-limiting embodiments, the same language model can be required to provide output corresponding to different roles, perspectives, characters, or different knowledge bases, as defined by the provided hints.

[0082] In any such embodiment, the outputs of two or more (e.g., each) language models, two or more versions of at least one language model, two or more instantiated proxies of at least one language model, and / or provided to two or more prompts for at least one language model can be further processed, such as aggregated, compared, or filtered, or used to determine (and provide) a consensus response. In one or more embodiments, the output from one language model (or version, instance, or proxy) can be provided as input to another language model for further processing and / or validation. In one or more embodiments, the language model can be required to generate or otherwise obtain output about the input source material, wherein the output is associated with the input source material. Such association can include, for example, generating captions or text portions embedded (e.g., as metadata) within the input source text or image. In one or more embodiments, the output of the language model can be used to determine the validity of the input source material for further processing or inclusion in a dataset. For example, the language model can be used to evaluate the presence (or absence) of a target word in a text portion or the presence (or absence) of an object in an image, wherein the text or image is annotated to indicate such presence (or absence). Alternatively, the determination from the language model can be used to determine whether the source material should be included in a curatorial dataset, for example, but not limited to this.

[0083] Figure 8A This is a block diagram of an example generative language model system 800 suitable for implementing at least some embodiments of the present disclosure, such as in combination with Figure 1-7 The embodiments described. As illustrative examples, systems 100, 200, and / or 400 may implement or otherwise utilize one or more components and / or processes of system 800. Figure 8A In the example shown, the generative language model system 800 includes a retrieval augmentation generation (RAG) component 892, an input processor 805, a tokenizer 810, an embedding component 820, a plugin / API 895, and a generative language model (LM) 830 (which may include LLM, VLM, multimodal LM, etc.).

[0084] At a high level, the input processor 805 can receive input 801, which includes text and / or other types of input data (e.g., audio data, video data, image data, sensor data (e.g., LiDAR, RADAR, ultrasound, etc.), 3D design data, CAD data, generic scene descriptor (USD) data (e.g., OpenUSD, etc.), depending on the architecture of the generative LM 830 (e.g., LLM / VLM / MMLM / etc.). In some embodiments, input 801 includes plain text in the form of one or more sentences, paragraphs, and / or documents. Additionally or alternatively, input 801 may include numerical sequences, pre-computed embeddings (e.g., word or sentence embeddings), and / or structured data (e.g., tabular format, JSON, or XML). In generative LM In some implementations of 830 capable of handling multimodal input, input 801 can combine text (or text that may be omitted) with image data, audio data, video data, design data, USD data, and / or other types of input data (e.g., but not limited to the data described herein). Taking raw input text as an example, input processor 805 can prepare the raw input text in various ways. For example, input processor 805 can perform various types of text filtering to remove noise from relevant text content (e.g., special characters, punctuation marks, HTML tags, stop words, portions of images, portions of audio, etc.). In examples involving stop words (common words that often have little semantic meaning), input processor 805 can remove stop words to reduce noise and allow the generative LM 830 to focus on more meaningful content. Input processor 805 can apply text normalization, for example, by converting all characters to lowercase, removing accent marks, and / or handling special cases (such as abbreviations or abbreviations) to ensure consistency. These are just a few examples; other types of input processing can be applied.

[0085] In some embodiments, RAG component 892 (which may include one or more RAG models, and / or may be performed using generative LM 830 itself) may be used to retrieve additional information to be used as part of input 801 or a prompt. RAGs can be used to enhance input to LLM / VLM / MMLM / etc. with external knowledge to make the answer to a specific question or query or request more relevant, for example, where specific knowledge is required. RAG component 892 may obtain this additional information from one or more external sources (e.g., basic information such as basic text / images / videos / audio / USD / CAD / etc.), which can then be fed along with the prompt to LLM / VLM / MMLM / etc. to improve the accuracy of the model's response or output.

[0086] For example, in some embodiments, in addition to the data retrieved using RAG component 892, input 801 may be generated using query or model inputs (e.g., questions, requests, etc.). In some embodiments, input processor 805 may analyze input 801 and communicate with RAG component 892 (or in embodiments, RAG component 892 may be part of input processor 805) to identify relevant text and / or other data to provide to generative LM 830 as additional context or information source, typically from which to identify responses, answers, or outputs 890. For example, when the input indicates that a user is interested in the required tire pressure for a particular brand and model of vehicle, RAG component 892 may use a RAG model, for example, to perform a vector search in the embedding space to retrieve tire pressure information or its corresponding text from a digital (embedded) version of the owner's manual for that particular vehicle brand and model. Similarly, when a user revisits a chatbot related to a particular product for sale or service, RAG component 892 may retrieve previously stored conversation history (or at least its summary) and provide the previous conversation history along with the current inquiry / request as part of input 801 to generative LM 830.

[0087] RAG component 892 can use various RAG techniques. For example, naïve RAG can be used, where documents are indexed, chunked, and applied to an embedding model to generate embeddings corresponding to chunks. User queries can also be applied to this embedding model and / or another embedding model of RAG component 892, and the embeddings of chunks can be compared with the embeddings of the query to identify the most similar / relevant embeddings to the query. These most similar / relevant embeddings can be provided to generative LM 830 to generate output.

[0088] In some embodiments, more advanced RAG techniques can be used. For example, chunks can undergo pre-retrieval processes (e.g., routing, rewriting, metadata analysis, expansion, etc.) before being passed to the embedding model. Furthermore, post-retrieval processes (e.g., re-ranking, hint compression, etc.) can be performed on the output of the embedding model before generating the final embedding, which is then used for comparison with the input query.

[0089] As a further example, modular RAG techniques can be used, such as those similar to Naive RAG and / or Advanced RAG, but also including features such as hybrid search, recursive retrieval and query engines, StepBack methods, subqueries and hypothetical document embeddings.

[0090] As another example, Graph RAG can use a knowledge graph as a source of context or factual information. Graph RAG can be implemented using a graph database as a source of contextual information sent to LLM / VLM / MMLM / etc. Instead of providing the model with data chunks extracted from larger documents (which can lead to a lack of context, factual accuracy, linguistic accuracy, etc.) (or anything other than providing the model with data chunks extracted from larger documents), Graph RAG can also provide the model with structured entity information by combining structured entity text descriptions with their many attributes and relationships, thus giving the model deeper insights. In implementing Graph RAG, the systems and methods described herein use graphs as content stores and extract relevant document chunks, requiring LLM / VLM / MMLM / etc. to use them to answer questions. In such embodiments, the knowledge graph may contain relevant textual content and metadata about the knowledge graph, or it may be integrated with a vector database. In some embodiments, Graph RAG can use the graph as a subject matter expert, where descriptions of concepts and entities relevant to the query / hint can be extracted and passed to the model as semantic context. These descriptions may include relationships between concepts. In other examples, the graph can be used as a database where a portion of a query / hint can be mapped to a graph query, the graph query can be executed, and LLM / VLM / MMLM / etc. can summarize the results. In such examples, the graph can store relevant factual information and can be used for queries (natural language queries) and entity links to graph query tools (NL to graph query tools). In some embodiments, the graph RAG (e.g., using a graph database) can be combined with standard (e.g., vector database) RAGs and / or other RAG types to benefit from a variety of approaches.

[0091] In any embodiment, the RAG component 892 can implement plugins, APIs, user interfaces, and / or other functions to perform RAG. For example, LLM / VLM / MMLM / etc. can use graph RAG plugins to run queries on knowledge graphs to extract relevant information to feed into the model, and can use standard or vector RAG plugins to run queries on vector databases. For example, the graph database can interact with the plugin's REST interface, thus decoupling the graph database from the vector database and / or the embedded model.

[0092] The tokenizer 810 can segment (e.g., processed) text data into smaller units (tags) for subsequent analysis and processing. Depending on the implementation, the tags can represent individual words, sub-words, characters, audio / video / images, etc. Word-based tokenization divides the text into individual words, treating each word as a separate tag. Sub-word tokenization breaks words down into smaller meaningful units (e.g., prefixes, suffixes, stems), enabling the generative LM 830 to understand morphological changes and process words outside the vocabulary more effectively. Character-based tokenization represents each character as a separate tag, enabling the generative LM 830 to process text at a fine-grained level. The choice of tokenization strategy can depend on factors such as the language being processed, the task at hand, and / or the characteristics of the training dataset. Therefore, the tokenizer 810 can transform (e.g., processed) text into a structured format according to the tokenization scheme implemented in a particular embodiment.

[0093] Embedding component 820 can use any known embedding technique to transform discrete tokens into semantically meaningful (e.g., dense, continuous vector) representations. For example, embedding component 820 can use pre-trained word embeddings (e.g., Word2Vec, GloVe, or FastText), one-hot encoding, Term Frequency-Inverse Document Frequency (TF-IDF) encoding, one or more embedding layers of a neural network, and / or others.

[0094] In some implementations where input 801 includes image data / video data, etc., input processor 801 may resize the data to a standard size compatible with the format of the corresponding input channel and / or normalize pixel values ​​to a common range (e.g., 0 to 1) to ensure consistent representation, and embedding component 820 may encode the image data using any known technique (e.g., using one or more convolutional neural networks (CNNs) to extract visual features). In some implementations where input 801 includes audio data, input processor 801 may resample the audio file to a consistent sampling rate for uniform processing, and embedding component 820 may use any known technique to extract and encode audio features, such as in the form of a spectrogram (e.g., a Mel spectrogram). In some implementations where input 801 includes video data, input processor 801 may extract frames or apply resizing to extracted frames, and embedding component 820 may extract features such as optical flow embedding or video embedding and / or encode temporal information or frame sequences. In some implementations where input 801 includes multimodal data, the embedded component 820 can use techniques such as early fusion (concatenation), late fusion (sequential processing), and attention-based fusion (e.g., self-attention, cross-attention) to fuse representations of different types of data (e.g., text, images, audio, USD, video, design, etc.).

[0095] Other components of the generative LM 830 and / or generative LM system 800 may use different types of neural network architectures depending on the implementation scheme. For example, a transducer-based architecture (such as the one used in models like GPT) may be implemented, and it may include a self-attention mechanism that weights the importance of different words or tokens in the input sequence and / or a feedforward network that processes the output of the self-attention layer, applying a nonlinear transformation to the input representation and extracting higher-level features. Some non-limiting example architectures include transducers (e.g., encoder-decoder, decoder-only, multimodal), RNNs, LSTMs, fusion models, diffusion models, cross-modal embedding models that learn a joint embedding space, graph neural networks (GNNs), hybrid architectures that combine different types of adversarial networks (such as generative adversarial networks or GANs or adversarial autoencoders (AAEs) for joint distribution learning), etc. Therefore, depending on the implementation scheme and architecture, the embedded component 820 can apply the encoded representation of the input 801 to the generative LM 830, and the generative LM 830 can process the encoded representation of the input 801 to generate an output 890, which may include response text and / or other types of data.

[0096] As described herein, in some embodiments, the generative LM 830 may be configured to access or use (or be able to access or use) plugins / APIs 895 (which may include one or more plugins, application programming interfaces (APIs), databases, data stores, repositories, etc.). For example, for certain tasks or operations where the generative LM 830 is not ideally suited, the model may have instructions (e.g., as a result of training, and / or based on instructions in a given prompt, such as instructions retrieved using RAG component 892) to access one or more plugins / APIs 895 (e.g., third-party plugins) to help process the current input. In such an example, when at least part of the prompt is related to a restaurant or weather, the model may access one or more restaurant or weather plugins (e.g., via one or more APIs), sending at least part of the prompt related to a particular plugin / API 895 to the plugin / API 895, which may process the information and return an answer to the generative LM 830, which may use the response to generate output 890. This process can be repeated (e.g., recursively) an arbitrary number of iterations and repeated with any number of plugins / APIs 895 until an output 890 that resolves each query / question / request / process / action / etc. from input 801 is generated. Therefore, the model can rely not only on its own knowledge acquired from training on a large dataset and / or from data retrieved using the RAG component 892, but also on the expertise or optimized properties of one or more external resources (e.g., plugins / APIs 895).

[0097] Figure 8B This is a block diagram of an example implementation scheme, where the generative LM 830 includes a converter encoder-decoder. For example, suppose the input text (e.g., "Who discovered gravity") is tokenized (e.g., by...) Figure 8A The tokenizer 810) is used for tokens such as words, and each token is encoded (e.g., by...). Figure 8A The embedding component 820 is a corresponding embedding (e.g., of size 512). Since these token embeddings do not typically represent the position of the tokens in the input sequence, positional encoding can be added to each token embedding using any known technique to encode the order relation and context of the tokens in the input sequence. Thus, the (e.g., the resulting) embeddings can be applied to one or more encoders 835 of the generative LM 830.

[0098] In the example implementation, encoder 835 forms an encoder stack, where each encoder includes a self-attention layer and a feedforward network. In the example converter architecture, each token (e.g., a word) flows through a separate path. Therefore, each encoder can accept a sequence of vectors, pass each vector through the self-attention layer, then through the feedforward network, and then up to the next encoder in the stack. Any known self-attention technique can be used. For example, to compute a self-attention score for each token (word), a query vector, a key vector, and a value vector can be created for each token. The self-attention score for a token pair can be computed by taking the dot product of the query vector and the corresponding key vector, normalizing the resulting score, multiplying by the corresponding value vector, and summing the weighted value vectors. The encoder can apply multi-head attention, where the attention mechanism is applied multiple times in parallel with different learned weight matrices. Any number of encoders can be cascaded to generate context vectors that encode the input. Attention projection layer 840 can transform the context vectors into attention vectors (keys and values) for decoder 845.

[0099] In the example implementation, decoder 845 forms a decoder stack, where each decoder includes a self-attention layer, an encoder-decoder self-attention layer that uses attention vectors (keys and values) from the encoder to focus on relevant parts of the input sequence, and a feedforward network. Similar to encoder 835, in the example converter architecture, each token (e.g., a word) flows through a separate path in decoder 845. During the first pass, decoder 845, classifier 850, and generation mechanism 855 can generate a first token, and generation mechanism 855 can apply the generated token as input during a second pass. This process can be repeated cyclically, generating tokens (e.g., words) and adding them to the output of the previous pass, and in subsequent passes applying token embeddings of positionally encoded composite sequences as input to decoder 845, generating one token at a time (called autoregression) until a symbol or token representing the end of the response is predicted. In each decoder, the self-attention layer is typically restricted to focusing only on preceding positions in the output sequence by applying a masking technique (e.g., setting future positions to negative infinity) before the softmax operation. In the example implementation, the encoder-decoder attention layer operates similarly to the (e.g., multi-head) self-attention operation in the encoder 835, except that it creates its query from the layer below it and obtains keys and values ​​(e.g., matrices) from the output of the encoder 835.

[0100] Therefore, decoder 845 can output some decoded (e.g., vector) representation of the input applied during a particular pass. Classifier 850 can include a multi-class classifier comprising one or more neural network layers and a softmax operation that transforms logit probabilities into probabilities, the neural network layers projecting the decoded (e.g., vector) representation onto corresponding dimensions (e.g., one dimension for each supported word or token in the output vocabulary). Thus, generation mechanism 855 can select or sample words or tokens based on corresponding predicted probabilities (e.g., selecting the word with the highest predicted probability) and append it to the output of the previous pass, thereby generating each word or token sequentially. Generation mechanism 855 can repeat this process, triggering successive decoder inputs and corresponding predictions until a symbol or token representing the end of the response is selected or sampled, at which point generation mechanism 855 can output the generated response.

[0101] Figure 8C This is a block diagram of an example implementation where the generative LM 830 includes a decoder-only converter architecture. For example, Figure 8C The decoder 860 can be used with Figure 8B The decoder 845 operates similarly, except... Figure 8C Each decoder 860 omits the encoder-decoder self-attention layer (because there is no encoder in this implementation). Therefore, decoders 860 can form a decoder stack, where each decoder includes a self-attention layer and a feedforward network. Furthermore, instead of encoding the input sequence, a symbol or tag indicating the end of the input sequence (or the beginning of the output sequence) can be appended to the input sequence, and the resulting sequence (e.g., a corresponding embedding with positional encoding) can be applied to decoder 860. Figure 8B Similar to decoder 845, each tag (e.g., a word) can flow through a separate path in decoder 860, and decoder 860, classifier 865, and generation mechanism 870 can use autoregression to generate one tag at a time sequentially until a symbol or tag indicating the end of the response is predicted. Classifier 865 and generation mechanism 870 can be combined with... Figure 8B The classifier 850 and the generation mechanism 855 operate similarly, wherein the generation mechanism 870 selects or samples each consecutive output label based on the corresponding predicted probability and appends it to the output of the previous iteration, generating each label sequentially until a symbol or label representing the end of the response is selected or sampled. The architectures described herein, and others, are merely examples, and other suitable architectures may be implemented within the scope of this disclosure.

[0102] Example computing device Figure 9 This is a block diagram of an example computing device 900 suitable for implementing some embodiments of the present disclosure, for example, in conjunction with Figure 1-7The embodiments described herein. As illustrative examples, systems 100, 200, and / or 400 may implement or otherwise utilize one or more components and / or processes of computing device 900. Computing device 900 may include interconnect systems 902 that are directly or indirectly coupled to the following devices: memory 904, one or more central processing units (CPUs) 906, one or more graphics processing units (GPUs) 908, communication interface 910, input / output (I / O) ports 912, input / output components 914, power supply 916, one or more presentation components 918 (e.g., one or more displays), and one or more logic units 920. In at least one embodiment, one or more computing devices 900 may include one or more virtual machines (VMs), and / or any of their components may include virtual components (e.g., virtual hardware components). For non-limiting examples, one or more GPUs 908 may include one or more vGPUs, one or more CPUs 906 may include one or more vCPUs, and / or one or more logic units 920 may include one or more virtual logic units. Thus, one or more computing devices 900 may include discrete components (e.g., a full GPU dedicated to computing device 900), virtual components (e.g., a portion of the GPU dedicated to computing device 900), or a combination thereof.

[0103] although Figure 9 The various blocks are shown as connected via interconnect system 902 using lines, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, presentation component 918 (such as a display device) may be considered I / O component 914 (e.g., if the display is a touchscreen). As another example, CPU 906 and / or GPU 908 may include memory (e.g., memory 904 may represent a storage device other than the memory of GPU 908, CPU 906, and / or other components). Therefore, Figure 9 The computing devices described are for illustrative purposes only. No distinction is made between such categories as “workstation,” “server,” “laptop computer,” “desktop computer,” “tablet computer,” “client device,” “mobile device,” “handheld device,” “game console,” “electronic control unit (ECU),” “virtual reality system,” and / or other device or system types, as all are considered within the scope of this category. Figure 9 Within the scope of computing devices.

[0104] Interconnect system 902 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 902 may include one or more bus or link types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Fast Peripheral Component Interconnect (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, CPU 906 may be directly connected to memory 904. Further, CPU 906 may be directly connected to GPU 908. Where there is a direct or point-to-point connection between components, interconnect system 902 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required to be included in computing device 900.

[0105] The memory 904 may include any computer-readable medium from a variety of computer-readable media. A computer-readable medium may be any available medium accessible by the computing device 900. Computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, computer-readable media may include computer storage media and communication media.

[0106] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 904 may store computer-readable instructions (e.g., representing one or more programs and / or one or more program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Universal Disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by computing device 900. As used herein, computer storage media does not include the signal itself.

[0107] Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals such as carrier waves or other transmission mechanisms, and includes any information transmission medium. The term "modulated data signal" can refer to a signal whose one or more characteristics are set or altered in a manner that encodes information in the signal. By way of example and not limitation, computer storage media can include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media). Any combination of the above should also be included within the scope of computer-readable media.

[0108] CPU 906 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 900 to perform one or more of the methods and / or processes described herein. Each CPU 906 may contain one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of handling numerous software threads simultaneously. CPU 906 may contain any type of processor and may contain different types of processors depending on the type of computing device 900 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 900, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors (such as math coprocessors), computing device 900 may also include one or more CPUs 906.

[0109] In addition to or in lieu of one or more CPUs 906, one or more GPUs 908 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 900 to perform one or more of the methods and / or processes described herein. One or more GPUs 908 may be integrated GPUs (e.g., having one or more CPUs 906) and / or one or more GPUs 908 may be discrete GPUs. In embodiments, one or more GPUs 908 may be coprocessors of one or more CPUs 906. GPUs 908 may be used by computing device 900 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, GPUs 908 may be used for general-purpose computing on a GPU (GPGPU). GPUs 908 may include hundreds or thousands of cores capable of handling hundreds or thousands of software threads simultaneously. GPUs 908 may generate pixel data for an output image in response to rendering commands (e.g., rendering commands received from CPUs 906 via a host interface). GPU 908 may include graphics memory (e.g., display memory) for storing pixel data or any other suitable data (e.g., GPGPU data). Display memory may be included as part of memory 904. GPU 908 may include two or more GPUs operating in parallel (e.g., via links). The links may directly connect the GPUs (e.g., using NVLINK) or may connect the GPUs via a switch (e.g., using NVSwitch). When combined, each GPU 908 may produce pixel data or GPGPU data for different portions of the output or for different outputs (e.g., a first GPU for a first image and a second GPU for an analog image). Each GPU may include its own memory or may share memory with other GPUs.

[0110] In addition to or in lieu of CPU 906 and / or GPU 908, logic unit 920 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 900 to perform one or more of the methods and / or processes described herein. In embodiments, one or more CPUs 906, one or more GPUs 908, and / or one or more logic units 920 may execute any combination of methods, processes, and / or portions thereof discretely or jointly. One or more logic units 920 may be a portion of one or more CPUs 906 and / or GPUs 908 and / or integrated into one or more CPUs 906 and / or GPUs 908, and / or one or more logic units 920 may be discrete components or otherwise external to CPUs 906 and / or GPUs 908. In embodiments, one or more logic units 920 may be coprocessors of one or more CPUs 906 and / or GPUs 908.

[0111] Examples of logic unit 920 include one or more processing cores and / or components thereof, such as a data processing unit (DPU), tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree lateral unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), programmable vision accelerator (PVA) (which may include one or more direct memory access (DMA) systems), one or more vision or vector processing units (VPU), and one or more pixel processing engines (PPE) (e.g.) Examples include 2D arrays of processing elements, each of which communicates north, south, east, and west with one or more other processing elements in the array; one or more decoupled accelerators or units (e.g., decoupled lookup table (DLUT) accelerators or units); vision processing units (VPUs); optical flow accelerators (OFAs); field-programmable gate arrays (FPGAs); neuromorphic chips; quantum processing units (QPUs); associative processing units (APUs); arithmetic logic units (ALUs); application-specific integrated circuits (ASICs); floating-point units (FPUs); input / output (I / O) elements; peripheral component interconnects (PCIs) or fast peripheral component interconnects (PCIe) elements; and so on.

[0112] The communication interface 910 may include one or more receivers, transmitters, and / or transceivers enabling the computing device 900 to communicate with other computing devices via electronic communication networks (including wired and / or wireless communications). The communication interface 910 may include components and functions for enabling communication over any of a plurality of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication over Ethernet or wirelessband), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, the logic unit 920 and / or the communication interface 910 may include one or more data processing units (DPUs) to directly transmit data received via a network and / or via interconnect system 902 to one or more GPUs 908 (e.g., memory of one or more GPUs 908).

[0113] I / O port 912 enables computing device 900 to be logically coupled to other devices including I / O component 914, one or more presentation components 918, and / or other components, some of which may be built into (e.g., integrated into) computing device 900. Illustrative I / O component 914 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, scanners, printers, wireless devices, etc. I / O component 914 can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological input generated by the user. In some cases, input may be transmitted to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, pen recognition, facial recognition, biometric recognition, on-screen and near-screen gesture recognition, air gestures, head and eye tracking, and touch recognition (as described in more detail below) associated with the display of computing device 900. Computing device 900 may include depth cameras for gesture detection and recognition, such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof. Additionally, the computing device 900 may include an accelerometer or gyroscope (e.g., as part of an inertial measurement unit (IMU)) that enables motion detection. In some examples, the computing device 900 may use the output of the accelerometer or gyroscope to render immersive augmented reality or virtual reality.

[0114] Power supply 916 may include a hardwired power supply, a battery power supply, or a combination thereof. Power supply 916 may provide power to computing device 900 so that the components of computing device 900 can operate.

[0115] The presentation component 918 may include a display (e.g., a monitor, touchscreen, television screen, head-up display (HUD), other display types, or combinations thereof), speakers, and / or other presentation components. The presentation component 918 may receive data from other components (e.g., GPU 908, CPU 906, etc.) and output the data (e.g., as images, videos, sounds, etc.).

[0116] Example Data Center Figure 10 An example data center 1000 that can be used in at least one embodiment of this disclosure is shown, for example, in combination with Figure 1-7 The embodiments described herein. As illustrative examples, systems 100, 200, and / or 400 may implement or otherwise utilize one or more components and / or processes of data center 1000. Data center 1000 may include a data center infrastructure layer 1010, a framework layer 1020, a software layer 1030, and / or an application layer 1040.

[0117] like Figure 10 As shown, the data center infrastructure layer 1010 may include a resource coordinator 1012, grouped computing resources 1014, and node computing resources (“nodes CRs”) 1016(1)-1016(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CRs 1016(1)-1016(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processors or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules and / or cooling modules, etc. In some embodiments, one or more node CRs from nodes CRs 1016(1)-1016(N) may correspond to servers having one or more of the aforementioned computing resources. In addition, in some embodiments, nodes CRs 1016(1)-1016(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more of nodes CRs 1016(1)-1016(N) may correspond to virtual machines (VMs).

[0118] In at least one embodiment, the grouped computing resources 1014 may include individual groups of node CRs 1016 housed within one or more racks (not shown), or multiple racks housed within a data center at different geographical locations (also not shown). Individual groups of node CRs 1016 within the grouped computing resources 1014 may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, several node CRs 1016, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0119] Resource coordinator 1022 may be configured or otherwise control one or more nodes CRs 1016(1)-1016(N) and / or groups of computing resources 1014. In at least one embodiment, resource coordinator 1022 may include a Software Design Infrastructure (“SDI”) management entity for data center 1000. Resource coordinator 1022 may include hardware, software, or some combination thereof.

[0120] In at least one embodiment, such as Figure 10 As shown, framework layer 1020 may include job scheduler 1028, configuration manager 1034, resource manager 1036, and / or distributed file system 1138. Framework layer 1020 may include a framework for software 1032 supporting software layer 1030 and / or one or more applications 1042 supporting application layer 1040. Software 1032 or application 1042 may respectively contain web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 1020 may be, but is not limited to, free and open-source software web application frameworks (such as Apache Spark) that can utilize distributed file system 1138 for large-scale data processing (e.g., "big data"). TM(Hereinafter referred to as "Spark") is a type of resource manager. In at least one embodiment, job scheduler 1028 may include Spark drivers to facilitate the scheduling of workloads supported by different layers of data center 1000. Configuration manager 1034 may be able to configure different layers, such as software layer 1030 and framework layer 1020 (which includes Spark and distributed file system 1138 for supporting large-scale data processing). Resource manager 1036 may be able to manage computing resources mapped to or allocated to clusters of distributed file system 1138 and job scheduler 1028 or to support clusters of distributed file system 1138 and job scheduler 1028. In at least one embodiment, clusters or groups of computing resources may include grouped computing resources 1014 in data center infrastructure layer 1010. Resource manager 1036 may coordinate with resource coordinator 1012 to manage these mapped or allocated computing resources.

[0121] In at least one embodiment, the software 1032 included in software layer 1030 may include software used in at least a portion of the nodes CRs 1016(1)-1016(N), the grouped computing resources 1014, and / or the distributed file system 1138 of framework layer 1020. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0122] In at least one embodiment, the application 1042 included in the application layer 1040 may include one or more types of applications used at least in part by nodes CRs 1016(1)-1016(N), grouped computing resources 1014, and / or the distributed file system 1138 of the framework layer 1020. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in combination with one or more embodiments.

[0123] In at least one embodiment, any of the configuration manager 1034, resource manager 1036, and resource coordinator 1012 can implement any number and type of self-modification actions based on any amount and type of data obtained in any technically feasible manner. Self-modification actions can free the data center operator of data center 1000 from making potentially poor configuration decisions and may prevent underutilization and / or poor performance of the data center.

[0124] According to one or more embodiments described herein, data center 1000 may include tools, services, software, or other resources to train one or more machine learning models or to use one or more machine learning models to predict or infer information. For example, one or more machine learning models can be trained by using the software and / or computing resources described above with respect to data center 1000 to compute weight parameters according to a neural network architecture. In at least one embodiment, a trained or deployed machine learning model corresponding to one or more neural networks can be used to infer or predict information using the resources described above with respect to data center 1000 by using weight parameters computed through one or more training techniques, such as, but not limited to, those described herein.

[0125] In at least one embodiment, the data center 1000 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the aforementioned resources. Furthermore, one or more of the software and / or hardware resources described above may be configured to allow a user to train or perform services that infer information, such as image recognition, speech recognition, or other artificial intelligence services.

[0126] Example network environment A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 9 This is implemented on one or more instances of computing devices 900—for example, each device may include similar components, features, and / or functions of one or more computing devices 900. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of a data center 1000, examples of which are described in this document. Figure 10 To describe in more detail.

[0127] Components of a network environment can communicate with each other via a network, which can be wired, wireless, or both. A network can include multiple networks or one of multiple networks. For example, a network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (such as the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. In cases where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0128] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein for the server can be implemented on any number of client devices.

[0129] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. The cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework supporting software at the software layer and / or application at the application layer. The software or application may respectively include network-based service software or applications. In embodiments, one or more client devices may use network-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs). The framework layer may be, but is not limited to, a free and open-source software network application framework that can use a distributed file system for large-scale data processing (e.g., "big data").

[0130] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, global, etc.). If the connection to a user (e.g., a client device) is relatively close to an edge server, the core server can assign at least a portion of the functionality to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0131] One or more client devices may include the information described in this article. Figure 9At least some of the components, features, and functions of one or more example computing devices 900 described. By way of example and not limitation, the client device may be implemented as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming device or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these depicted devices, or any other suitable device.

[0132] This disclosure can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions, such as program modules, that are executed by a computer or other machine, such as a personal data assistant or other handheld device. Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, and more specialized computing devices. This disclosure can also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked via a communication network.

[0133] As used herein, the phrase "and / or" relating to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, "element A, element B, and / or element C" can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Furthermore, "at least one of element A or element B" can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, "at least one of element A and element B" can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0134] This document describes in detail the subject matter of this disclosure to meet statutory requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the discloser has envisioned that the claimed subject matter may be embodied in other ways to include steps different from or similar combinations of steps described herein in conjunction with other current or future techniques. Moreover, although the terms "step" and / or "block" may be used herein to imply different elements of the method employed, these terms should not be construed as suggesting any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

Claims

1. One or more processors, including a processing circuitry system, said processing circuitry system being used for: Computer program code is generated using one or more neural networks to obtain performance information of one or more programs, at least in part, based on one or more user queries instructing one or more programs to be executed by one or more graphics processing units (GPUs); and The computer program code is executed to generate one or more responses to the one or more user queries.

2. The one or more processors of claim 1, wherein, The processing circuitry system is used to obtain the performance information by executing the computer program code to generate one or more performance reports.

3. The one or more processors of claim 1, wherein, The computer program code includes one or more computer program scripts.

4. The one or more processors of claim 1, wherein, The processing circuitry is used to generate one or more responses using the one or more neural networks, based at least in part on the one or more user queries and the performance information.

5. The processors according to claim 1, wherein, The one or more programs include one or more computing kernels.

6. The processors according to claim 1, wherein, The processing circuitry is used to obtain occupancy information related to the one or more programs in order to generate the one or more responses.

7. The processors according to claim 1, wherein, The one or more neural networks include one or more language models.

8. The processors according to claim 1, wherein, The one or more processors are included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for performing collaborative content creation for 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more multimodal language models; A system that implements one or more large-scale language model LLMs; A system that implements one or more Visual Language Models (VLMs); A system for generating synthetic data; A system that uses AI to generate synthetic data; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

9. A system comprising one or more processors, said one or more processors being used to: Obtain one or more queries from one or more users, wherein the one or more queries at least indicate one or more programs; Computer program code is generated using one or more neural networks to obtain performance information of one or more graphics processing units (GPUs) for executing the one or more programs, based at least in part on the one or more queries; and The computer program code is executed to generate one or more responses to the one or more queries.

10. The system according to claim 9, wherein, The one or more processors are used to obtain the performance information based at least in part on a profile report generated by software associated with the one or more GPUs.

11. The system according to claim 9, wherein, The one or more processors are used to generate the one or more responses based at least in part on documentation associated with the one or more GPUs.

12. The system according to claim 9, wherein, The one or more processors are used to execute one or more Retrieval Enhancement Generation (RAG) operations to generate the one or more responses.

13. The system according to claim 9, wherein, The system is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for performing collaborative content creation for 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more multimodal language models; A system that implements one or more large-scale language model LLMs; A system that implements one or more Visual Language Models (VLMs); A system for generating synthetic data; A system that uses AI to generate synthetic data; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

14. A method comprising: Obtain one or more queries, wherein the one or more queries at least indicate one or more programs executed by one or more graphics processing units (GPUs); Using one or more neural networks to generate computer program code to obtain performance information of one or more programs based at least in part on the one or more queries; as well as The performance information is obtained using the computer program code to generate one or more responses to the one or more queries.

15. The method of claim 14, further comprising: The computer program code is generated at least in part based on one or more application programming interface (API) tools.

16. The method of claim 14, further comprising: The chat interface is used to obtain one or more queries from one or more users.

17. The method of claim 14, wherein, The performance information includes measurements of resource usage of the one or more GPUs obtained from executing the one or more programs.

18. The method of claim 14, further comprising: Access one or more network-based resources to generate the one or more responses.

19. The method of claim 14, wherein, The computer program code includes one or more Python scripts.

20. The method of claim 14, wherein the method is performed by at least one of the following systems: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform simulation operations; Systems used to perform digital twin operations; A system for performing optical transmission simulation; A system for performing collaborative content creation for 3D assets; A system used to perform deep learning operations; A system used to perform remote operations; Systems used for performing real-time streaming; A system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; Systems implemented using edge devices; Systems implemented using robots; Systems used to perform conversational AI operations; A system that implements one or more multimodal language models; A system that implements one or more large-scale language model LLMs; A system that implements one or more Visual Language Models (VLMs); A system for generating synthetic data; A system that uses AI to generate synthetic data; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

Images