Automated software patch generation

Abstract

The present disclosure relates to automated software patch generation. Embodiments of the present disclosure relate to applications, platforms, architectures, etc. for automatically generating a reliable fix or patch for a reproducible defect in source code. Specifically, using one or more artificial intelligence (AI) models, a hint is processed based at least in part on the source code, the defect, and contextual information for the defect, thereby generating a revised version of the source code. Further, the applications, platforms, architectures, etc. can determine whether the revised version of the source code resolves the defect and apply the changes implemented in the revised version of the source code to the source code.

Description

Background Technology

[0001] Modern software development uses automated testing techniques, such as unit testing, integration testing, static analysis, and fuzzing, to identify defects in software systems. While these methods are effective at detecting potential problems, analyzing and fixing these problems still requires a significant amount of human effort from experienced software engineers. This process is often tedious and time-consuming.

[0002] In recent years, large language models (LLMs) have emerged as a potential tool for code analysis and bug fixing. These models, trained on large codebases, can analyze code snippets and propose fixes. However, current LLM-based solutions have some limitations. For example, a major limitation is their narrow scope, as these LLM-based solutions only operate at the function level and lack access to the broader program context, including dependencies, data types, and execution flow. This limited perspective can lead to incomplete or incorrect fixes. Furthermore, there is currently no automated verification mechanism for the generated patches, making manual verification by engineers essential. Attached Figure Description

[0003] The following will describe in detail the system and method for automating software patch generation with reference to the accompanying drawings, wherein: Figure 1 It is a schematic block diagram of software lifecycle management according to one or more aspects of this disclosure; Figure 2 It is based on one or more aspects of this disclosure Figure 1 Example diagram of defect patching module 135 used for automatic patch generation; Figure 3 It is based on one or more aspects of this disclosure Figure 2 Example diagram of the defect detection component in the image; Figure 4 It is based on one or more aspects of this disclosure Figure 2 Example diagram of the defect analysis component in the image; Figure 5 It is based on one or more aspects of this disclosure Figure 2 Example diagram of the patch recommendation component; Figure 6 It is based on one or more aspects of this disclosure Figure 2 Example diagram of the patch application component; Figure 7 It is based on one or more aspects of this disclosure Figure 2 Example diagram of the patch verification component; Figure 8 It is based on one or more aspects of this disclosure Figure 2Example diagram of the patch approval component in the document; Figure 9 It is a flowchart of automatically generating patches according to one or more aspects of this disclosure, and also a block diagram of an example generative language model system; Figure 10A It is a block diagram of an example generative language model system according to one or more aspects of this disclosure; Figure 10B It is a block diagram of an example generative language model including a transformer encoder-decoder according to one or more aspects of this disclosure; Figure 10C It is a block diagram of an example generative language model containing a decoder-only transformer architecture according to one or more aspects of this disclosure; Figure 11 It is a block diagram of an example computing device according to one or more aspects of this disclosure; Figure 12 This is a block diagram of an example data center according to one or more aspects of this disclosure. Detailed Implementation

[0004] The embodiments described herein relate to systems and methods for automatically generating reliable fixes or patches for reproducible defects in source code. Specifically, the system identifies defects, such as crashes, hangs, or anomalous behavior, by executing a test suite. The system generates detailed contextual information about the defect, including identifiers, file paths, function names, line numbers, defect types, stack traces, and log extracts. The system uses an AI model to process this contextual information to generate a corrected version of the source code. These corrections are verified by compiling and executing test cases on the corrected code. If no defects are detected, the changes are committed and integrated into the main code branch. If conflicts arise during integration, they are resolved using predefined reconciliation rules or manual intervention.

[0005] In some embodiments, source code is analyzed to find software defects. The source code can be copied and compiled into a binary file. This binary file can then be tested by executing one or more test cases in a test suite. Each test case in the test suite contains one or more test inputs, which can be input through input processing mechanisms such as command-line arguments, standard input, file-based input, network protocols, API calls, and environment variables. Each test input can be, for example, a variant of a valid input used to explore edge cases, strategically generated input to probe potential vulnerabilities, or input systematically derived from known format specifications or protocol definitions. During testing, diagnostic tools (e.g., cleanup tools) can be used to monitor execution to track code coverage, memory usage, and runtime behavior, providing detailed information about defects, and / or to provide source code analysis tools (e.g., libclang) to generate code contexts. Cleanup tools can include address cleanup tools, memory cleanup tools, thread cleanup tools, undefined behavior cleanup tools, leak cleanup tools, type cleanup tools, and / or real-time cleanup tools. Address cleanup tools detect memory corruption issues. Memory cleanup tools detect access violations to uninitialized memory. Thread analyzers identify synchronization issues such as race conditions and deadlocks. Undefined behavior cleanup tools detect undefined behavior patterns during code execution. Leak cleanup tools identify heap memory leaks at runtime. Type cleanup tools detect type alias conflicts at runtime. Real-time cleanup tools detect violations of real-time requirements, such as unsafe calls from functions with deterministic runtime requirements. Code context can include call graphs, function dependencies, and data types, providing deeper insight into the defective parts of the binary.

[0006] A defect is a situation where a binary file crashes, hangs, or exhibits other abnormal behavior. Each defect detected during test case execution can be logged as a test log. The test log includes contextual information for the corresponding detected defect. This contextual information includes, for example, diagnostic tool identifiers, test case identifiers, file paths, function names, line numbers, defect types, stack traces, structured data, log extracts, reproducible scenarios, or diagnostic data. In some embodiments, the test log for a corresponding detected defect may also include a trace log. A trace log is a detailed record of program execution, capturing function calls, system events, and variable states to help developers debug problems, monitor performance, and analyze system behavior.

[0007] Each test log can be analyzed to generate a defect analysis log, which can be included in a defect analysis report. The defect analysis log provides a detailed analysis of one or more corresponding detected defects in the respective test log, including, for example, the source code associated with the corresponding detected defect, a summary of the corresponding detected defect, the possible root cause of the corresponding detected defect, the historical context of the corresponding detected defect, and / or related recurring errors associated with the corresponding detected defect.

[0008] In some embodiments, the processing logic generates input requests (e.g., prompts) based on defect analysis logs for an artificial intelligence (AI) model (e.g., an LLM) to process the corresponding detected defects and provide corrections. In some embodiments, the input request may include a solution strategy to assist the AI ​​model in processing the corresponding detected defects in the defect analysis log. The input request can be submitted to the AI ​​model. The AI ​​model can output source code modifications. In some embodiments, source code modifications can be represented as code diffs, which show the differences between the source code and a corrected version of the source code. Code diffs can highlight additions, deletions, and / or modifications between the source code and the corrected version of the source code.

[0009] The corrected version of the source code can be analyzed to ensure that issues are properly resolved, code quality is good, implementation is correct, and / or errors are prevented (e.g., verifying the corrected source code). In some embodiments, the AI ​​model analyzes the corrected version of the source code in the role of an agent-as-judge and validates it against a predefined checklist to ensure proper integration, test coverage, and / or complete implementation (e.g., avoiding commented-out code). In some embodiments, another AI model (e.g., an additional LLM running as an LLM reviewer) performs validation and additional verification steps, separate from the AI ​​model that generated the corrected version of the source code. This additional AI model can check integration, test coverage, and / or implementation completeness, while also performing quality assessments of the approach, ranking multiple solutions, and / or verifying compliance with standardization guidelines beyond the predefined checklist. If the verification of the corrected version of the source code fails, the corrected version of the source code, along with feedback on the verification results (e.g., insufficient test coverage, remaining defects, etc.), can be provided to the AI ​​model to further refine the corrected version of the source code.

[0010] Once the verified version of the source code is successful, the code differences can be applied to a copy of the source code by analyzing the changes specified in the code differences. In some embodiments, lines marked as added are inserted, lines marked as removed are deleted, and / or modified lines are replaced with their updated versions. Thus, an updated version of the source code reflecting the expected changes can be generated.

[0011] An updated version of the source code can be compiled into an updated binary. The updated binary can be tested by re-executing one or more test cases in the test suite. If a new defect is detected in the corresponding test case, the processing logic can determine whether the newly detected defect matches a corresponding detected defect. If the newly detected defect matches a corresponding detected defect, the processing logic can update the test log of the corresponding detected defect to include additional contextual information about the newly detected defect. If the newly detected defect does not match a corresponding detected defect, the processing logic can generate a new test log containing contextual information (and its corresponding trace log) for the newly detected defect. In some embodiments, the processing logic updates the defect analysis log of the corresponding detected defect and / or generates a new defect analysis log for the newly detected defect. The processing logic can further refine the updated source code version based on the updated defect analysis log (or the new defect analysis log) (e.g., code differences used to generate the updated version of the source code). Once the corresponding detected defect (or the new defect analysis log) is resolved, the code differences can be submitted for approval as a suggested solution for fixing or patching the source code. Upon receiving approval for the code differences, the processing logic can apply the code differences to the source code to correct it. In some embodiments, the processing logic can automatically review and apply the code differences to correct the source code without receiving approval. Therefore, in some embodiments, code correction can be performed without any user input (e.g., the entire process of defect detection, isolation, patching, and review is automated).

[0012] In some embodiments, the systems and methods described herein can be performed in a simulated environment (e.g., NVIDIA DriveSIM, NVIDIA ISAAC GYM, NVIDIA ISAAC SIM, etc.) using simulated data (e.g., simulated sensor data from simulated sensors of virtual or simulated machines). For example, simulated sensor data (e.g., processed using one or more machine learning models, neural networks, etc.) can be used to identify, detect, and / or classify lane lines, road boundary lines, other lines, vertical structures / features, etc., in a simulated environment by using points on curves and / or one or more curve fitting algorithms, and this information can be used to perform operations associated with virtual machines in the environment (e.g., control, navigation, planning, etc.). These simulated operations can be used to test the performance of underlying algorithms, systems, and / or processes before deploying them to the real world. In some cases, simulation can be used to generate synthetic training data—e.g., training data containing regions of interest and / or subregions of interest in a simulated environment. In some embodiments, other methods besides simulation or as an alternative to simulation can be used to generate synthetic training data. For example, synthetic training data can be generated using neural rendering fields (NERF), Gaussian sputtering techniques, diffusion models, electrostatic models (e.g., Poisson flow generative models (PFGM), etc.). This synthetic training data (as supplementary or alternative to real-world data) can then be processed to determine geometry, curvature, semantic information, classification information, and / or other information relevant to features of interest, such as lines, longitudinal features (e.g., telephone poles), and / or other features in environments like driving environments or warehouses. In any example, such as using a simulated environment for testing, validation, training, etc., one or more optical propagation algorithms (e.g., ray tracing and / or path tracing algorithms) can be used to render or otherwise generate the simulated environment and / or associated training data. In some embodiments, the simulated environment and / or one or more of its objects, features, or components can be generated or managed in a 3D content collaboration platform (e.g., NVIDIA's OMNIVERSE) for industrial digitization, generative physics AI, and / or other use cases, applications, or services. For example, a content collaboration platform or system may include a system that uses generic scene descriptors (USD) (such as OpenUSD) data to manage objects, features, scenes, etc., in simulated environments, digital environments, etc. The platform may incorporate realistic physical simulations, such as using NVIDIA's PhysX SDK, to simulate real physics and physical interactions in platform-hosted simulations.This platform can integrate OpenUSD with ray tracing / path tracing / light transport simulations (such as NVIDIA's RTX rendering technology) into software tools and simulation workflows for building, training, deploying, or testing AI systems—such as systems for testing, validating, training (e.g., machine learning models, neural networks, etc.) and / or other tasks related to automobiles, robots, machines, or other applications.

[0013] In some embodiments, remote operation or control of vehicles or other machines can be performed using remote control or a remote operating system. For example, the systems and methods described herein can be used to identify lane lines, road boundary lines, longitudinal features, etc., which can be included in the visualization or mapping of the environment to help remote operators control autonomous or semi-autonomous machines in the environment, or to provide waypoints or other control or navigation instructions.

[0014] In some examples, the machine learning models described herein (e.g., deep neural networks, language models, LLMs, VLMs, multimodal language models, visual language action (VLA) models, perceptual models, tracking models, fusion models, transformer models, diffusion models, encoder-only models, decoder-only models, encoder-decoder models, neural rendering field (NERF) models, etc.) can be packaged as microservices—such as inference microservices (e.g., NVIDIA NIM)—which may include containers (e.g., operating system (OS) level virtualization packages) that may include an application programming interface (API) layer, a server layer, a runtime layer, and / or a model “engine.” For example, an inference microservice may include the container itself and the model (e.g., weights and biases). In some cases, such as when the machine learning model is small enough (e.g., has a sufficiently small number of parameters), the model may be included within the container itself. In other examples, such as when the model is large, the model may be hosted / stored in the cloud (e.g., in a data center) and / or hosted locally and / or at the edge (e.g., on a local server or computing device, but outside the container). In these embodiments, the model may be accessed via one or more APIs, such as REST APIs. Therefore, in some embodiments, the machine learning model described herein may be deployed as an inference microservice to accelerate model deployment on any cloud, data center, or edge computing system while ensuring data security. For example, the inference microservice may include one or more APIs, pre-configured containers for simplified deployment, an optimized inference engine (e.g., software builds using standardized AI model deployment and execution, such as NVIDIA's Triton Inference Server, and / or one or more APIs for high-performance deep learning inference, which may include inference runtime and model optimizations to provide low latency and high throughput for production applications, such as NVIDIA's TensorRT), and / or enterprise management data for telemetry (e.g., including identity, metrics, health checks, and / or monitoring). The machine learning model described herein may be deployed as part of a microservice, along with accelerated infrastructure that has single-command deployment capabilities and can be orchestrated and automatically scaled on accelerated infrastructure via container orchestration systems (e.g., scaling to data center scale on a single device). Therefore, the inference microservice may include a machine learning model (e.g., a model optimized for high-performance inference), inference runtime software for executing the machine learning model and providing output / response to inputs (e.g., user queries, prompts, etc.), and enterprise management software for providing health checks, identity verification, and / or other monitoring. In some embodiments, the inference microservice may include software for in-situ replacement and / or updating of the machine learning model. During replacement or updating, the software performing the replacement / update maintains the user configuration of both the inference runtime software and the enterprise management software.

[0015] In some embodiments, the systems and methods described herein can be deployed in voice or smart self-service terminal applications. For example, a self-service terminal, tablet, smart display, or other device may include one or more onboard processors (e.g., CPU, GPU, deep learning accelerator, SoC) and memory and / or storage devices (e.g., for storing models, image databases, etc.). In some embodiments, the self-service terminal / tablet / display may use one or more network interface cards (NICs) and / or data processing units (DPUs) to communicate with one or more locally hosted servers / computing devices and / or one or more remote servers / computing devices (e.g., located in one or more data centers). In such examples, the self-service terminal may use one or more APIs (such as, but not limited to, REST APIs) to communicate with machine learning models (e.g., language models, LLM, VLM, MMLM, diffusion models, transformer models, NeRF, DNN, etc.) and / or image databases hosted on local and / or remote servers.

[0016] In one or more embodiments, the systems and methods described herein can be deployed in gaming applications. For example, game consoles, PCs, tablets, or other gaming devices may include one or more onboard and / or remote processors (e.g., CPUs, GPUs, deep learning accelerators, SoCs) and memory and / or storage devices (e.g., for storing game models, game resources, player data, etc.). These devices may use one or more machine learning models (e.g., diffusion models, converter models, neural radiation field (NeRF) models, language models (e.g., LLM, VLM, MMLM, etc.), DNNs, etc.) to enhance gameplay, generate real-time dynamic content, and personalize the user experience based on in-game behavior or pre-stored player profiles. In some embodiments, these systems may be deployed in cloud gaming environments (e.g., NVIDIA's GeForce Now). In this case, client devices (e.g., smart displays, tablets, or game controllers) can be used to interact with the game, while machine learning models and / or visual rendering may be performed on one or more remote servers / computing devices (e.g., in one or more data centers). The language model, AI processing, and rendering described in this article can run in the cloud, processing player input received from end-user devices (e.g., based on controller, keyboard, mouse, joystick, AR / VR / MR, etc.), generating appropriate in-game responses, rendering content, and sending or transmitting the content to the end-user device. One or more data processing units (DPUs) and / or network interface cards (NICs) can be used when receiving and / or sending data from end-users or edge devices.

[0017] In some embodiments, the systems and methods described herein can be deployed in video conferencing applications. For example, video conferencing devices (such as dedicated conferencing units, computers, tablets, and / or smartphones) may include one or more onboard processors (e.g., CPUs, GPUs, deep learning accelerators, SoCs) and memory and / or storage devices (e.g., for storing video, audio, or other communication-related data). These systems may use machine learning models (e.g., diffusion models, transducer models, neural radiation field (NeRF) models, language models (e.g., LLM, VLM, MMLM, etc.)) to enhance video conferencing capabilities, including real-time or near real-time transcription, speaker segmentation, language translation, automatic speech recognition (ASR), and / or background noise reduction. In one or more embodiments, these systems enable users to interact with the video conferencing platform using natural language input. For example, users may issue voice commands to schedule, join, or leave a meeting, or manage participants and screen sharing. One or more data processing units (DPUs) and / or network interface cards (NICs) may be used when receiving and / or sending data from end users or edge devices.

[0018] In some embodiments, the systems and methods described herein can be deployed in robotic applications. For example, a robot or robotic system may include one or more onboard processors (e.g., CPU, GPU, hardware-based deep learning accelerator (DLA), hardware-based programmable vision accelerator (PVA) (which may include one or more vector processing units (VPU), direct memory access (DMA) systems, and / or pixel processing engines (PPE), hardware-based optical flow accelerators (OFA), SoC, etc.) and memory and / or storage devices (e.g., for storing control algorithms, sensor data, and one or more machine learning models). The robotic system can use these processors to execute one or more machine learning models (e.g., language models), visual language models (VLM), large language models (LLM), visual-language-action (VLA) models, multimodal language models (MMLM), etc., enabling it to autonomously or semi-autonomously perform complex tasks, such as interacting with and / or manipulating static and / or dynamic objects, or navigating in an environment using sensors such as cameras, LiDAR, RADAR, and ultrasonic sensors. The system may use sensor fusion technology to combine data from multiple sensors. Data from sensors (e.g., cameras, infrared, LiDAR, RADAR, accelerometers) is combined to create a comprehensive model of the robot's surrounding environment. This data can be processed locally on the robot or sent to a remote server to perform computationally intensive tasks such as 3D mapping or simultaneous localization and mapping (SLAM). In one or more embodiments, data from a single robot (e.g., sensor data, task status, or environmental conditions) can be uploaded to the cloud, where a centralized AI model analyzes the data and distributes optimization commands to the entire fleet. In some embodiments, machine learning models described herein (e.g., language models, VLM, VLA, LLM, MMLM, diffusion models, NeRF models, DNN, etc.) can be used to enable the robot to perceive and reason about its environment, and / or communicate with one or more other robots and / or humans in the environment. In some embodiments, the robot can communicate with one or more locally hosted servers / computing devices and / or one or more remote servers / computing devices (e.g., located in one or more data centers) using one or more network interface cards (NICs) and / or data processing units (DPUs).

[0019] In some embodiments, the systems and methods described herein may be deployed in in-vehicle infotainment (IVI) systems or in-vehicle experience (IX) applications. For example, an infotainment system within a vehicle (e.g., a car, truck, drone, construction equipment, robot, semi-autonomous vehicle, or autonomous vehicle) may include one or more onboard processors (e.g., CPU, GPU, hardware-based deep learning accelerator (DLA), hardware-based programmable vision accelerator (PVA) (which may include one or more vector processing units (VPU), direct memory access (DMA) systems, and / or pixel processing engines (PPE), hardware-based optical flow accelerators (OFA), SoCs, etc.) and memory and / or storage devices (e.g., for storing control algorithms, sensor data, and one or more machine learning models) and memory and / or storage devices (e.g., for storing entertainment content, navigation data, and user preferences). The system can use these processors to execute one or more machine learning models (e.g., language models) to enable functions such as voice control, personalized media recommendations, dynamic navigation, and real-time communication with other services via network connectivity. The onboard infotainment system may also use natural language processing (NLP) models to enable voice-based interaction. One or more machine learning models may be stored locally or accessed via one or more APIs that connect to cloud services, enabling the system to process requests in real-time or near real-time.

[0020] While this article may describe examples of using machine learning models (such as neural networks), this is not intended to limit the scope. For example, but not limited to, any of the various machine learning models and / or neural networks described herein can include any type of machine learning model, such as those using linear regression, logistic regression, decision trees, support vector machines (SVM), Naive Bayes, k-nearest neighbors (KNN), k-means clustering, random forests, dimensionality reduction algorithms, gradient boosting algorithms, neural networks (e.g., autoencoder neural networks, artificial neural networks (ANN), convolutional neural networks (CNN), recurrent neural networks (RNN), perceptrons, long short-term memory (LSTM) networks, multilayer perceptron (MLP) networks, deep stacked networks (DSN), generative pre-trained (GPT) models or networks, feedforward networks, radial basis function ANNs, self-organizing maps (SOM), Kohonen maps, Hopfield networks, Boltzmann machines, deep belief neural networks, deconvolutional neural networks, generative adversarial networks (GANs), liquid... State machines, modular neural networks, sequence-to-sequence models, networks using transformer architectures, state-space models (SSM) (e.g., networks using Mamba architectures (e.g., Mamba-1, Mamba2, etc.), networks using selective state-space models, networks using structured state-space sequence models, etc.), diffusion models (e.g., diffusion probability models, score-based generative models, etc.), neural radiation field (NeRF) models, Gaussian sputtering models, Kolmogorov-Arnold networks (KAN), encoder-only architecture models, decoder-only architecture models, encoder-decoder architecture models, generative machine learning models, language models, large language models (LLM), visual language models (VLM), multimodal language models (MMLM), large action models (LAM), visual-language-action (VLA) models, etc.) and / or other types of machine learning models.

[0021] In some embodiments, one or more Transformer Engines (TEs) may be implemented. Transformer Engines can utilize micro-tensor scaling to optimize performance and accuracy, such as enabling 16-bit floating-point (FP16), 8-bit floating-point (FP8), and / or 4-bit floating-point (FP4) AI processing. For example, a Transformer Engine can use 16-bit or 8-bit floating-point precision and 8-bit or 4-bit floating-point data formats, combined with software algorithms, to improve the performance and capabilities of AI. By reducing mathematical operations to 8 or 4 bits, TEs can train larger networks faster without sacrificing accuracy. For example, TEs may include libraries for accelerating Transformer Models on processing devices such as GPUs, thereby providing better performance with lower memory utilization during training and inference. When TEs are used in conjunction with other technologies, such as high-speed interconnects between nodes (e.g., using switches, such as NVLink Switches) and Tensor Cores that support mixed-precision computations, such as micro-scaling precision support, server clusters may be more capable of training large networks (e.g., billions of parameters) at high speeds. Therefore, it can support tensor core precision for FP64, TF32, BF16, FP16, FP8, INT8, FP6 and FP4, as well as CUDA core precision for FP64, FP32, FP16 and BF16.

[0022] The advantages of this disclosure include, but are not limited to, automatic defect detection and automatic generation of reliable software patches for detected defects, significantly reducing the time and effort required to correct software and / or firmware errors compared to manual correction processes. Furthermore, the embodiments facilitate seamless integration and testing of code modifications, minimizing human intervention, thereby improving efficiency and ensuring higher accuracy in problem resolution.

[0023] Figure 1 A schematic block diagram of a software lifecycle management (SLM) system 100 that performs automated patch generation according to one or more aspects of this disclosure is shown. The SLM system 100 manages the iterative process of software development, in which developers create, test, and refine computer-readable instructions (referred to as source code) to implement functionality. Among other components, the SLM system 100 includes a repository 110, a development engine 120, and a testing engine 130.

[0024] Repository 110 implements a storage structure for organizing and tracking software development artifacts (“artifacts”), where source code files are the primary artifacts of development operations. In some embodiments, the storage structure is hierarchical, organizing artifacts into directories and subdirectories. Repository 110 tracks individual artifacts and maintains associated metadata, which may include file paths, permissions, timestamps, and author information. These artifacts include source code files and may include supporting elements such as configuration files, build scripts, documentation, and other elements required for project development.

[0025] Repository 110 employs a version tracking structure to maintain relationships between different versions of artifacts, particularly between different versions of source code files. In some embodiments, the version tracking structure is implemented using a directed acyclic graph (DAG), where nodes in the DAG represent specific versions and edges represent relationships between versions. The version tracking structure stores version states, maintains version history, and implements state management mechanisms for modifying version data. These mechanisms ensure data consistency during concurrent access to and modification of the version tracking structure.

[0026] Development engine 120 manages software development operations (“development operations”) by coordinating version control and development workflows. Development engine 120 performs read and write operations on the storage structure and version tracking structure of repository 110. In some embodiments, development engine 120 includes a version control (“VC”) module 122 that performs basic version control operations. These VC module 122 operations include, but are not limited to, creating artifact versions (“commits”), tracking changes between commits, and maintaining relationships between different version artifacts. VC module 122 retrieves version status from the version tracking structure of repository 110 and updates the version status when a commit is created or a version relationship is modified. VC module 122 implements branch creation operations, tracks the status of each branch, and maintains relationships between branches. In some embodiments, VC module 122 designates certain branches as master branches, which maintain stable versions of artifacts. When integrating changes from other branches, VC module 122 implements conflict detection and resolution mechanisms to maintain code integrity.

[0027] The test engine 130 works in conjunction with the development engine 120 to validate changes to artifacts before they are integrated into the main branch. The test engine 130 executes test suites, analyzes the test results, and provides a validation status to the development engine 120. In some embodiments, the development engine 120 uses this validation status to determine whether changes can be merged into the branches.

[0028] In some embodiments, the test engine 130 includes a defect patching module 135. The defect patching module 135 may be implemented using a processor and optionally one or more other components or engines (e.g., development engine 120).

[0029] Figure 2 It conforms to one or more aspects of this disclosure. Figure 1The example block diagram of the defect patching module 135 shown is used for automatically generating patches. In some embodiments, the defect patching module 135 includes a defect detection component 210, a defect analysis component 220, a patch recommendation component 230, a patch application component 240, a patch verification component 250, and a patch approval component 260. It should be understood that the operations of one or more of the defect detection component 210, defect analysis component 220, patch recommendation component 230, patch application component 240, patch verification component 250, and / or patch approval component 260 can be combined into a single component. Furthermore, or alternatively, the operations of one or more of the defect detection component 210, defect analysis component 220, patch recommendation component 230, patch application component 240, patch verification component 250, and / or patch approval component 260 can be split into multiple components.

[0030] In some embodiments, the defect patching module 135 is activated in response to a commit operation in the VC module 122, such as when a proposal is made to integrate changes to source code files into the main branch. This component can also be activated as part of a continuous integration and continuous delivery (CI / CD) process at predetermined time intervals, or manually triggered during development. The CI / CD process automates the building, testing, and deployment of changes to source code files and related artifacts to ensure consistent verification and delivery.

[0031] Once the defect patching module 135 is triggered, the defect detection component 210 initiates tests on the source code to verify syntax, functionality, interaction, and / or performance. The defect detection component 210 may execute a test suite on the source code to identify defects and provide a test report. The test report is a detailed record of the test suite and its results, including information about the detected defects. The test report can provide valuable insights into the behavior and quality of the tested source code. In some embodiments, the defect detection component 210 provides the test report, along with execution-related information of the test suite (e.g., trace logs), to the defect analysis component 220.

[0032] The defect analysis component 220 analyzes the test report and performs related tasks to generate a defect analysis report. The defect analysis report is a comprehensive document detailing the analysis of the defects detected in the test report, including their identification, root causes, and historical patterns. The defect analysis component 220 then provides the defect analysis report to the patch recommendation component 230.

[0033] The patch recommendation component 230 uses one or more artificial intelligence (AI) models, such as a large language model (LLM)-based code generator or a rule-based engine, to generate source code modifications (optionally represented as code differences) to address each detected defect identified by the defect detection component 210. The patch recommendation component 230 may further validate the code differences to ensure compliance with coding standards, proper integration of code differences, and / or comprehensive test coverage. The patch recommendation component 230 may further refine one or more code differences until they are validated. In some embodiments, the patch recommendation component 230 provides the code differences to the patch application component 240.

[0034] Patch application component 240 can create a new branch (e.g., a fix branch) in version control component (VC module) 122 to allow for independent modifications from the main branch. Patch application component 240 can apply code differences and create commits containing descriptive messages. In some embodiments, patch application component 240 pushes changes to repository 110 and generates a pull request to formally integrate the source code changes in the fix branch into the main branch.

[0035] In some embodiments, patch verification component 250 initiates testing on a fix branch (e.g., modified source code in a fix branch generated from source code modifications) in response to the generation of a pull request. Similar to defect detection component 210, patch verification component 250 executes a test suite on the modified source code to identify defects. Patch verification component 250 may modify the test report to reflect the resolution of previously detected defects, the recurrence of previously detected defects, and / or the introduction of new defects. If the test report indicates that previously detected defects have been resolved and no new defects have been introduced, patch verification component 250 notifies patch approval component 260 that the modified source code in the fix branch has passed verification. Otherwise, patch approval component 260 may provide the modified test report to defect analysis component 220 to repeat the process. In other words, the operations of defect analysis component 220, patch recommendation component 230, patch application component 240, and patch verification component 250 may be repeated until the modified source code in the fix branch passes verification.

[0036] In one embodiment, upon receiving notification that the modified source code from the fix branch has passed verification, the patch approval component 260 notifies the designated reviewer that a pull request to formally integrate the source code modifications in the fix branch into the main branch is ready. The patch approval component 260 may receive input from the designated reviewer. This input may indicate approval, rejection, or a request for further modifications to the source code in the fix branch. The patch approval component 260 may then cause the development engine 120 to merge the source code modifications from the fix branch into the main branch. In some embodiments, the patch approval component 260 may automatically determine whether to approve, reject, or request additional modifications without the involvement of a designated reviewer. Therefore, the patch approval component 260 can use an automated decision-making mechanism to cause the development engine 120 to merge the source code modifications from the fix branch into the main branch without first receiving user input.

[0037] Figure 3 It is based on one or more aspects of this disclosure Figure 2 The diagram shows an example of the defect detection component 210. In one embodiment, the defect detection component 210 includes a test execution subcomponent 310, a defect detection subcomponent 320, and a report generation subcomponent 330.

[0038] Test execution subcomponent 310 provides a structured environment in which test suites are executed, monitored, and evaluated. This environment supports various types of testing or test strategies. Unit tests examine individual components in isolation. Integration tests verify that components work together correctly. Functional tests check whether the system meets user requirements. System tests evaluate the behavior of the entire system. Performance tests measure how the system handles different loads. Security tests identify vulnerabilities. Regression tests ensure that new changes do not break existing functionality. Static analysis tests examine code without running it. Fuzzing uses random input to find unexpected behavior. Test suites (e.g., fuzzing suites) include multiple test cases, each containing one or more test inputs. Test inputs may include variants of valid inputs to the software under test for exploring boundary conditions; they may also include strategically generated inputs for probing potential vulnerabilities; and / or may include inputs derived from format specifications and / or protocol definitions. In embodiments, test execution subcomponent 310 manages system resources, establishes the test environment, injects test inputs, and / or captures execution outputs. Test execution subcomponent 310 can enable diagnostic tools (e.g., cleanup tools) to provide detailed information about defects. Alternatively, the test execution subcomponent 310 may also enable source code analysis tools (such as libclang) to generate a code context. Cleanup tools may include address cleaners, memory cleaners, thread cleaners, undefined behavior cleaners, leak cleaners, type cleaners, and / or real-time cleaners. Address cleaners detect memory corruption issues. Memory cleaners detect violations of access to uninitialized memory. Thread analyzers identify synchronization issues, such as race conditions and deadlocks. Undefined behavior cleaners detect undefined behavior patterns during code execution. Leak cleaners identify heap memory leaks at runtime. Type cleaners detect type alias violations at runtime. Real-time cleaners detect violations of real-time requirements, such as unsafe calls from functions with deterministic runtime requirements. The code context may include a call graph, function dependencies, and / or data types to gain deeper insight into the defective parts of the software (e.g., binaries). In an embodiment, the test execution subcomponent 310 executes a test suite containing multiple test cases designed to verify functionality and identify defects.

[0039] In this embodiment, the test execution subcomponent 310 creates a copy of the source code and compiles it into a binary file, while adding appropriate instrumentation code for code coverage tracking, memory usage monitoring, and / or runtime behavior observation. The test execution subcomponent 310 can provide test input through various input processing mechanisms, including command-line arguments, standard input, file-based input, network protocols, API calls, and environment variables.

[0040] In this embodiment, for each test case, during the test execution of the test suite, the defect detection subcomponent 320 evaluates the execution results, diagnostic tool outputs, and system responses to identify defects in the corresponding test case. Defects may manifest as exception return codes, diagnostic tool findings, assertion failures, error outputs, crashes, timeouts, and / or exceptions. In some embodiments, a corresponding trace log may be generated for each test case. The trace log is a detailed record of program execution, capturing function calls, system events, and variable states to help developers debug problems, monitor performance, and analyze system behavior.

[0041] For each defect detected in the corresponding test case, the defect detection subcomponent 320 can provide the report generation subcomponent 330 with contextual information about the detected defect. Contextual information may include diagnostic tool identifiers, test case identifiers, file paths, function names, line numbers, defect types, stack traces, structured data, log extracts, reproducible scenarios, diagnostic data, etc. The diagnostic tool identifier indicates the diagnostic tool that detected the defect. The test case identifier indicates which test case found the defect. The function name indicates which function contains the defect. The file path indicates which source code file contains the defect. The line number indicates which line in the source code contains the defect. The defect type indicates what type of defect occurred, such as heap-use-after-free or data race. The stack trace indicates which function call sequence caused the defect. Structured data (generated by the source code analysis tool) may include a call graph showing function relationships, dependencies between files and types, and data type definitions and their relationships and layouts. Log extracts capture relevant portions of error logs generated during test execution, analyzer output, system, application, debugging, and / or execution, providing additional debugging context. Reproducing the scenario can indicate which inputs and conditions led to the defect. Diagnostic data can indicate the memory state, performance metrics, and environmental conditions present at the time the defect occurred.

[0042] The report generation subcomponent 330 creates a test report in response to the first execution of the test suite on the source code. During test report creation, the report generation subcomponent 330 includes test suite metadata at the beginning of the test report. Test suite metadata may include information about the test suite, such as a change reference identifier, test suite identifier, timestamp, and / or environment information. The change reference identifier can indicate which commit, pull request, and / or build initiated the test. The test suite identifier can uniquely identify a specific test suite. The timestamp indicates the start time of the test suite. Environment information may include detailed information about compiler flags, operating system, and / or other system configurations. The test report can be generated in plain text or JSON format.

[0043] When the report generation subcomponent 330 receives context information for each detected defect, it includes or generates a test log, which can be added to the test report, containing the received context information (and its corresponding trace log).

[0044] Figure 4 It is based on one or more aspects of this disclosure Figure 2 An example diagram of the defect analysis component 220 is shown. In one embodiment, the defect analysis component 220 includes a defect classification subcomponent 410, a context generation subcomponent 420, and a report generation subcomponent 430.

[0045] For each test log in the test report, the defect classification subcomponent 410 can identify the root cause of the defect associated with the corresponding test log. In some embodiments, the defect classification subcomponent 410 generates a summary of the corresponding test log based on the contextual information of the defect associated with the corresponding test log. This summary may include a diagnostic tool identifier, defect type, file path, and / or line number, all derived from the contextual information of the defect associated with the corresponding test log. For example, the summary may use a template such as: "[Diagnostic tool identifier] reports the presence of [defect type] at [line number] in [file path]". In some embodiments, the defect classification subcomponent 410 identifies the possible root cause category of the corresponding test log based on the defect type. Possible root cause categories may include, for example, mismanagement of memory, concurrency, logical errors, undefined behavior, special cases, and / or other defect types. The mismanagement of memory category may include, for example, heap use after free, buffer overflow, invalid memory access, and / or other memory-related errors. The concurrency category may include synchronization defects, such as data races, deadlocks, and / or race conditions. Logical error categories can include programming errors, such as off-by-one errors, null pointer dereferences, and / or computational errors. Undefined behavior categories include operations not defined by the programming language specification. Special case categories include edge cases and boundary conditions that trigger defects.

[0046] In an embodiment, the context generation subcomponent 420 retrieves historical versions of source code from repository 110. These historical versions are generated from commits, merges, and / or fixes of functions similar to functions identified by function names in the context information of defects associated with the corresponding test logs. The context generation subcomponent 420 can extract functions from these historical versions of source code as the historical context of the corresponding test logs. The context generation subcomponent 420 can retrieve source code files using file paths in the context information of defects associated with the corresponding test logs. In some embodiments, the context generation subcomponent 420 maintains a record of recurring errors and their corresponding defects. Recurring errors refer to defects that manifest in a similar manner in different parts of the source code, characterized by common defect types, execution patterns, and / or coding practices that lead to the defects. For example, recurring errors and / or defects may include common concurrency errors, memory usage patterns known to cause leaks, and / or frequently encountered "one-off" scenarios. Therefore, the context generation subcomponent 420 can identify recurring errors associated with the corresponding test logs by referring to the record of recurring errors to determine whether a defect associated with the corresponding test log matches a recurring error in the record of recurring errors. In other words, recurring error records can provide additional insights into whether historical problem patterns indicate the existence of similar problems.

[0047] In this embodiment, the context generation subcomponent 420 provides the report generation subcomponent 430 with a summary of the corresponding test log, the possible root cause category of the corresponding test log, the historical context of the corresponding test log, and / or recurring errors related to the corresponding test log.

[0048] In this embodiment, the report generation subcomponent 430 creates a defect analysis report in response to the creation of a test report. During the creation of the defect analysis report, the report generation subcomponent 430 includes test suite metadata at the beginning of the defect analysis report. The defect analysis report can be generated in plain text or JSON format. Other structured or unstructured data formats can also be used. When the report generation subcomponent 430 receives a summary of the corresponding test log, the possible root cause category of the corresponding test log, the historical context of the corresponding test log, and / or the related duplicate errors of the corresponding test log, the report generation subcomponent 430 can add the defect analysis log of the corresponding test log to the defect analysis report.

[0049] In an embodiment, the defect analysis log for the corresponding test log includes a defect identifier associated with the corresponding test log, source code, a summary of the corresponding test log, a possible root cause category of the corresponding test log, the historical context of the corresponding test log, and / or related recurring errors of the corresponding test log (provided by the context generation subcomponent 420). The defect identifier can be used to uniquely reference each specific defect in the test report. In some embodiments, the defect identifier is assigned incrementally as defects in the test report are analyzed. Depending on the embodiment, the defect analysis log for the corresponding test log may include metadata (e.g., references to previously reported issues) of previously reported problems that may be related to any aspect of the defect associated with the corresponding test log.

[0050] Figure 5 It is based on one or more aspects of this disclosure Figure 2 The diagram shows an example of the patch recommendation component 230. In one embodiment, the patch recommendation component 230 includes a patch strategy selection subcomponent 510, a patch generation subcomponent 520, and a verification subcomponent 530.

[0051] For each defect analysis log in the defect analysis report, the patch strategy selection subcomponent 510 can use the possible root cause category in the corresponding defect analysis log to identify the corresponding solution strategy. In an embodiment, each possible root cause category corresponds to a solution strategy. The solution strategy may include a memory safety solution strategy for the possible root cause category of mismanagement, a concurrency solution strategy for the possible root cause category of concurrency issues, and / or a logic solution strategy for the possible root cause category of logical errors. Memory safety solution strategies may include boundary checks, adjusting pointer lifetimes, and / or switching to safer function calls. Concurrency solution strategies may include introducing locks, atomic operations, and / or reordering instructions. Logic solution strategies may include changing conditional statements and / or handling corner cases.

[0052] The patch generation subcomponent 520 includes a patch generation artificial intelligence (AI) model 525. The patch generation AI model 525 may be a large language model (LLM)-based code generator used to generate source code modifications that can be used to correct defects captured in the defect analysis report. In some embodiments, the patch generation AI model 525 described herein may use one or more generative language models (e.g., such as...). Figures 10A-10C The above), one or more computing devices or components thereof (e.g., such as Figure 11 (as described above) and / or one or more data centers or components thereof (e.g., such as...) Figure 12 This is achieved through (as described above).

[0053] In one embodiment, for each defect analysis log in the defect analysis report, the patch generation subcomponent 520 provides a prompt for the corresponding defect analysis log to the patch generation AI model 525. This prompt may include the corresponding defect analysis log and optionally include the solution strategy for that defect analysis log.

[0054] The patch generation AI model 525 can process the provided input and output one or more suggestions for fixing the source code (e.g., source code modifications). In some embodiments, source code modifications for a corresponding defect analysis log are represented as code diffs of the corresponding defect analysis log, showing the differences between two versions of the source code (e.g., the difference between the current version of the source code and the suggested version). Source code diffs typically indicate modifications, additions, and / or deletions in the source code. For example, inserted lines can be marked with "+", "+a", or "a+", while deleted lines can be marked with "-", "-a", or "a-+". In some embodiments, the patch generation subcomponent 520 can provide a brief explanation of the purpose of the source code modifications for the corresponding defect analysis log and / or defect. For example, detailed information on how to implement a specific solution strategy to fix the defect associated with the corresponding defect analysis log can be provided. If needed, the patch generation subcomponent 520 can prompt the AI ​​model 525 to create additional test cases for the test suite. For example, the AI ​​model can identify and generate test defects and the missing test cases required for their fix. If existing test cases cannot fully test defects and their fixes, subcomponent 520 can also prompt AI model 525 to modify existing test cases. These additional test cases and modifications ensure comprehensive verification and coverage of source code changes.

[0055] In an embodiment, the verification subcomponent 530 performs preliminary verification on suggested source code modifications (e.g., code differences) for a corresponding defect (e.g., for a corresponding defect analysis log) to analyze whether the changes conform to established coding standards, ensure correct integration and test coverage, detect regressions, and / or verify a complete implementation, avoiding unintentionally disabled code (e.g., avoiding commented-out code). Coding standards may include project-specific checklists and / or industry standard practices. In some embodiments, the preliminary verification of source code modifications (e.g., code differences) for a corresponding defect analysis log may be performed by a patch generation AI model 525 (agent-as-judge). In some embodiments, the patch generation AI model 525 outputs the status of the preliminary verification of the code differences for the corresponding defect analysis log (e.g., pass or fail) and detailed information about the reasons for the preliminary verification status in response to prompts.

[0056] In some embodiments, the verification subcomponent 530 may include a patch verification AI model 535 (e.g., which may act as an LLM-as-judge), designated to perform preliminary verification of source code modifications (e.g., code differences) to corresponding defects (e.g., defect analysis logs) generated by the patch generation AI model 525. LLM-as-judge (or LLM as a judge) is a method in which one or more models (e.g., the patch verification AI model 535) are used to evaluate outputs, which are typically derived from other models (e.g., the patch generation AI model 525). In some embodiments, the patch verification AI model 535 described herein may use one or more generative language models (e.g., such as...). Figures 10A-10C The above), one or more computing devices or components thereof (e.g., such as Figure 11 (as described above) and / or one or more data centers or components thereof (e.g., such as...) Figure 12 This is achieved through (as described above). In an embodiment, the verification subcomponent 530 provides hints to the patch verification AI model 535.

[0057] In some embodiments, the patch verification AI model 535, similar to the patch generation AI model 525, outputs a preliminary verification status and detailed information about the reasons for the preliminary verification status. Furthermore, the output of the patch verification AI model 535 may include additional preliminary verifications, such as checking integration, test coverage, and implementation integrity, while also performing method quality assessments, ranking multiple solutions, and verifying their compliance with standardization guidelines beyond coding specifications.

[0058] In response to an unsuccessful initial verification status (e.g., a failure result), the verification subcomponent 530 can provide another hint to the patch generation AI model 525 to output further improved source code modifications, which can be used to correct defects based on details about the reason for the initial verification status. In response to a successful initial verification status (e.g., a pass result), the verification subcomponent 530 can notify the patch application component 240 that the initial verification was successful.

[0059] Figure 6 It is based on one or more aspects of this disclosure Figure 2 An example diagram of the patch application component 240 is shown. In an embodiment, the patch application component 240 includes a branch creation subcomponent 610, a conflict resolution subcomponent 620, and a pull request creation subcomponent 630.

[0060] In response to the patch component 240 receiving a notification of initial successful verification, the branch creation subcomponent 610 can create a repair branch in VC122. In some embodiments, the repair branch may have a predefined naming convention indicating that it is created for a defect associated with a corresponding defect analysis log. For example, "auto-fix-defect-" + defect identifier from the corresponding defect analysis log + "-" + diagnostic tool identifier from the corresponding defect analysis log. In some embodiments, the branch creation subcomponent 610 applies source code modifications (e.g., code differences from the corresponding defect analysis log) to the repair branch by analyzing differences between files and updating the contents of the repair branch accordingly. As previously described, code differences specify which lines of code to add, modify, and / or delete in a particular file. In some embodiments, the branch creation subcomponent 610 processes these instructions sequentially, making the specified changes to the files in the repair branch while preserving any unmodified content.

[0061] If a conflict arises between the fix branch and the main branch, the conflict resolution subcomponent 620 performs conflict resolution. A conflict occurs if changes in the fix branch overlap or conflict with changes in the main branch, preventing automatic merging. In an embodiment, the conflict resolution subcomponent 620 attempts to resolve the conflict by programmatically merging changes using predefined reconciliation rules. For example, predefined reconciliation rules might include accepting the latest changes, retaining all changes in a specified order, or applying a language-specific resolution strategy, for example, based on the type of code conflict. If the conflict cannot be resolved, the conflict resolution subcomponent 620 can flag the conflict for manual intervention.

[0062] In one embodiment, once the code diffs from the corresponding defect analysis logs are applied, the pull request creation subcomponent 630 creates a commit in VC 122 to establish a permanent snapshot of the changes, making them traceable and rollbackable if needed. This commit may contain a descriptive message with a title summarizing the changes to the source code and / or a description detailing the modifications. In some embodiments, the title may have a predefined naming convention. For example, “Auto-fix” + defect identifier from the corresponding defect analysis log + “Defect” + defect identifier from the corresponding defect analysis log + “(” + defect type from the corresponding defect analysis log + “)”. After the commit, the pull request creation subcomponent 630 can push these changes, making them accessible in the repository 110 managed by VC 122, thus enabling other developers to access these changes and create backups of their work. The pull request creation subcomponent 630 can then create a pull request or merge request with a title and description reflecting the purpose of the changes, serving as a formal proposal to integrate the changes into another branch (e.g., the main code branch). Pull requests facilitate code reviews by including references to raw logs and analytics data, enable automated testing, and provide a centralized location to discuss and track the status of proposed changes.

[0063] Figure 7 It is based on one or more aspects of this disclosure Figure 2 An example diagram of the patch verification component 250 is shown. In one embodiment, the patch verification component 250 includes a test execution subcomponent 710 and a test verification subcomponent 720.

[0064] In some embodiments, before subsequently executing the test suite, the test execution subcomponent 710 may update the test suite using additional test cases generated by the patch generation AI model 525 from the verification subcomponent 530, or modifications to existing test cases. Similar to test execution subcomponent 310, test execution subcomponent 710 may enable diagnostic tools and / or source code analysis tools. In embodiments, similar to test execution subcomponent 310, test execution subcomponent 710 may execute the test suite.

[0065] In some embodiments, the test verification subcomponent 720, similar to the defect detection subcomponent 320, evaluates execution results, diagnostic tool outputs, and system responses to identify defects for each test case. If no defects are detected for the test suite, the test verification subcomponent 720 notifies the patch approval component 260 that the fix branch verification was successful.

[0066] In this embodiment, for each defect detected in the corresponding test case, the test verification subcomponent 720 determines whether the corresponding defect matches an existing defect in the test report. If the corresponding defect matches an existing defect in the test report, the test verification subcomponent 720 may include additional contextual information in the contextual information of the existing defect in the test report that matches the corresponding defect. If the corresponding defect does not match an existing defect in the test report, the test verification subcomponent 720, similar to the defect detection subcomponent 320 and the report generation subcomponent 330, may generate a test log and include it in the test report, which contains the contextual information of the corresponding defect (and its corresponding trace log).

[0067] Figure 8 It is based on one or more aspects of this disclosure Figure 2 The diagram shows an example of a patch approval component 260. In one embodiment, the patch approval component 260 includes a notification subcomponent 810 and an input processing subcomponent 820. In some embodiments, in response to the patch approval component 260 receiving successful verification of the fix branch, the notification subcomponent 810 notifies the designated reviewer that the pull request to formally integrate the source code changes in the fix branch into the main branch is ready. The input processing subcomponent 820 may receive input from the designated reviewer, instructing them to decide whether to approve the new version of the code (e.g., a patch), request code modifications, or reject code changes. Typically, the designated reviewer's decision may depend on whether the fix is ​​technically and logically correct and maintainable, and / or whether any other changes are required.

[0068] In one embodiment, in response to receiving approval from a designated reviewer, the input processing subcomponent 820 causes the development engine 120 to merge the source code modifications in the fix branch into the main branch. In some embodiments, the input processing subcomponent 820 may perform (or cause the development engine 120 to perform) a pre-production staging check, update documentation, and / or monitor regression performance in the production environment. The pre-production check validates changes in a pre-production environment mirrored in a production setup to verify system integration and functionality. Documentation updates ensure that technical guidelines, API references, and code comments accurately reflect the implemented changes. Performance monitoring in production tracks system metrics and response times to identify any regressions or unexpected behavior that occur after change deployment.

[0069] Figure 9A flowchart depicts an example method 900 for automatically generating patches according to one or more aspects of this disclosure. This method can be executed by processing logic, which may include hardware (circuit, special-purpose logic, etc.), computer-readable instructions (running on a general-purpose computer system or a special-purpose machine), or a combination of both. In the illustrative example, method 900 can be executed by a processor (e.g., a computing device). Alternatively, part or all of method 900 can be executed by another module or machine. It should be noted that... Figure 9 The blocks shown can be executed simultaneously or in a different order than that shown. In an embodiment, method 900 is... Figure 1 The SLM system 100 and / or shown Figure 2 The defect repair module 135 shown is executed.

[0070] In block 910 of method 900, processing logic identifies a first defect in the source code. To identify the first defect in the source code, the processing logic may compile a copy of the source code to generate a first binary file. The processing logic may execute test cases on the first binary file. In some embodiments, the processing logic detects the first defect during the execution of test cases on the first binary file. In some embodiments, the processing logic detects the first defect during the execution of test cases on the first binary file by determining that the first binary file exhibits one of the following: crash, hang, or anomalous behavior. Depending on the embodiment, the processing logic may perform operations such as obfuscation or unit testing to automatically identify the first defect in the source code.

[0071] In block 920, the processing logic obtains the context information for the first defect. The context information may include diagnostic tool identifiers, test case identifiers, file paths, function names, line numbers, defect types, stack traces, structured data, log extracts, reproducible scenarios, and / or diagnostic data. To obtain the context information for the first defect, the processing logic enables the generation of context information during the execution of the first binary file, as described above.

[0072] In block 930, the processing logic uses one or more artificial intelligence (AI) models to process the first hint to generate a corrected version of the source code. The first hint may be based at least in part on the source code, the first defect, and contextual information about the first defect. The one or more AI models may be one or more large language models (LLMs).

[0073] In block 940, the processing logic determines whether a corrected version of the source code output by one or more AI models resolves the first defect. To determine whether the corrected version of the source code resolves the first defect, the processing logic may compile a copy of the corrected version of the source code to generate a second binary file. The processing logic may enable the generation of context information during the execution of the second binary file. The processing logic may execute test cases on the second binary file. The processing logic may use the same operations described above to determine whether a second defect is detected. In response to determining that a second defect is not detected, the processing logic may determine that the corrected version of the source code has resolved the first defect.

[0074] According to a specific embodiment, in response to determining that a second defect has been detected, the processing logic can use the above-described techniques to identify contextual information related to the second defect. The processing logic can use one or more AI models to process the second prompt to generate an improved, corrected version of the source code. The second prompt may be based in part on the first prompt, the corrected version of the source code, the second defect, and / or the contextual information of the second defect. The processing logic can determine whether the improved, corrected version of the source code resolves the second defect. In response to determining that the improved, corrected version of the source code resolves the second defect, the processing logic can apply the changes implemented in the improved, corrected version to the source code.

[0075] In block 950, in response to determining that the corrected version of the source code resolves the first defect, the processing logic applies the changes implemented in the corrected version of the source code to the source code. To apply the changes implemented in the corrected version of the source code, the processing logic can identify the differences between the corrected version of the source code and the current version of the source code. This difference represents the changes implemented in the corrected version of the source code. The processing logic can then apply the changes implemented in the corrected version of the source code to the source code.

[0076] The systems and methods described herein can be used for a variety of purposes, including but not limited to machine (e.g., robots, vehicles, construction machinery, warehouse vehicles / machines, autonomous machines, semi-autonomous machines and / or other types of machines) control, machine motion, machine driving, synthetic data generation, model training (e.g., using real data, augmented data and / or synthetic data, such as synthetic data generated using simulation platforms or systems, synthetic data generation techniques, including but not limited to the techniques described herein), perception, augmented reality (AR), virtual reality (VR), mixed reality (MR), robotics, security and surveillance (e.g., in smart city implementations), autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or role simulation and / or digital twins, data center processing, conversational AI, optical transport simulation (e.g., ray tracing, path tracing, etc.), distributed or collaborative content creation for 3D assets (e.g., using common scene descriptor (USD) data, such as OpenUSD, and / or other data types), cloud computing, generative artificial intelligence (e.g., using one or more diffusion models, transformer models, etc.) and / or any other suitable application.

[0077] The disclosed embodiments may be included in a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots or robotic platforms, aviation systems, medical systems, marine systems, smart area monitoring systems, systems performing deep learning operations, systems performing simulation operations (e.g., in driving or vehicle simulations, in robot simulations, in smart city or surveillance simulations, etc.), systems performing digital twin operations (e.g., used in conjunction with collaborative content creation platforms or systems, including but not limited to, another platform, system, or service using NVIDIA's OMNIVERSE and / or USD or OpenUSD data types), systems implemented using edge devices, systems containing one or more virtual machines (VMs), and systems performing... Systems that perform synthetic data generation operations (e.g., using one or more neural radiation fields (NERF), Gaussian lattice techniques, diffusion models, transformer models, etc.), systems that are at least partially implemented in a data center, systems that perform conversational AI operations, systems that implement one or more language models (such as one or more large language models (LLM), one or more visual language models (VLM), one or more multimodal language models, etc.), systems that perform optical transmission simulations, systems that perform collaborative content creation for 3D assets (e.g., using generic scene descriptor (USD) data, such as OpenUSD, computer-aided design (CAD) data, 2D and / or 3D graphics or design data and / or other data types), systems that are at least partially implemented using cloud computing resources, and / or other types of systems.

[0078] Example language model In at least some embodiments, language models such as Large Language Models (LLM), Visual Language Models (VLM), Multimodal Language Models (MMLM), and / or other types of generative artificial intelligence (AI) can be implemented. These models may be able to understand, summarize, translate, and / or otherwise generate text (e.g., natural language text, code, etc.), images, videos, computer-aided design (CAD) assets, OMNIVERSE and / or METAVERSE file information (e.g., USD formats such as OpenUSD), and / or the like based on context provided in input prompts or queries. In embodiments, these language models may be considered “large” because they are trained on massive datasets and have architectures with a large number of learnable network parameters (weights and biases)—e.g., millions or billions of parameters. LLM / VLM / MMLM / etc. can be implemented for summarizing textual data, analyzing data (e.g., text, images, videos, etc.), extracting insights from data (e.g., text, images, videos, etc.), and generating new text / images / videos / etc. in a user-specified style, tone, and / or format. In some embodiments, the LLM / VLM / MMLM / etc. disclosed herein may be specifically designed for text processing, while in others, a multimodal LLM may be implemented to accept, understand, and / or generate text and / or other types of content, such as images, audio, 2D and / or 3D data (e.g., USD format) and / or video. For example, a Visual Language Model (VLM) or more specifically a Multimodal Language Model (MMLM) may be implemented to accept images, video, audio, text, 3D designs (e.g., CAD) and / or other input data types and / or generate or output images, video, audio, text, 3D designs and / or other output data types.

[0079] Various types of LLM / VLM / MMLM / etc. architectures can be implemented in various embodiments. For example, different architectures can be implemented using different techniques to understand and generate outputs (e.g., text, audio, video, images, 2D and / or 3D design or asset data, etc.). In some embodiments, LLM / VLM / MMLM / etc. architectures (e.g., recurrent neural networks (RNNs) or long short-term memory networks (LSTMs)) can be used, while in other embodiments, converter architectures (e.g., architectures relying on self-attention and / or cross-attention (e.g., between contextual data and textual data) mechanisms) can be used to understand and recognize relationships between words or tokens and / or contextual data (e.g., other text, video, images, design data, USD, etc.). One or more generative processing pipelines including LLM / VLM / MMLM / etc. may also include one or more diffusion blocks (e.g., noise reduction blocks). The LLM / VLM / MMLM / etc. of this disclosure may include encoder and / or decoder blocks. For example, discriminative or encoder-only models (e.g., BERT (Bidirectional Encoder Representations from Transformers)) can be implemented for tasks involving language understanding (e.g., classification, sentiment analysis, question answering, and named entity recognition). As another example, generative or decoder-only models (e.g., GPT (Generative Pretrained Transformer)) can be implemented for tasks involving language and content generation (e.g., text completion, story generation, and dialogue generation). LLM / VLM / MMLM / etc., including encoder and decoder components (e.g., T5 (Text-to-Text Transformer)), can be implemented to understand and generate content, such as for translation and summarization. These examples are not intended to be limiting and any architecture type (including, but not limited to, those described herein) can be implemented depending on the specific implementation and the task performed using LLM / VLM / MMLM / etc.

[0080] In various embodiments, LLM / VLM / MMLM / etc. can be trained using unsupervised learning, where LLM / VLM / MMLM / etc. learns patterns from a large amount of unlabeled text / audio / video / image / design / USD / etc. data. Due to the extensive training, in these embodiments, the model may not require task-specific or domain-specific training. An LLM / VLM / MMLM / etc. extensively pre-trained on a large amount of unlabeled data can be referred to as a base model and can excel at various tasks, such as question answering, summarizing, filling in missing information, translation, and image / video / design / USD / data generation. Some LLM / VLM / MMLM / etc. can be customized for specific use cases using techniques such as cue tuning, fine-tuning, retrieval augmentation generation (RAG), adding adapters (e.g., custom neural networks and / or neural network layers to tune or adjust cues or labels to bias the language model towards a specific task or domain), and / or using optimization models for specific tasks and / or other fine-tuning or customization techniques for specific domains.

[0081] In some embodiments, the LLM / VLM / MMLM / etc. disclosed herein can be implemented using various model alignment techniques. For example, in some embodiments, guardrails can be implemented to identify incorrect or unwanted inputs (e.g., prompts) and / or outputs of the model. In this process, the system can use guardrails and / or other model alignment techniques to prevent the processing of specific unwanted inputs using LLM / VLM / MMLM / etc., and / or to prevent the output or presentation of information generated by LLM / VLM / MMLM / etc. (e.g., displays, audio outputs, etc.). In some embodiments, one or more additional models (or layers thereof) can be implemented to identify problems with the model's inputs and / or outputs. For example, these "protective" models can be trained to identify "safe" or otherwise okay or desired inputs and / or outputs and / or "unsafe" or otherwise unwanted inputs and / or outputs for a particular application / implementation. Therefore, the LLM / VLM / MMLM / etc. disclosed herein are unlikely to output language / text / audio / video / design data / USD data / etc. that may be offensive, vulgar, inappropriate, insecure, out of scope, and / or unwanted for a particular application / implementation.

[0082] In some embodiments, an LLM / VLM / etc. can be configured or able to access or use one or more plugins, application programming interfaces (APIs), databases, data stores, repositories, etc. For example, for certain tasks or operations where the model is not ideally suited, the model may have instructions for accessing one or more plugins (e.g., third-party plugins) to help process the current input (e.g., as a result of training, and / or based on instructions in a given prompt). In such an example, when at least part of the prompt relates to restaurants or weather, the model can access one or more restaurant or weather plugins (e.g., via one or more APIs) to retrieve relevant information. Another example is that if at least part of the response requires mathematical computation, the model can access one or more mathematical plugins or APIs to help solve the problem, and then the response from the plugins and / or APIs can be used in the model's output. This process can be repeated (e.g., recursively) an arbitrary number of iterations, using any number of plugins and / or APIs, until a response to each query / question / request / process / operation / etc. can be generated in response to the input prompt. Therefore, models can rely not only on their own knowledge gained from training on large datasets, but also on the expertise or optimized properties of one or more external resources (such as APIs, plugins, etc.).

[0083] In some embodiments, multiple language models (e.g., LLM / VLM / MMLM / etc., multiple instances of the same language model, and / or multiple hints provided to the same language model or instances of the same language model) can be implemented, executed, or accessed (e.g., using one or more plugins, user interfaces, APIs, databases, data stores, repositories, etc.) to provide output in response to the same query or in response to separate parts of a query. In at least one embodiment, the same input query and hints (e.g., a set of constraints, condition generators, etc.) can be provided to multiple language models (e.g., language models with different architectures, language models trained on different (e.g., updated) data corpora). In one or more embodiments, the language models can be different versions of the same base model. In one or more embodiments, at least one language model can be instantiated as multiple agents, for example, providing more than one hint to constrain, guide, or otherwise influence the style, content, or character of the provided output. In one or more exemplary non-limiting embodiments, the same language model can be required to provide output corresponding to different roles, perspectives, characters, or different knowledge bases, as defined by the provided hints.

[0084] In any such embodiment, the outputs of two or more (e.g., each) language models, two or more versions of at least one language model, two or more instantiated proxies of at least one language model, and / or provided to two or more prompts for at least one language model can be further processed, such as aggregated, compared, or filtered, or used to determine (and provide) a consensus response. In one or more embodiments, the output from one language model (or version, instance, or proxy) can be provided as input to another language model for further processing and / or validation. In one or more embodiments, the language model can be required to generate or otherwise obtain output about the input source material, wherein the output is associated with the input source material. Such association can include, for example, generating captions or text portions embedded (e.g., as metadata) within the input source text or image. In one or more embodiments, the output of the language model can be used to determine the validity of the input source material for further processing or inclusion in a dataset. For example, the language model can be used to evaluate the presence (or absence) of a target word in a text portion or the presence (or absence) of an object in an image, wherein the text or image is annotated to indicate such presence (or absence). Alternatively, the determination from the language model can be used to determine whether the source material should be included in a curatorial dataset, for example, but not limited to this.

[0085] Figure 10A This is a block diagram of an example generative language model system 1000 according to one or more aspects of this disclosure. Figure 10A In the example shown, the generative language model system 1000 includes a retrieval augmentation (RAG) component 1092, an input processor 1005, a tokenizer 1010, an embedding component 1020, a plugin / API 1095, and a generative language model (LM) 1030 (which may include LLM, VLM, multimodal LM, etc.).

[0086] At a high level, the input processor 1005 can receive input 1001, which includes text and / or other types of input data (e.g., audio data, video data, image data, sensor data (e.g., LiDAR, RADAR, ultrasound, etc.), 3D design data, CAD data, generic scene descriptor (USD) data (e.g., OpenUSD, etc.), depending on the architecture of the generative LM 1030 (e.g., LLM / VLM / MMLM, etc.). In some embodiments, input 1001 includes plain text in the form of one or more sentences, paragraphs, and / or documents. Additionally or alternatively, input 1001 may include numerical sequences, pre-computed embeddings (e.g., word or sentence embeddings), and / or structured data (e.g., tabular format, JSON, or XML). In the generative LM In some implementations of 1030 capable of handling multimodal input, input 1001 can combine text (or text that may be omitted) with image data, audio data, video data, design data, USD data, and / or other types of input data (e.g., but not limited to the data described herein). Taking raw input text as an example, input processor 1005 can prepare the raw input text in various ways. For example, input processor 1005 can perform various types of text filtering to remove noise from the relevant text content (e.g., special characters, punctuation marks, HTML tags, stop words, portions of images, portions of audio, etc.). In examples involving stop words (common words that often have little semantic meaning), input processor 1005 can remove stop words to reduce noise and allow the generative LM 1030 to focus on more meaningful content. Input processor 1005 can apply text normalization, for example, by converting all characters to lowercase, removing accent marks, and / or handling special cases (such as abbreviations or abbreviations) to ensure consistency. These are just a few examples; other types of input processing can be applied.

[0087] In some embodiments, RAG component 1092 (which may include one or more RAG models, and / or may be performed using generative LM 1030 itself) may be used to retrieve additional information to be used as part of input 1001 or a prompt. RAGs can be used to enhance input to LLM / VLM / MMLM / etc. with external knowledge to make the answer to a specific question or query or request more relevant, for example, where specific knowledge is required. RAG component 1092 may obtain this additional information (e.g., basic information such as basic text / images / videos / audio / USD / CAD / etc.) from one or more external sources, and then feed it along with the prompt to LLM / VLM / MMLM / etc. to improve the accuracy of the model's response or output.

[0088] For example, in some embodiments, in addition to the data retrieved using RAG component 1092, input 1001 may also be generated using query or model inputs (e.g., questions, requests, etc.). In some embodiments, input processor 1005 may analyze input 1001 and communicate with RAG component 1092 (or in embodiments, RAG component 1092 may be part of input processor 1005) to identify relevant text and / or other data to provide to generative LM 1030 as additional context or information sources, typically from which responses, answers, or outputs 1090 are identified. For example, when the input indicates that a user is interested in the required tire pressure for a particular brand and model of vehicle, RAG component 1092 may use a RAG model, for example, to perform a vector search in the embedding space to retrieve tire pressure information or its corresponding text from a digital (embedded) version of the owner's manual for that particular vehicle brand and model. Similarly, when a user revisits the chatbot related to a specific product sale or service, the RAG component 1092 can retrieve previously stored conversation history (or at least its summary) and provide the previous conversation history, along with the current inquiry / request, as part of the generative LM 1030 as input 1001.

[0089] RAG component 1092 can use various RAG techniques. For example, naïve RAG can be used, where documents are indexed, chunked, and applied to an embedding model to generate embeddings corresponding to chunks. User queries can also be applied to this embedding model and / or another embedding model of RAG component 1092, and the embeddings of chunks can be compared with the embeddings of the query to identify the most similar / relevant embeddings to the query. These most similar / relevant embeddings can be provided to generative LM 1030 to generate output.

[0090] In some embodiments, more advanced RAG techniques can be used. For example, chunks can undergo pre-retrieval processes (e.g., routing, rewriting, metadata analysis, expansion, etc.) before being passed to the embedding model. Furthermore, post-retrieval processes (e.g., re-ranking, hint compression, etc.) can be performed on the output of the embedding model before generating the final embedding, which is then used for comparison with the input query.

[0091] As a further example, modular RAG techniques can be used, such as those similar to Naive RAG and / or Advanced RAG, but also including features such as hybrid search, recursive retrieval and query engines, StepBack methods, subqueries and hypothetical document embeddings.

[0092] As another example, Graph RAG can use a knowledge graph as a source of context or factual information. Graph RAG can be implemented using a graph database as a source of contextual information sent to LLM / VLM / MMLM / etc. Instead of providing the model with data chunks extracted from larger documents (which can lead to a lack of context, factual accuracy, linguistic accuracy, etc.) (or anything other than providing the model with data chunks extracted from larger documents), Graph RAG can also provide the model with structured entity information by combining structured entity text descriptions with their many attributes and relationships, thus giving the model deeper insights. In implementing Graph RAG, the systems and methods described herein use graphs as content stores and extract relevant document chunks, requiring LLM / VLM / MMLM / etc. to use them to answer questions. In such embodiments, the knowledge graph may contain relevant textual content and metadata about the knowledge graph, or it may be integrated with a vector database. In some embodiments, Graph RAG can use the graph as a subject matter expert, where descriptions of concepts and entities relevant to the query / hint can be extracted and passed to the model as semantic context. These descriptions may include relationships between concepts. In other examples, the graph can be used as a database where a portion of a query / hint can be mapped to a graph query, the graph query can be executed, and LLM / VLM / MMLM / etc. can summarize the results. In such examples, the graph can store relevant factual information and can be used for queries (natural language queries) and entity links to graph query tools (NL to graph query tools). In some embodiments, the graph RAG (e.g., using a graph database) can be combined with standard (e.g., vector database) RAGs and / or other RAG types to benefit from a variety of approaches.

[0093] In any embodiment, the RAG component 1092 may implement plugins, APIs, user interfaces, and / or other functions to perform RAG. For example, LLM / VLM / MMLM / etc. may use graph RAG plugins to run queries on knowledge graphs to extract relevant information to feed into the model, and may use standard or vector RAG plugins to run queries on vector databases. For example, the graph database may interact with the plugin's REST interface, thus decoupling the graph database from the vector database and / or the embedded model.

[0094] The tokenizer 1010 can segment (e.g., processed) text data into smaller units (tags) for subsequent analysis and processing. Depending on the implementation, the tags can represent individual words, sub-words, characters, audio / video / images, etc. Word-based tokenization divides the text into individual words, treating each word as a separate tag. Sub-word tokenization breaks words down into smaller meaningful units (e.g., prefixes, suffixes, stems), enabling the generative LM 1030 to understand morphological changes and process words outside the vocabulary more effectively. Character-based tokenization represents each character as a separate tag, enabling the generative LM 1030 to process text at a fine-grained level. The choice of tokenization strategy can depend on factors such as the language being processed, the task at hand, and / or the characteristics of the training dataset. Therefore, the tokenizer 1010 can transform (e.g., processed) text into a structured format according to the tokenization scheme implemented in a particular embodiment.

[0095] Embedding component 1020 can use any known embedding technique to transform discrete tokens into semantically meaningful (e.g., dense, continuous vector) representations. For example, embedding component 1020 can use pre-trained word embeddings (e.g., Word2Vec, GloVe, or FastText), one-hot encoding, Term Frequency-Inverse Document Frequency (TF-IDF) encoding, one or more embedding layers of a neural network, and / or others.

[0096] In some implementations where input 1001 includes image data / video data, etc., input processor 1005 may resize the data to a standard size compatible with the format of the corresponding input channel and / or normalize pixel values ​​to a common range (e.g., 0 to 1) to ensure consistent representation, and embedding component 1020 may encode the image data using any known technique (e.g., using one or more convolutional neural networks (CNNs) to extract visual features). In some implementations where input 1001 includes audio data, input processor 1005 may resample the audio file to a consistent sampling rate for uniform processing, and embedding component 1020 may use any known technique to extract and encode audio features, such as in the form of a spectrogram (e.g., a Mel spectrogram). In some implementations where input 1001 includes video data, input processor 1005 may extract frames or apply resizing to extracted frames, and embedding component 1020 may extract features such as optical flow embedding or video embedding and / or encode temporal information or frame sequences. In some implementations where input 1001 includes multimodal data, the embedded component 1020 can use techniques such as early fusion (concatenation), late fusion (sequential processing), and attention-based fusion (e.g., self-attention, cross-attention) to fuse representations of different types of data (e.g., text, images, audio, USD, video, design, etc.).

[0097] Other components of the generative LM 1030 and / or generative LM system 1000 may use different types of neural network architectures depending on the implementation scheme. For example, a transducer-based architecture (such as the one used in models like GPT) may be implemented, and it may include a self-attention mechanism that weights the importance of different words or tokens in the input sequence and / or a feedforward network that processes the output of the self-attention layer, applying a nonlinear transformation to the input representation and extracting higher-level features. Some non-limiting example architectures include transducers (e.g., encoder-decoder, decoder-only, multimodal), RNNs, LSTMs, fusion models, diffusion models, cross-modal embedding models that learn a joint embedding space, graph neural networks (GNNs), hybrid architectures that combine different types of adversarial networks (such as generative adversarial networks or GANs or adversarial autoencoders (AAEs) for joint distribution learning), etc. Therefore, depending on the implementation scheme and architecture, the embedded component 1020 can apply the encoded representation of the input 1001 to the generative LM 1030, and the generative LM 1030 can process the encoded representation of the input 1001 to generate an output 1090, which may include response text and / or other types of data.

[0098] As described herein, in some embodiments, the generative LM 1030 may be configured to access or use (or be able to access or use) plugins / APIs 1095 (which may include one or more plugins, application programming interfaces (APIs), databases, data stores, repositories, etc.). For example, for certain tasks or operations where the generative LM 1030 is not ideally suited, the model may have instructions (e.g., as a result of training, and / or based on instructions in a given prompt, such as instructions retrieved using RAG component 1092) to access one or more plugins / APIs 1095 (e.g., third-party plugins) to help process the current input. In such an example, when at least part of the prompt is related to a restaurant or weather, the model may access one or more restaurant or weather plugins (e.g., via one or more APIs), sending at least part of the prompt related to a particular plugin / API 1095 to the plugin / API 1095, which may process the information and return an answer to the generative LM 1030, which may use the response to generate output 1090. This process can be repeated (e.g., recursively) an arbitrary number of iterations and repeated using any number of plugins / APIs 1095 until an output 1090 that resolves each query / question / request / process / action / etc. from input 1001 is generated. Therefore, the model can rely not only on its own knowledge acquired from training on a large dataset and / or from data retrieved using the RAG component 1092, but also on the expertise or optimized properties of one or more external resources (e.g., plugins / APIs 1095).

[0099] Figure 10B This is a block diagram of an example implementation scheme, where the generative LM 1030 includes a converter encoder-decoder. For example, suppose the input text (e.g., "Who discovered gravity") is tokenized (e.g., by...) Figure 10A The tokenizer 1010 is used for tokens such as words, and each token is encoded (e.g., by...). Figure 10A The embedding component 1020 is a corresponding embedding (e.g., of size 512). Since these token embeddings do not typically represent the position of the tokens in the input sequence, positional encoding can be added to each token embedding using any known technique to encode the order relation and context of the tokens in the input sequence. Thus, the (e.g., the resulting) embeddings can be applied to one or more encoders 1035 of the generative LM1030.

[0100] In the example implementation, encoder 1035 forms an encoder stack, where each encoder includes a self-attention layer and a feedforward network. In the example converter architecture, each token (e.g., a word) flows through a separate path. Therefore, each encoder can accept a sequence of vectors, pass each vector through the self-attention layer, then through the feedforward network, and then up to the next encoder in the stack. Any known self-attention technique can be used. For example, to compute a self-attention score for each token (word), a query vector, a key vector, and a value vector can be created for each token. The self-attention score for a token pair can be computed by taking the dot product of the query vector and the corresponding key vector, normalizing the resulting score, multiplying by the corresponding value vector, and summing the weighted value vectors. The encoder can apply multi-head attention, where the attention mechanism is applied multiple times in parallel with different learned weight matrices. Any number of encoders can be cascaded to generate context vectors that encode the input. Attention projection layer 1040 can transform the context vectors into attention vectors (keys and values) for decoder 1045.

[0101] In the example implementation, decoder 1045 forms a decoder stack, where each decoder includes a self-attention layer, an encoder-decoder self-attention layer that uses attention vectors (keys and values) from the encoder to focus on relevant parts of the input sequence, and a feedforward network. Similar to encoder 1035, in the example converter architecture, each token (e.g., a word) flows through a separate path in decoder 1045. During the first pass, decoder 1045, classifier 1050, and generation mechanism 1055 can generate a first token, and generation mechanism 1055 can apply the generated token as input during a second pass. This process can be repeated cyclically, generating tokens (e.g., words) and adding them to the output of the previous pass, and in subsequent passes applying token embeddings of positionally encoded composite sequences as input to decoder 1045, generating one token at a time (called autoregression) until a symbol or token representing the end of the response is predicted. In each decoder, the self-attention layer is typically restricted to focusing only on preceding positions in the output sequence by applying a masking technique (e.g., setting future positions to negative infinity) before the softmax operation. In the example implementation, the encoder-decoder attention layer operates similarly to the (e.g., multi-head) self-attention operation in encoder 1035, except that it creates its query from the layer below it and obtains keys and values ​​(e.g., matrices) from the output of encoder 1035.

[0102] Therefore, decoder 1045 can output some decoded (e.g., vector) representation of the input applied during a particular pass. Classifier 1050 can include a multi-class classifier comprising one or more neural network layers and a softmax operation that transforms logit probabilities into probabilities, the neural network layers projecting the decoded (e.g., vector) representation onto corresponding dimensions (e.g., one dimension for each supported word or token in the output vocabulary). Thus, generation mechanism 1055 can select or sample words or tokens based on corresponding predicted probabilities (e.g., selecting the word with the highest predicted probability) and append it to the output of the previous pass, thereby generating each word or token sequentially. Generation mechanism 1055 can repeat this process, triggering successive decoder inputs and corresponding predictions until a symbol or token representing the end of the response is selected or sampled, at which point generation mechanism 1055 can output the generated response.

[0103] Figure 10C This is a block diagram of an example implementation where the generative LM 1030 includes a decoder-only converter architecture. For example, Figure 10C The decoder 1060 can be used with Figure 10B The decoder 1045 operates similarly, except... Figure 10C Each decoder 1060 omits the encoder-decoder self-attention layer (because there is no encoder in this implementation). Therefore, the decoders 1060 can form a decoder stack, where each decoder includes a self-attention layer and a feedforward network. Furthermore, instead of encoding the input sequence, a symbol or tag indicating the end of the input sequence (or the beginning of the output sequence) can be appended to the input sequence, and the resulting sequence (e.g., a corresponding embedding with positional encoding) can be applied to the decoder 1060. Figure 10B Similar to decoder 1045, each tag (e.g., a word) can flow through a separate path in decoder 1060, and decoder 1060, classifier 1065, and generation mechanism 1070 can use autoregression to generate one tag at a time sequentially until a symbol or tag indicating the end of the response is predicted. Classifier 1065 and generation mechanism 1070 can be... Figure 10B The classifier 1050 and generation mechanism 1055 operate similarly, wherein the generation mechanism 1070 selects or samples each consecutive output label based on the corresponding predicted probability and appends it to the output of the previous iteration, generating each label sequentially until a symbol or label representing the end of the response is selected or sampled. The architectures described herein, and others, are merely examples, and other suitable architectures may be implemented within the scope of this disclosure.

[0104] Example computing device Figure 11This is a block diagram of an example computing device 1100 suitable for implementing some embodiments of the present disclosure. The computing device 1100 may include an interconnect system 1102 directly or indirectly coupled to the following devices: memory 1104, one or more central processing units (CPUs) 1106, one or more graphics processing units (GPUs) 1108, a communication interface 1110, input / output (I / O) ports 1112, input / output components 1114, a power supply 1116, one or more presentation components 1118 (e.g., one or more displays), and one or more logic units 1120. In at least one embodiment, one or more computing devices 1100 may include one or more virtual machines (VMs), and / or any component thereof may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 1108 may include one or more vGPUs, one or more CPUs 1106 may include one or more vCPUs, and / or one or more logic units 1120 may include one or more virtual logic units. Thus, one or more computing devices 1100 may include discrete components (e.g., a full GPU dedicated to computing device 1100), virtual components (e.g., a portion of the GPU dedicated to computing device 1100), or combinations thereof.

[0105] although Figure 11 The various blocks are shown as connected via interconnect system 1102 using lines, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, presentation component 1118 (such as a display device) may be considered I / O component 1114 (e.g., if the display is a touchscreen). As another example, CPU 1106 and / or GPU 1108 may include memory (e.g., memory 1104 may represent a storage device other than the memory of GPU 1108, CPU 1106, and / or other components). Accordingly, Figure 11 The computing devices described are for illustrative purposes only. No distinction is made between such categories as “workstation,” “server,” “laptop computer,” “desktop computer,” “tablet computer,” “client device,” “mobile device,” “handheld device,” “game console,” “electronic control unit (ECU),” “virtual reality system,” and / or other device or system types, as all are considered within the scope of this category. Figure 11 Within the scope of computing devices.

[0106] Interconnect system 1102 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 1102 may include one or more bus or link types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Fast Peripheral Component Interconnect (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. As an example, CPU 1106 may be directly connected to memory 1104. Further, CPU 1106 may be directly connected to GPU 1108. In cases where there is a direct or point-to-point connection between components, interconnect system 1102 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required to be included in computing device 1100.

[0107] The memory 1104 may include any computer-readable medium from a variety of computer-readable media. A computer-readable medium may be any available medium accessible by the computing device 1100. Computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. By way of example and not limitation, computer-readable media may include computer storage media and communication media.

[0108] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 1104 may store computer-readable instructions (e.g., representing one or more programs and / or one or more program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Universal Disc (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by computing device 1100. As used herein, computer storage media does not include the signal itself.

[0109] Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals such as carrier waves or other transmission mechanisms, and includes any information transmission medium. The term "modulated data signal" can refer to a signal whose one or more characteristics are set or altered in a manner that encodes information in the signal. By way of example and not limitation, computer storage media can include wired media (such as wired networks or direct wired connections) and wireless media (such as acoustic, RF, infrared, and other wireless media). Any combination of the above should also be included within the scope of computer-readable media.

[0110] CPU 1106 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1100 to perform one or more of the methods and / or processes described herein. Each CPU 1106 may contain one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of handling numerous software threads simultaneously. CPU 1106 may contain any type of processor and may contain different types of processors depending on the type of computing device 1100 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 1200, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors (such as math coprocessors), computing device 1100 may also include one or more CPUs 1106.

[0111] In addition to or in lieu of one or more CPUs 1106, one or more GPUs 1108 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1100 to perform one or more of the methods and / or processes described herein. One or more GPUs 1108 may be integrated GPUs (e.g., with one or more CPUs 1106) and / or one or more GPUs 1108 may be discrete GPUs. In embodiments, one or more GPUs 1108 may be coprocessors of one or more CPUs 1106. GPUs 1108 may be used by computing device 1100 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, GPUs 1108 may be used for general-purpose computing on a GPU (GPGPU). GPUs 1108 may include hundreds or thousands of cores capable of handling hundreds or thousands of software threads simultaneously. GPUs 1108 may produce pixel data of an output image in response to rendering commands (e.g., rendering commands received from CPUs 1106 via a host interface). GPU 1108 may include graphics memory (e.g., display memory) for storing pixel data or any other suitable data (e.g., GPGPU data). Display memory may be included as part of memory 1104. GPU 1108 may include two or more GPUs operating in parallel (e.g., via links). The links may directly connect the GPUs (e.g., using NVLINK) or may connect the GPUs via a switch (e.g., using NVSwitch). When combined, each GPU 1108 may produce pixel data or GPGPU data for different portions of the output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU may include its own memory or may share memory with other GPUs.

[0112] In addition to or in lieu of CPU 1106 and / or GPU 1108, logic unit 1120 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1100 to perform one or more of the methods and / or processes described herein. In embodiments, one or more CPUs 1106, one or more GPUs 1108, and / or one or more logic units 1120 may execute any combination of methods, processes, and / or portions thereof discretely or jointly. One or more logic units 1120 may be a portion of one or more CPUs 1106 and / or GPUs 1108 and / or integrated into one or more CPUs 1106 and / or GPUs 1108, and / or one or more logic units 1120 may be discrete components or otherwise external to CPUs 1106 and / or GPUs 1108. In an embodiment, one or more of the logic units 1120 may be coprocessors of one or more of the CPU 1106 and / or one or more of the GPU 1108.

[0113] Examples of logic unit 1120 include one or more processing cores and / or components thereof, such as a data processing unit (DPU), tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree lateral unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), programmable vision accelerator (PVA)—which may include one or more direct memory access (DMA) systems, one or more vision or vector processing units (VPUs), and one or more pixel processing engines (PPEs). —For example, including 2D arrays of processing elements (each processing element communicates with one or more other processing elements in the array in the north, south, east, and west directions), one or more decoupled accelerators or units (e.g., decoupled lookup table (DLUT) accelerators or units), vision processing units (VPUs), optical flow accelerators (OFAs), field-programmable gate arrays (FPGAs), neuromorphic chips, quantum processing units (QPUs), associative processing units (APUs), arithmetic logic units (ALUs), application-specific integrated circuits (ASICs), floating-point units (FPUs), input / output (I / O) elements, peripheral component interconnects (PCIs) or fast peripheral component interconnects (PCIe) elements, etc.

[0114] Communication interface 1110 may include one or more receivers, transmitters, and / or transceivers that allow computing device 1100 to communicate with other computing devices via electronic communication networks (including wired and / or wireless communications). Communication interface 1110 may include components and functions that allow communication over any of a plurality of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication over Ethernet or wirelessband), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, logic unit 1120 and / or communication interface 1110 may include one or more data processing units (DPUs) to directly transmit data received via a network and / or via interconnect system 1102 to one or more GPUs 1108 (e.g., memory of one or more GPUs 1108).

[0115] I / O port 1112 enables computing device 1100 to be logically coupled to other devices including I / O component 1114, one or more presentation components 1118, and / or other components, some of which may be built into (e.g., integrated into) computing device 1100. Illustrative I / O component 1114 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, scanners, printers, wireless devices, etc. I / O component 1114 can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological input generated by the user. In some cases, input may be transmitted to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, pen recognition, facial recognition, biometric recognition, on-screen and near-screen gesture recognition, air gestures, head and eye tracking, and touch recognition (as described in more detail below) associated with the display of computing device 1200. Computing device 1100 may include depth cameras for gesture detection and recognition, such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof. Additionally, computing device 1100 may include an accelerometer or gyroscope that allows motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, computing device 1100 may use the output of the accelerometer or gyroscope to render immersive augmented reality or virtual reality.

[0116] Power supply 1116 may include a hardwired power supply, a battery power supply, or a combination thereof. Power supply 1116 may provide power to computing device 1100 to allow the components of computing device 1100 to operate.

[0117] The presentation component 1118 may include a display (e.g., a monitor, touchscreen, television screen, head-up display (HUD), other display types, or combinations thereof), speakers, and / or other presentation components. The presentation component 1118 may receive data from other components (e.g., GPU 1108, CPU 1106, etc.) and output the data (e.g., as images, videos, sounds, etc.).

[0118] Example Data Center Figure 12 An example data center 1200 that may be used in at least one embodiment of this disclosure is shown. The data center 1200 may include a data center infrastructure layer 1210, a framework layer 1220, a software layer 1230, and / or an application layer 1240.

[0119] like Figure 12 As shown, the data center infrastructure layer 1210 may include a resource coordinator 1212, grouped computing resources 1214, and node computing resources (“nodes CRs”) 1216(1)-1216(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CRs 1216(1)-1216(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (FPGAs), graphics processing units or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules and / or cooling modules, etc. In some embodiments, one or more node CRs from nodes CRs 1216(1)-1216(N) may correspond to servers having one or more of the aforementioned computing resources. In addition, in some embodiments, nodes CRs 1216(1)-1216(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more of nodes CRs 1216(1)-1216(N) may correspond to virtual machines (VMs).

[0120] In at least one embodiment, the grouped computing resources 1214 may include individual groups of node CRs 1216 housed within one or more racks (not shown), or a plurality of racks housed within a data center in different geographical locations (also not shown). Individual groups of node CRs 1216 within the grouped computing resources 1214 may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, a plurality of node CRs 1216, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0121] Resource coordinator 1212 may be configured or otherwise control one or more nodes CR 1216(1)-1216(N) and / or grouped computing resources 1214. In at least one embodiment, resource coordinator 1212 may include a Software Design Infrastructure (“SDI”) management entity for data center 1200. Resource coordinator 1212 may include hardware, software, or some combination thereof.

[0122] In at least one embodiment, such as Figure 12As shown, framework layer 1220 may include job scheduler 1233, configuration manager 1234, resource manager 1236, and / or distributed file system 1238. Framework layer 1220 may include a framework of software 1232 supporting software layer 1230 and / or one or more applications 1242 supporting application layer 1240. Software 1232 or application 1242 may respectively contain web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 1220 may be, but is not limited to, a free and open-source software web application framework (such as Apache Spark™ (hereinafter referred to as "Spark")) that can leverage distributed file system 1238 for large-scale data processing (e.g., "big data"). In at least one embodiment, job scheduler 1233 may include Spark drivers to facilitate the scheduling of workloads supported by different layers of data center 1200. Configuration manager 1234 may be able to configure different layers, such as software layer 1230 and framework layer 1220 (which includes Spark and distributed file system 1238 for supporting large-scale data processing). Resource manager 1236 may be able to manage computing resources mapped to or allocated to clusters of distributed file system 1238 and job scheduler 1233, or allocated to support clusters of distributed file system 1238 and job scheduler 1233. In at least one embodiment, clustered or grouped computing resources may include grouped computing resources 1214 in data center infrastructure layer 1210. Resource manager 1236 may coordinate with resource coordinator 1212 to manage these mapped or allocated computing resources.

[0123] In at least one embodiment, the software 1232 included in software layer 1230 may include software at least partially used by nodes CR 1216(1)-1216(N), grouped computing resources 1214, and / or the distributed file system 1238 of framework layer 1220. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0124] In at least one embodiment, the application 1242 included in the application layer 1240 may include one or more types of applications used at least in part by nodes CRs 1216(1)-1216(N), grouped computing resources 1214, and / or the distributed file system 1238 of the framework layer 1220. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in combination with one or more embodiments.

[0125] In at least one embodiment, any of the configuration manager 1234, resource manager 1236, and resource coordinator 1212 can implement any number and type of self-modification actions based on any amount and type of data obtained in any technically feasible manner. Self-modification actions can free the data center operator of data center 1200 from making potentially poor configuration decisions and may prevent underutilization and / or poor performance of the data center.

[0126] According to one or more embodiments described herein, data center 1200 may include tools, services, software, or other resources to train one or more machine learning models or to use one or more machine learning models to predict or infer information. For example, one or more machine learning models may be trained by using the software and / or computing resources described above with respect to data center 1200 to compute weight parameters according to a neural network architecture. In at least one embodiment, a trained or deployed machine learning model corresponding to one or more neural networks may be used to infer or predict information using the resources described above with respect to data center 1200 by using weight parameters computed through one or more training techniques, such as, but not limited to, those described herein.

[0127] In at least one embodiment, the data center 1200 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the aforementioned resources. Furthermore, one or more of the software and / or hardware resources described above may be configured to allow a user to train or perform services that infer information, such as image recognition, speech recognition, or other artificial intelligence services.

[0128] Example network environment A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 11 This is implemented on one or more instances of computing devices 1100—for example, each device may include similar components, features, and / or functions of one or more computing devices 1100. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of a data center 1200, examples of which are described in this document. Figure 12 To describe in more detail.

[0129] Components of a network environment can communicate with each other via a network, which can be wired, wireless, or both. A network can include multiple networks or one of multiple networks. For example, a network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (such as the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. In cases where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0130] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein for the server can be implemented on any number of client devices.

[0131] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. The cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework supporting software at the software layer and / or application at the application layer. The software or application may respectively include network-based service software or applications. In embodiments, one or more client devices may use network-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs). The framework layer may be, but is not limited to, a free and open-source software network application framework that can use a distributed file system for large-scale data processing (e.g., "big data").

[0132] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, global, etc.). If the connection to a user (e.g., a client device) is relatively close to an edge server, the core server can assign at least a portion of the functionality to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0133] One or more client devices may include the information described in this article. Figure 11 At least some of the components, features, and functions of one or more example computing devices 1100 described. By way of example and not limitation, the client device may be implemented as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming device or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these depicted devices, or any other suitable device.

[0134] This disclosure can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions, such as program modules, that are executed by a computer or other machine, such as a personal data assistant or other handheld device. Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, and more specialized computing devices. This disclosure can also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked via a communication network.

[0135] As used herein, the phrase "and / or" relating to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, "element A, element B, and / or element C" can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Furthermore, "at least one of element A or element B" can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Further, "at least one of element A and element B" can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0136] This document describes in detail the subject matter of this disclosure to meet statutory requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the discloser has envisioned that the claimed subject matter may be embodied in other ways to include steps different from or similar combinations of steps described herein in conjunction with other current or future techniques. Moreover, although the terms "step" and / or "block" may be used herein to imply different elements of the method employed, these terms should not be construed as suggesting any particular order among or between the various steps disclosed herein, unless the order of the steps is explicitly described.

Claims

1. One or more processors, including a processing circuitry system, said processing circuitry system being used for: Use processing equipment to identify defects in the source code; Obtain the contextual information of the defect; The prompts are processed using one or more artificial intelligence (AI) models to generate a corrected version of the source code, the prompts being generated based at least on the source code, the defect, and the contextual information of the defect; Determine whether the corrected version of the source code output using the one or more AI models resolves the defect; as well as In response to the determination that the corrected version of the source code resolves the defect, the changes implemented in the corrected version of the source code are applied to the source code.

2. The processors according to claim 1, wherein, Identifying the defects in the source code includes: Compile a copy of the source code to generate a binary file; Execute test cases on the binary file; and The defect is detected during the execution of the test cases on the binary file.

3. The processor according to claim 2, wherein, Detecting the defect during the execution of the binary file includes: Determine if the binary file exhibits one of the following conditions: crash, hang, or abnormal behavior.

4. The processors according to claim 1, wherein, The context information for obtaining the defect includes: The generation of context information is enabled during the execution of the binary file, wherein the binary file is generated based at least on a copy of the compiled source code; Execute test cases on the binary file; The defect was detected during the execution of the binary file; and Identify the contextual information associated with the defect.

5. The processors according to claim 1, wherein, Determining whether the corrected version of the source code resolves the defect includes: Compile a copy of the corrected version of the source code to generate a second binary file; Enable the generation of context information during the execution of the second binary file; Execute test cases on the second binary file; Determine whether a second defect has been detected; and In response to determining that the second defect was not detected, it is determined that the corrected version of the source code resolves the defect.

6. The processors according to claim 5, wherein, The one or more processors are also configured to perform the following operations in response to determining that a second defect has been detected: Identify contextual information associated with the second defect; The second prompt is processed using one or more AI models to generate an improved, corrected version of the source code, the second prompt being generated based at least on the prompt, the corrected version of the source code, the second defect, and contextual information regarding the second defect; Determine whether the improved, revised version of the source code resolves the second defect; as well as In response to the determination that the improved and corrected version of the source code resolves the second defect, the changes implemented in the improved and corrected version of the source code are applied to the source code.

7. The processors according to claim 1, wherein, Applying the changes implemented in the corrected version of the source code to the source code includes: Identify the differences between the corrected version of the source code and the source code, wherein the differences represent the changes implemented in the corrected version of the source code; and The changes implemented in the corrected version of the source code are applied to the source code.

8. The processors according to claim 1, wherein, The one or more AI models include one or more large language models (LLM), one or more visual language models (VLM), or one or more multimodal language models (MMLM).

9. The processors according to claim 1, wherein, The one or more processors are also used for: Perform at least one of fuzz testing or unit testing to automatically identify the defects in the source code.

10. The processors according to claim 1, wherein, The context information includes at least one of the following: diagnostic tool identifier, test case identifier, file path, function name, line number, defect type, stack trace, structured data, log extract, reproducible scenario, or diagnostic data.

11. The processors according to claim 1, wherein, The one or more processors are included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system for performing one or more simulation operations; A system for performing one or more digital twin operations; A system for performing one or more optical transmission simulations; A system for performing collaborative content creation for 3D assets; A system for performing one or more wireless cellular transmissions using a wireless cellular network; A system that provides one or more cloud gaming applications; A system for performing one or more deep learning operations; Systems implemented using edge devices; Systems implemented using robots; A system for performing one or more generative AI operations; A system for performing one or more conversational AI operations; A system for performing operations using one or more large language model LLMs; A system for performing operations using one or more visual language models (VLMs); A system for performing operations using one or more multimodal language models (MMLMs); A system for performing operations using one or more visual-language-action (VLA) models; A system for performing one or more conversational AI operations; A system for performing one or more synthetic data generation operations; A system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; A system that uses or deploys one or more inference microservices; A system that combines one or more machine learning models with operating system-level virtualization packages (e.g., containers) and deploys them into services or microservices; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

12. A system comprising: One or more processors are used for: Identify defects in the source code; Obtain the contextual information of the defect; The prompts are processed using one or more artificial intelligence (AI) models to generate a corrected version of the source code, the prompts being generated based at least on the source code, the defect, and the contextual information of the defect; Determine whether the corrected version of the source code output by the one or more AI models resolves the defect; as well as In response to determining that the corrected version of the source code resolves the defect, the changes implemented in the corrected version of the source code are applied to the source code.

13. The system according to claim 12, wherein, In order to identify the aforementioned defects in the source code, the one or more processors are used to: Compile a copy of the source code to generate a binary file; Execute test cases on the binary file; as well as The defect is detected during the execution of the test cases on the binary file.

14. The system according to claim 13, wherein, In order to detect the defect during the execution of the binary file, the one or more processors are configured to: Determine if the binary file exhibits one of the following conditions: crash, hang, or abnormal behavior.

15. The system according to claim 12, wherein, In order to obtain the context information of the defect, the one or more processors are configured to: The generation of context information is enabled during the execution of the binary file, wherein the binary file is generated by compiling a copy of the source code; Execute test cases on the binary file; The defect is detected during the execution of the binary file; as well as Identify the contextual information associated with the defect.

16. The system according to claim 12, wherein, To determine whether the corrected version of the source code resolves the defect, the one or more processors are configured to: Compile a copy of the corrected version of the source code to generate a second binary file; Enable the generation of context information during the execution of the second binary file; Execute test cases on the second binary file; Determine whether a second defect has been detected; as well as In response to determining that the second defect was not detected, it is determined that the corrected version of the source code resolves the defect.

17. The system according to claim 16, wherein, The one or more processors are also used for: Identify contextual information associated with the second defect; The second prompt is processed using one or more AI models to generate an improved, corrected version of the source code, the second prompt being generated based at least on the prompt, the corrected version of the source code, the second defect, and the contextual information regarding the second defect; Determine whether the improved, revised version of the source code resolves the second defect; as well as In response to the determination that the improved and corrected version of the source code resolves the second defect, the changes implemented in the improved and corrected version of the source code are applied to the source code.

18. The system according to claim 12, wherein, In order to apply the changes implemented in the corrected version of the source code to the source code, the one or more processors are configured to: Identify the differences between the corrected version of the source code and the source code, wherein the differences represent the changes implemented in the corrected version of the source code; as well as The changes implemented in the corrected version of the source code are applied to the source code.

19. The system according to claim 12, wherein, The one or more AI models include one or more large language models (LLM), one or more visual language models (VLM), or one or more multimodal language models (MMLM).

20. The system according to claim 12, wherein, The one or more processors are also used for: Perform at least one of fuzz testing or unit testing to automatically identify the defects in the source code.

21. The system according to claim 12, wherein, The context information includes at least one of the following: diagnostic tool identifier, test case identifier, file path, function name, line number, defect type, stack trace, structured data, log extract, reproducible scenario, or diagnostic data.

22. The system according to claim 12, wherein, The system is at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system for performing one or more simulation operations; A system for performing one or more digital twin operations; A system for performing one or more optical transmission simulations; A system for performing collaborative content creation for 3D assets; A system for performing one or more wireless cellular transmissions using a wireless cellular network; A system that provides one or more cloud gaming applications; A system for performing one or more deep learning operations; Systems implemented using edge devices; Systems implemented using robots; A system for performing one or more generative AI operations; A system for performing one or more conversational AI operations; A system for performing operations using one or more large language model LLMs; A system for performing operations using one or more visual language models (VLMs); A system for performing operations using one or more multimodal language models (MMLMs); A system for performing operations using one or more visual-language-action (VLA) models; A system for performing one or more conversational AI operations; A system for performing one or more synthetic data generation operations; A system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; A system that uses or deploys one or more inference microservices; A system that combines one or more machine learning models with operating system-level virtualization packages (e.g., containers) and deploys them into services or microservices; A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

23. A method comprising: Use processing equipment to identify defects in the source code; Obtain contextual information regarding the defect; The first prompt is processed using one or more artificial intelligence (AI) models to generate a corrected version of the source code, the first prompt being generated based at least on the source code, the defect, and the contextual information regarding the defect; Determine whether the corrected version of the source code output by the one or more AI models resolves the defect; as well as In response to determining that the corrected version of the source code resolves the defect, the changes implemented in the corrected version of the source code are applied to the source code.

Images