Method, device and system for modifying machine learning model

Abstract

The invention provides a method, equipment and a system for modifying a machine learning model. The method includes performing, by a machine learning model, a generation process to predict a first output, generating, via a processor, a potential space based on an input to the machine learning model, determining, via the processor, intermediate decision parameters based on the potential space, and generating, via the processor, a second output based on the intermediate decision parameters. Changing, via the processor, a structure of the machine learning model to generate a modified machine learning model to perform a modified generation process conditioned by the intermediate decision parameters, and generating, by the modified machine learning model, a second output comprising content associated with the input.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and benefit to U.S. Provisional Application Serial No. 63 / 650,334, entitled “SYSTEM AND METHOD FOR ACCELERATING TRAINING AND INFERENCE OF LARGE-SCALE GENERATIVE AI MODELS”, filed May 21, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Some aspects of embodiments of this disclosure relate to a method, apparatus, and system for modifying machine learning models. Background Technology

[0004] Conditional generative models can be leveraged across various domains due to their ability to capture complex data distributions while allowing conditional generation based on a given context. Variational generative autoencoders are widely used models that learn latent representations that generate controlled and diverse outputs by conditioned on specific input variables. Variational generative autoencoders can model the dependencies between latent variables and data points under given conditions. They have become a powerful tool in machine learning tasks including language modeling, image generation, and autonomous systems.

[0005] While they have been successful in many scenarios, there may be limitations associated with variational generative autoencoders (GGEs) because the output may be generated solely based on latent variables and the input as conditions. Therefore, in some cases, GGEs may lack the flexibility to incorporate intermediate control factors that could play a crucial role in the generation process, thus limiting the model's ability to capture more complex dependencies and potentially restricting its applicability in real-world scenarios.

[0006] The field of artificial intelligence (AI) has made progress in machine learning models, such as language models. For example, large language models (LLMs) have been developed for various natural language processing tasks, leading to the development of related AI-based services. However, the large model size and heavy computational cost of LLMs pose significant challenges to model training and inference, especially in resource-constrained environments. Systems and methods can be adapted to reduce model complexity to accommodate the computational operations of various hardware platforms (e.g., conventional hardware platforms) and to dynamically tailor LLMs for different downstream tasks.

[0007] This background section is intended to provide context only, and the disclosure of any embodiments or concepts in this section does not constitute an admission that the embodiments or concepts are prior art. Summary of the Invention

[0008] Some aspects of embodiments of this disclosure relate to computing systems with improved memory management.

[0009] According to some embodiments of this disclosure, a method for modifying a machine learning model is provided, the method comprising: performing a generation process by the machine learning model to predict a first output; generating a latent space via a processor based on inputs to the machine learning model; determining intermediate decision parameters via the processor based on the latent space; modifying the structure of the machine learning model via the processor based on the intermediate decision parameters to generate a modified machine learning model to perform a modified generation process conditioned on the intermediate decision parameters; and generating a second output by the modified machine learning model including content associated with the inputs.

[0010] The second output can be conditional on latent variables and intermediate decision parameters.

[0011] Generating a latent space can include embedding the input and previously generated output from a machine learning model into a data distribution.

[0012] Intermediate decision parameters can be determined based on the input and the relationship inferred from the embedding.

[0013] Determining intermediate decision parameters may include sampling the latent space based on latent variables.

[0014] Changing the structure of a machine learning model can include modifying the generation process to infer indirect relationships between latent variables and the second output.

[0015] Machine learning models can be large language models.

[0016] Changing the structure of a machine learning model can include performing structured pruning tasks on large language models.

[0017] Changing the structure of a machine learning model can include applying a structured pruning task to dynamically prune large language models using rules conditioned on intermediate decision parameters.

[0018] The structured pruning task can be used to dynamically remove at least one of parameters, heads, nodes, edges, or weights from a large language model.

[0019] The structured pruning task can generate a large language model that has been pruned down from a large language model, and can generate outputs conditioned on intermediate decision parameters.

[0020] Structured pruning tasks can include rules conditioned on intermediate decision parameters and latent variables.

[0021] The method may also include determining a second intermediate decision parameter based on a second latent variable and a second input, and generating a structured modification task conditioned on the second intermediate decision parameter.

[0022] The content can include automatically generated images, text, audio, and video based on the input.

[0023] According to some other embodiments of this disclosure, an apparatus is provided comprising one or more processors configured to perform a generation process to predict a first output using a machine learning model, generate a latent space based on inputs to the machine learning model, determine intermediate decision parameters based on the latent space, modify the structure of the machine learning model based on the intermediate decision parameters to generate a modified machine learning model to perform a modified generation process conditioned on the intermediate decision parameters, and generate a second output including content associated with the inputs based on the modified machine learning model.

[0024] The one or more processors can be configured to perform the generation of a second output conditioned on intermediate decision parameters.

[0025] The one or more processors can also be configured to perform the generation of a latent space by embedding the input and previously generated output from the machine learning model into the data distribution.

[0026] The one or more processors can also be configured to perform the determination of intermediate decision parameters by sampling the latent space based on latent variables.

[0027] The one or more processors can also be configured to perform tasks that alter the structure of a machine learning model by performing structured pruning tasks.

[0028] According to some other embodiments of this disclosure, a system is provided, including processing circuitry and a memory storing instructions that, when executed by the processing circuitry, cause the processing circuitry to perform a generation process to predict a first output using a machine learning model, generate a latent space based on input to the machine learning model, determine intermediate decision parameters based on the latent space, modify the structure of the machine learning model based on the intermediate decision parameters to generate a modified machine learning model to perform a modified generation process conditioned on the intermediate decision parameters, and generate a second output including content associated with the input based on the modified machine learning model. Attached Figure Description

[0029] Non-limiting and non-exhaustive embodiments of this disclosure are described with reference to the following figures, wherein, unless otherwise stated, the same reference numerals refer to the same parts in the various views.

[0030] Figure 1AThis is a block diagram depicting a computer device for dynamically modifying a machine learning model conditioned on intermediate decision parameters, according to some embodiments of the present disclosure.

[0031] Figure 1B This describes an example configuration for dynamically modifying a large language model conditioned on intermediate decisions, according to some embodiments of the present disclosure. Figure 1A A block diagram of a variational generative (VG) machine learning (ML) model processor (e.g., a VG-ML processor) in a computer device.

[0032] Figure 2 It is a description of some embodiments according to this disclosure and by Figure 1B A diagram illustrating the relationships related to intermediate decisions implemented by the VG-ML model processor.

[0033] Figure 3 This is a flowchart depicting the operations of a method for dynamically modifying a machine learning model conditioned on intermediate decision parameters, according to some embodiments of the present disclosure.

[0034] Figure 4 This is a flowchart depicting the operations of a method for modifying a machine learning model according to some embodiments of the present disclosure.

[0035] Throughout the various views in the accompanying drawings, corresponding reference numerals indicate the corresponding components. Those skilled in the art will understand that the elements in the drawings are shown for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements, layers, and regions in the figures may be exaggerated relative to other elements, layers, and regions to aid in clarity and understanding of the various embodiments. Furthermore, common but easily understood elements and components unrelated to the description of the embodiments may not be shown to minimize obstruction of the views of these various embodiments and to ensure clarity of description. Detailed Implementation

[0036] The aspects of this disclosure and its implementation methods can be more readily understood by referring to the detailed description and accompanying drawings of one or more embodiments. Hereinafter, embodiments will be described in more detail with reference to the accompanying drawings. However, the described embodiments may be implemented in various different forms and should not be construed as being limited to the embodiments shown herein. Rather, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects of this disclosure to those skilled in the art; therefore, descriptions of processes, elements, and techniques unnecessary for those skilled in the art to fully understand the aspects and features of this disclosure may be omitted.

[0037] Unless otherwise stated, the same reference numerals, characters, or combinations thereof denote the same elements throughout the drawings and written description, and therefore their description will not be repeated. Those skilled in the art will understand that the elements in the drawings are shown for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements, layers, and areas in the figures may be exaggerated relative to other elements, layers, and areas to aid in clarity and understanding of the various embodiments. Furthermore, common but well-understood elements and components unrelated to the description of the embodiments may not be shown to minimize obstruction of the view of these various embodiments and to make the description clearer.

[0038] In the detailed description, numerous specific details are set forth for illustrative purposes to provide a thorough understanding of the various embodiments. However, it will be apparent that various embodiments may be practiced without these specific details or with one or more equivalent arrangements.

[0039] It should be understood that although the terms “zeroth,” “first,” “second,” “third,” etc., may be used herein to describe various elements, components, regions, layers, and / or portions, these elements, components, regions, layers, and / or portions should not be limited by these terms. These terms are used to distinguish one element, component, region, layer, or segment from another element, component, region, layer, or segment. Therefore, without departing from the spirit and scope of this disclosure, the first element, component, region, layer, or portion described below may be referred to as the second element, component, region, layer, or portion.

[0040] It should be understood that when an element or component is referred to as being "on," "connected to," or "coupled to" another element or component, it can be directly on, connected to, or coupled to the other element or component, or there may be one or more intermediate elements or components. However, "direct connection / direct coupling" means that one component is directly connected to or coupled to another component without any intermediate components. Similarly, other expressions describing relationships between components, such as "between," "directly between," or "adjacent to" and "directly adjacent to," can be interpreted in a similar way. Furthermore, it will be understood that when an element or component is referred to as being "between" two elements or components, it can be the only element or component between the two elements or components, or there may be one or more intermediate elements or components.

[0041] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “an” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “comprising,” “having,” and “including” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, each of the terms “or” and “and / or” includes any and all combinations of one or more of the associated listed items. For example, the expression “A and / or B” means A, B, or A and B.

[0042] For the purposes of this disclosure, expressions such as “at least one of…” modify the entire list of elements when preceding it, and do not modify individual elements in the list. For example, “at least one of X, Y, or Z,” “at least one of X, Y, and Z,” and “at least one selected from the group consisting of X, Y, and Z” can be interpreted as only X, only Y, only Z, or any combination of two or more of X, Y, and Z, such as XYZ, XYY, YZ, and ZZ.

[0043] As used herein, the terms “substantially,” “about,” “approximately,” and similar terms are used as approximate terms rather than terms of degree and are intended to describe the inherent bias of a measured or calculated value that will be recognized by one of ordinary skill in the art. As used herein, “about” or “approximately” includes the value and means within an acceptable range of deviation for a particular value, as determined by one of ordinary skill in the art, taking into account the measurement in question and the errors associated with the measurement of the particular quantity (i.e., limitations of the measurement system). For example, “about” may mean within one or more standard deviations, or within ±30%, 20%, 10%, or 5% of the value. Furthermore, the use of “may” in describing embodiments of this disclosure means “one or more embodiments of this disclosure.”

[0044] When one or more embodiments can be implemented differently, a particular process sequence can be performed differently than the described sequence. For example, two consecutively described processes can be performed substantially simultaneously or in the reverse order of their description.

[0045] Any component or combination of components described herein (e.g., in any system diagram included herein) may be used to perform one or more operations of any flowchart included herein. Furthermore, (i) the operations are merely examples and may involve various additional operations not explicitly covered, and (ii) the temporal order of the operations may vary.

[0046] The electronic or electrical devices and / or any other related devices or components according to embodiments of this disclosure described herein can be implemented using any suitable hardware, firmware (e.g., application-specific integrated circuits), software, or a combination of software, firmware, and hardware. For example, various components of these devices may be formed on an integrated circuit (IC) chip or a separate IC chip. Furthermore, various components of these devices may be implemented on a flexible printed circuit film, tape-on-a-carrier package (TCP), printed circuit board (PCB), or formed on a substrate.

[0047] Furthermore, the various components of these devices may be processes or threads running on one or more processors in one or more computing devices, executing computer program instructions and interacting with other system components to perform the various functions described herein. The computer program instructions are stored in memory, which may be implemented in the computing device using standard memory devices such as random access memory (RAM). The computer program instructions may also be stored in other non-transitory computer-readable media, such as CD-ROMs, flash drives, etc. Moreover, those skilled in the art will recognize that, without departing from the spirit and scope of the embodiments of this disclosure, the functions of various computing devices may be combined or integrated into a single computing device, or the functions of a particular computing device may be distributed across one or more other computing devices.

[0048] Any functionality described herein (including any functionality that can be implemented by a host, device, etc., or a combination thereof) can be implemented in hardware, software, firmware, or any combination thereof, including, for example, hardware and / or software combinational logic, sequential logic, timers, counters, registers, state machines, volatile memory (such as dynamic RAM (DRAM) and / or static RAM (SRAM)), non-volatile memory (including flash memory), persistent memory (such as cross-grid non-volatile memory), memory with varying bulk resistance, phase-change memory (PCM), etc., and / or any combination thereof, complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), central processing units (CPUs) including complex instruction set computer (CISC) processors and / or reduced instruction set computer (RISC) processors, graphics processing units (GPUs), neural processing units (NPUs), tensor processing units (TPUs), data processing units (DPUs), etc., which execute instructions stored in any type of memory. In some embodiments, one or more components may be implemented as a system-on-a-chip (SoC).

[0049] Any computing device disclosed herein can be implemented using any connector configuration, such as Serial Advanced Technology Attachment (SATA), Small Computer System Interface (SCSI), Serial Attached SCSI (SAS), U.2, etc., in any form factor, such as 3.5-inch, 2.5-inch, 1.8-inch, M.2, Enterprise and Data Center Standard Form Factor (EDSFF), NF1, etc. Any computing device disclosed herein can be implemented and / or used entirely or partially with server chassis, server racks, data centers, data centers, edge data centers, mobile edge data centers, and / or any combination thereof.

[0050] Any device disclosed herein that can be implemented as a storage device can be implemented using any type of non-volatile storage medium based on solid-state media, magnetic media, optical media, etc. For example, in some embodiments, the storage device (e.g., a computing storage device) can be implemented as an SSD based on NAND flash memory, persistent memory such as cross-grid non-volatile memory, memory with varying bulk resistance, PCM, etc., or any combination thereof.

[0051] Any communication connections and / or communication interfaces disclosed herein may be implemented using any type of interface and / or protocol utilizing one or more interconnects, one or more networks, networks of networks (e.g., the Internet), or combinations thereof. Examples include Peripheral Component Interconnect Fast (PCIe), Non-Volatile Memory Fast (NVMe), NVMe-over-fabric (NVMe-oF), Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Direct Memory Access (DMA) Remote DMA (RDMA), RDMA over Converged Ethernet (ROCE), Fibre Channel, InfiniBand, SATA, SCSI, SAS, Internet Wide Area RDMA Protocol (iWARP) and / or coherent protocols (such as Compute Fast Link (CXL), CXL.mem, CXL.cache, CXL.10, etc.), Gen-Z, Open Coherent Accelerator Processor Interface (OpenCAPI), Cache Coherent Interconnect for Accelerators (CCIX), Advanced Scalable Interface (AXI), any generation of wireless networks including 2G, 3G, 4G, 5G, 6G, etc., any generation of Wi-Fi, Bluetooth, Near Field Communication (NFC), etc., or any combination thereof.

[0052] In some embodiments, the software stack may include a communication layer that implements one or more communication interfaces, protocols, such as PCIe, NVMe, CXL, Ethernet, NVMe-oF, TCP / IP, etc., to enable the host and / or applications running on the host to communicate with computing devices or storage devices.

[0053] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms (such as those defined in common dictionaries) shall be interpreted as having the same meaning as they have in the relevant field and / or the context of this specification, and shall not be interpreted as having an idealized or overly formal meaning unless expressly defined herein.

[0054] As mentioned above, the AI ​​field can experience advancements in machine learning models (e.g., language models). For example, LLMs have been developed for various natural language processing tasks, leading to the development of related AI-based services. However, the large model size and heavy computational cost of LLMs can pose significant challenges to model training and inference, especially in resource-constrained environments. Systems and methods can be adapted to reduce model size and / or complexity to accommodate computational operations on various hardware platforms (e.g., conventional hardware platforms) and to dynamically adjust the structure of LLMs to suit different downstream tasks.

[0055] Figure 1A This is a block diagram depicting a computer device 100 for dynamically modifying a machine learning model 122 conditioned on intermediate decision parameters d 124, according to some embodiments of the present disclosure.

[0056] As used in this article, “dynamically embedding variational generation into a machine learning model” means dynamically (e.g., iteratively) updating, modifying, and / or adjusting a given machine learning model for improved performance or for performing new tasks.

[0057] like Figure 1A As shown, computer device 100 (e.g., one or more computers and / or one or more computer systems) may include memory 102 (e.g., memory and / or storage devices), processor 104, and VG-ML model processor 110, which is configured to implement variational generation functions to efficiently and flexibly process machine learning models. In general description, VG-ML model processor 110 may be configured to perform one or more functions related to the variational generation aspect of machine learning modeling, as disclosed herein, which may involve determining intermediate decision parameters d 124 that introduce variations into the relationship modeled between latent variables z 123 and output y 125 based on the defined representation conditions of the input c 121.

[0058] According to some embodiments, machine learning models generated, trained, and / or utilized by the VG-ML model processor 110 can be dynamically modified to be robust, flexible, and / or conditionally capable of generating different outputs. Dynamic modification of the machine learning model can involve changing the model's structure, parameters, or behavior on an ad-hoc basis during training or runtime (e.g., inference) so that the model does not need to be retrained from scratch. Furthermore, dynamically modifying the machine learning model can allow the model to adapt to new data, environments, or requirements substantially in real-time (or pseudo-real-time) as it is used. As an example, dynamic modification of the machine learning model occurring in different instances and / or time periods during the training phase can generate significantly different structures for the model in each iteration, instance, or time period. Reference Figure 1A The VG-ML model processor 110 can dynamically modify the machine learning model 122 in a first iteration, which generates a first modified machine learning model 126, and can then dynamically modify the machine learning model 122 in a second iteration, such that a second modified machine learning model 126 with a different structure from the first modified machine learning model 126 is generated.

[0059] For example, the machine learning model 122 can be dynamically modified for each iteration relative to a given output y 125 and a given input c 121. For each iteration, the output y 125 and the input c 121 can be different, which can lead to a machine learning model 126 with significant dynamic optimization and / or modification. Figure 1A In this context, the VG-ML model processor 110 may include processing circuitry configured to implement a variational generation function, including: an input embedding circuit 111; a decision encoder circuit 112; a decision decoder circuit 113; and an ML model modification circuit 114. According to some embodiments, the machine learning models generated, trained, and / or utilized by the VG-ML model processor 110 are generative AI models. For example, some generative AI models are suitable for AI applications, where complex and high-dimensional data with diversity and specificity can be generated. In the case of large language models, the model can generate complex and high-dimensional data for various applications, including natural language applications, image generation applications, multimodal applications, and one or more other AI-based applications.

[0060] Computer device 100 may include a computer system capable of performing AI-related functions, including model training, computation, inference, and various AI-based applications. For example, computer device 100 may be implemented as, but is not limited to, a desktop PC, laptop computer, smartphone, tablet PC, server, etc. Computer device 100 may also refer to a system in which a cloud computing environment is established. However, the exemplary embodiments are not limited thereto. Computer device 100 may be implemented as any system, device, or apparatus capable of performing AI-based and / or ML-based applications and functions, as described herein. In some embodiments, computer device 100 may implement a database search engine that supports AI-based capabilities, such as using AI to understand and generate human-like text based on a large amount of text data that can be stored and searched within a database. For example, a user can input text into computer device 100 representing natural language that a human user would speak in conversation, such as words, sentences, phrases, etc. Subsequently, computer device 100 can utilize a machine learning model 122 (e.g., a large language model) trained and optimized by VG-ML model processor 110 to return (e.g., predictively return) recommended words, phrases, or sentences deemed most appropriate from the database in order to generate text provided to the user by the search engine in a coherent format similar to human conversation (e.g., based on the user's sequential word input).

[0061] In some embodiments, processor 104 may be configured to further utilize one or more machine learning models. For example, processor 104 may be configured to utilize machine learning model 122, modified machine learning model 126, large language models, deep learning models, neural networks, large-scale generative AI models, etc. In some embodiments, VG-ML model processor 110 may be configured to generate, train, and / or optimize machine learning model 122, modified machine learning model 126, large language models, deep learning models, neural networks, large-scale generative AI models, etc. VG-ML model processor 110 may be configured to generate, train, and / or optimize models to enable functions that support AI-based and / or ML-based capabilities of computer device 100. For example, machine learning model 122 may be trained to support several capabilities of computer device 100, which may include: high-level natural language processing (e.g., chatbots); image generation; multimodal processing (e.g., speech, audio, text, video, image, etc.); autonomous systems; and / or the like.

[0062] According to some embodiments, the VG-ML model processor 110 is configured to perform (e.g., configured to perform) structured and / or dynamic modifications of a machine learning model 122 based on intermediate decision parameters d 124, utilizing variational generation functions disclosed herein. This can improve the machine learning model and its applications (e.g., achieve optimization of the machine learning model and its applications). In some embodiments, the VG-ML model processor 110 is configured to perform pruning (e.g., structured pruning) of a large language model based on intermediate decision parameters d 124 that can be determined according to variational generation aspects disclosed herein. The VG-ML model processor 110 can generate one or more structured pruning tasks involving token-by-token adaptive pruning rules. Token-by-token adaptive pruning rules can enhance the context awareness of the pruned large language model by dynamically adjusting the model structure (e.g., selectively removing less important parameters from the model's neural network) to maximize computational efficiency (e.g., reduce computational overhead), minimize model size, and / or provide overall improvements to the large language model (e.g., optimization) without significantly sacrificing performance. In some embodiments, the disclosed VG-ML model processor 110 may be configured to perform (e.g., execute) any suitable structured and / or dynamic modifications to the machine learning model 122 to generate a modified machine learning model 126. In some embodiments, the VG-ML model processor 110 may be configured to perform one or more structured and / or dynamic modifications in addition to or in addition to structured pruning. The structured and / or dynamic modifications may be based on the variational generation aspects disclosed herein. For example, the output y 125 of the modified machine learning model 126 may be based on conditional independence and additional variation layers introduced by intermediate decision parameters d 124. The structured and / or dynamic modifications may provide enhancements (e.g., optimizations) related to the ML model, including but not limited to: efficient processing; high computational efficiency; reduced latency; reduced power consumption; enhanced flexibility and interpretability; improved accuracy and / or generation quality (e.g., by explicitly modeling intermediate factors affecting the data generation process); and / or the like. In some embodiments, the VG-ML model processor 110 may be configured to perform structured and / or dynamic modifications to the machine learning model 122, including but not limited to: pruning (e.g., unstructured pruning); multimodal output selection (e.g., determining the output modality based on conditions and intermediate factors); expert blending (e.g., selecting one or more experts to contribute to the ML model output); and / or the like.

[0063] VG-ML model processor 110 may include various processing circuitry (e.g., may include one or more processors) and may be configured to implement operations related to generating, training, and / or utilizing ML models (e.g., utilizing large-scale generative AI models). VG-ML model processor 110 may be implemented as a standalone processor, one or more microprocessors, a controller, a CPU, or a processing device utilizing any suitable hardware, firmware (e.g., application-specific integrated circuits), software, or a combination of software, firmware, and hardware. VG-ML model processor 110 may be configured to implement various variational generation functions as described herein related to generating, training, and / or utilizing ML models.

[0064] The variational generation function executed by the VG-ML model processor 110 can involve modeling complex data distributions across various domains, thereby leveraging the ability to conditionally generate different outputs. In the realm of AI, a limitation (e.g., a challenge) that may be experienced when dealing with machine learning models is the direct dependency developed between the latent variable z and the output y of a given input c. This direct dependency (e.g., between the latent variable z and the output y) can constrain the applicability of machine learning models in AI-based applications that may require the generation of different outputs. Limitations may arise because the conditions indicated by the input c may already be fixed as inputs, restricting the flexibility to introduce additional intermediate factors that can more effectively control the generation of the output y.

[0065] To address this and other issues, the disclosed VG-ML model processor 110 can be configured differently to determine intermediate decision parameters d 124, which incorporate mediating factors into its generation process. By modeling probabilistic sampling of the latent space 129 using latent variables z 123, the mediating factors can be based on the latent variables z 123 inferred through variational methods and the input c 121 (also referred to as "p(d|z, c)"). As used herein, a "generation process" refers to the generation of statistically probable outputs by a machine learning model based on one or more inputs to the machine learning model. Given the intermediate decision parameters d 124 and the input c 121, the variational generation aspect implemented by the VG-ML model processor 110 can provide (e.g., can be implemented) conditional independence between the latent variables z 123 and the output y 125.

[0066] VG-ML model processor 110 can be configured to determine intermediate events represented by intermediate decision parameters d 124. In some embodiments, VG-ML model processor 110 can train a large language model, and intermediate decision parameters d 124 can represent intermediate events and / or conditions that may significantly affect the accuracy and / or robustness of the model's generation process. For example, relative to a large language model, intermediate decision parameters d 124 can represent mediating factors (e.g., intermediate factors) that may be associated with the language-based generation process, such as sentence structure, style in language modeling, or a specific model configuration used for pruning. Subsequently, determining the intermediate decision parameter d 124 allows the machine learning model to model the predictive inference between the output y125 and the input c 121 based on the intermediate decision parameter d 124 (also referred to as "p(y|d, c)") (e.g., indirectly dependent on the intermediate decision parameter d 124, rather than directly dependent on the latent variable z 123 by modeling p(y|z, c)), in a manner influenced (e.g., indirectly by) the latent variable z 123 and the defined mediation conditions.

[0067] VG-ML model processor 110 can be configured to perform one or more dynamic and / or structured modification tasks on machine learning model 122, resulting in a flexible generative process performed by the modified machine learning model 126, which is widely applicable to a variety of AI-based applications where complex and high-dimensional data can be generated. VG-ML model processor 110 can be configured to leverage variational generative aspects to define and / or perform various dynamic and / or structured modification tasks, including conditional image generation, language model pruning, modeling using expert hybrids, etc. As used herein, a “structured modification task” refers to a task performed according to defined rules and / or strategies (e.g., by the processor) to alter the structure of the machine learning model (e.g., changing parameters, type, format, etc.).

[0068] Still referencing Figure 1AThe input embedding circuit 111 is shown as receiving an input c 121 and an output y 125. For example, during training, the output y 125 may be provided as input to the input embedding circuit 111. The input c 121 may be data (e.g., characters, text, images, videos, audio, etc.) deemed suitable for being fed (e.g., provided) to a machine learning model 122 generated, trained, and / or optimized by the VG-ML model processor 110. The output y 125 may be a prediction or decision made by the modified machine learning model 126 based on an instance of the input c 121 (e.g., a current or previous instance). In some embodiments, the input c 121 may be an input token of a large language model. An input token may be a basic unit of text that the model processes as input. Thus, the input c 121 as an input token may be a word, character, subword, one or more phrases, and / or one or more sentences that are fed into the machine learning model 122 (e.g., a large language model) as part of a prompt. In some embodiments, the output y 125 may be an output token of a large language model. An output token can be a unit of information generated by a large language model after the large language model processes the input c 121. Therefore, the output y 125, as an output token, can be a single character, one or more words, one or more sub-words, one or more phrases, and / or one or more sentences output from a modified machine learning model 126 (e.g., a large language model). In some embodiments, the input embedding circuit 111 is configured to perform a process of transforming the input c 121 and the output y 125 (e.g., feedback from the output y 125) (which can be received as raw text) into a format that the machine learning model 122 can understand and utilize. The functions performed by the input embedding circuit 111 can involve tokenization, embedding, and / or positional encoding. In some embodiments, the input embedding circuit 111 can iteratively receive the input c 121 and the output y 125 according to the training process of the machine learning model 122 executed by the VG-ML model processor 110. In some embodiments, the input embedding circuit 111 can be included (e.g., optionally included) in the configuration of the VG-ML model processor 110. For example, in some embodiments, the input embedding circuit 111 may be included during the processing of a large language model (e.g., it may be adapted for the processing of a large language model).

[0069] The features associated with input c 121 can define the relationships observed by machine learning model 122 to learn inferences over time and ultimately generate an output y 125 corresponding to a given input c 121. In other words, output y 125 can be conditionally generated by a modified machine learning model 126 based on input c 121, which indicates the specific context and / or conditions associated with the inference. In such an embodiment, input c 121 may also be referred to as conditions.

[0070] Still referencing Figure 1A In some embodiments, the decision encoder circuit 112 may receive an input c 121 and an output y 125 formatted by the input embedding circuit 111. For example, during the training (or retraining) phase of the machine learning model 122, the input embedding circuit 111 may perform (e.g., may perform) a mapping of the output y 125 corresponding to a given input c 121 to a latent space 129, which may be represented as a data distribution (e.g., a Gaussian distribution, etc.). During the training (or retraining) process, the decision encoder circuit 112 may be configured to iteratively embed (e.g., provide or record) one or more instances of the input c 121 and the output y 125 into the data distribution, which may generate a latent space, and the positions of these points in the latent space (e.g., a continuous vector space) may be meaningful for the machine learning algorithm. As used herein, “embedding” refers to the process of mapping a set of data points to a lower-dimensional space, where similar data points are grouped together closer together to model and / or infer patterns. The decision encoder circuit 112 may learn the embedding relationship between the input c 121 and the corresponding output y 125. For example, embedding relations can be defined by standard deviation, and various contextual and / or conditional relations are established as a latent space 129 by iteratively performing embeddings of input c 121 and output y 125 (e.g., iteratively performing embeddings against the complete training dataset) during the training process of machine learning model 122. Decision encoder circuit 112 can be configured to model the dependencies between latent variable z 123 and data points in the latent space 129 corresponding to output y 125, given conditions associated with input c 121. The encoding implemented by decision encoder circuit 112 to generate latent space 129 can be mathematically represented as:

[0071] q(z|y, c) (Equation 1)

[0072] Where q refers to the approximate posterior distribution modeled as a normal distribution, z refers to the latent variable 123, y refers to the output y125, q(z|y, c) refers to the conditional relation, and c refers to the input 121 used as the condition.

[0073] In some embodiments, the decision decoder circuit 113 can be configured to perform sampling of the latent space 129 (output from the decision encoder circuit 112) using the latent variable z 123, and based on this sampling, the decision decoder circuit 113 can: determine intermediate decision parameters d 124; and generate a structured and / or dynamically modified task of a machine learning model 122 conditioned on the input c 121 and the intermediate decision parameters d 124. The decision decoder circuit 113 can be configured (e.g., can have different capabilities to) introduce intermediate events via the intermediate decision parameters d 124, which contribute to (e.g., modify) additional direct dependencies between the latent variable z 123 and the output y 125. The decision decoder circuit 113 can implement a variational generative aspect involving conditional independence and factoring the generative process into two components (which can be generally described as determining the intermediate decision parameters d 124) and generating a structured and / or dynamically modified task of the machine learning model 122 based on the intermediate decision parameters d 124. The decision decoder circuit 113 can perform probability sampling based on the input c 121 (which represents the defined conditions) using the latent variable z 123 to apply the defined latent space 129 (generated by the decision encoder circuit 112) and determine the intermediate decision parameters d 124. The decoding process implemented by the decision decoder circuit 113 can be mathematically represented as:

[0074] p(d|z, c) (Equation 2)

[0075] Where p refers to conditional likelihood, d refers to intermediate decision parameter d124, z refers to latent variable d123, d|z, c refers to conditional relation, and c refers to input 121 used as condition.

[0076] Subsequently, the decision decoder circuit 113 can perform a structured and / or dynamically modified task to determine an intermediate decision parameter d 124, which models the intermediate factors influencing the data generation process using the structured and / or dynamically modified task. The formula executed by the decision decoder circuit 113 can later be used to infer the indirect relationship between the latent variable z 123 and the output y 125, providing an additional layer of flexibility. For example, the latent variable z 123 can influence the intermediate decision parameter d 124 (determined by the decision decoder circuit 113) based on a given input c 121; and then the output y 125 (determined by the ML model modification circuit 114) can be generated by a modified machine learning model 126 conditioned on the intermediate decision parameter d 124 and the input c 121.

[0077] According to an embodiment, the variational generation aspect implemented by the decision decoder circuit 113 can be modeled by sampling a latent space 129 using latent variables z123 (e.g., "p(d|z, c)") and introduced via variational inference to represent the mediating factors by computing intermediate decision parameters d124, which are based on latent variables z123 and input c121. By achieving conditional independence between latent variables z123 and output y125 given intermediate decision parameters d124 (and input c121), the decision decoder circuit 113 enables the modified machine learning model 126 to ultimately generate the final output y125, which is also conditional on the mediating events and / or factors. As disclosed herein, the function of the decision decoder circuit 113 is to determine the intermediate decision parameters d124, which can implement (e.g., provide) a structured and flexible generation process that can be performed by the modified machine learning model 126. For example, the VG-ML model processor 110 can generate, train, and / or improve (e.g., optimize) the generation process performed by the modified machine learning model 126, and thus the generation process can be more robust and flexible to be widely applicable to a variety of AI-based applications.

[0078] In some embodiments, the decision decoder circuit 113 can implement variational generation aspects, which generate structured and flexible generative processes, particularly for large language models. Thus, in some embodiments, the decision decoder circuit 113 can predict intermediate events and / or decisions that may be relevant factors in training a large language model. By determining intermediate decision parameters d 124, desired tuning of the large language model can be generated to adjust inferred relationships and predictions in a language-related manner. As an example, the decision decoder circuit 113 can be configured to determine intermediate decision parameters d 124, which represent mediating conditions that can influence natural language processing, such as specific sentence structures, style in language modeling, and / or a specific model configuration given (e.g., based on) latent variables z 123 and input c 121. In some embodiments, the decision decoder circuit 113 can be configured to generate structured pruning tasks for large language models as specific types of structured and / or dynamic modification tasks. The structured pruning task defined by the decision decoder circuit 113 can be a pruning strategy that determines the fit and / or reduction (e.g., minimization) of a large language model configuration by defining controlled selections of parameters, heads, weights, etc., to remove from the neural network configuration of the model. Depending on the variational generation aspect implemented by the decision decoder circuit 113, the structured pruning task can be a token-by-token adaptive pruning strategy and / or rule that enhances the context awareness of the pruned large language model by: dynamically adjusting the structure of the model; adapting to individual tokens by sampling from the learned latent distribution; and being improved (e.g., optimized) in the design of a transformer-based autoregressive large language model.

[0079] In some embodiments, the ML model modification circuit 114 can be configured to dynamically modify the configuration of the machine learning model 122 (e.g., change its structure) by applying a structured and / or dynamic modification task (determined by the decision decoder circuit 113), and thus generate a modified machine learning model 126. As a result of the function performed by the ML model modification circuit 114, the modified machine learning model 126 may have a configuration that has been dynamically adjusted and / or changed in a manner involving one or more additional intermediate factors represented by the intermediate decision parameter d 124 (and further conditioned on the input c 121) that can more effectively control the generation of the output y 125. Thus, the machine learning model 122 can be modified by the ML model modification circuit 114 to generate the modified machine learning model 126, thereby generating the output y 125 conditioned on the mediating events and / or factors represented by the intermediate decision parameter d 124. In other words, the ML model modification circuit 114 can be configured to generate the modified machine learning model 126 to adapt to inferring an indirect (rather than a direct) relationship between the latent variable z 123 and the output y 125. The ML model modification circuit 114 can be configured to generate a modified machine learning model 126 in a manner that allows the intermediate decision parameters d 124 to shape the final output y 125, which is influenced by the indirect latent variable z 123. The modification process implemented by the ML model modification circuit 114 can be mathematically represented as follows:

[0080] p(y|d, c) (Equation 3)

[0081] Where p refers to conditional likelihood, y refers to output y 125, d refers to intermediate decision parameter d 124, y|d, c refers to conditional relation, and c refers to input c 121 used as condition.

[0082] In some embodiments, the ML model modification circuit 114 can be configured to dynamically prune a large language model by removing at least one of weights, parameters, neurons, and / or heads from the configuration of a model (e.g., a model including a neural network) according to a structured pruning task (determined by the decision decoder circuit 113). In some embodiments, the dynamically pruned machine learning model can be configured to implement an indirect relationship with the output y 125 based on intermediate decision parameters d 124, and thus can predict the output y 125 with fewer parameters specifically for the input c 121 while maintaining context awareness and flexibility. Therefore, according to some embodiments, the ML model modification circuit 114 can perform the function of maintaining (or enhancing) overall context awareness by leveraging structured pruning, i.e., using an optimized and / or minimized (e.g., fewer parameters) large language model, as disclosed herein.

[0083] In some embodiments, the functions of the decision decoder circuit 113 and / or the ML model modification circuit 114 can be performed during the training and / or inference phases of the ML model utilized by the computer device 100. According to some embodiments, the inference utilizing the machine learning model functions of the decision decoder circuit 113 and / or the ML model modification circuit 114 can be performed by another separate computer device or processor.

[0084] According to the embodiments disclosed herein, the VG-ML model processor 110, including the circuitry described herein, can utilize various variational generation aspects to achieve functionality, including but not limited to: adaptively capturing the characteristics of input tokens to enhance the overall context awareness of the machine learning model, i.e., using fewer parameters from pruning; utilizing a probabilistic sampling process that draws from the learned latent distribution; and implementing structured pruning (conditional on intermediate decision parameter d 124) that can maximize computational efficiency and minimize the size and / or complexity of the model.

[0085] Memory 102 may store data and / or machine learning models associated with AI-based applications. In some embodiments, memory 102 may store models generated, trained, and / or modified by VG-ML model processor 110 to perform complex ML tasks, including advanced natural language processing (e.g., chatbots), image generation, multimodal processing (e.g., speech, audio, text, video, image, etc.), autonomous systems, etc. For example, multimodal processing may include generating video based on input text. In some embodiments, memory 102 may be configured to store machine learning model 122, modified machine learning model 126, large language models, deep learning models, neural networks, large-scale generative AI models, etc. Examples of data stored in memory 102 may include training datasets, data related to variational generative aspects implemented by VG-ML model processor 110 (e.g., data distribution, mean, variance, embeddings, etc.), data related to specific AI-based applications implemented by computer device 100 (e.g., data for natural language processing applications such as chatbots, database searches, etc.), etc. In some embodiments, memory 102 may store basic data comprising multiple sentences. The base data can be the data required to generate a language model. In other words, a language model can be generated by training on the base data. In some embodiments, memory 102 may also store a language model pruned by VG-ML model processor 110, as discussed above.

[0086] Processor 104 may include various processing circuits and can control the overall operation of computer device 100, including AI-based and / or ML-based applications supported by ML models generated, trained, and modified by VG-ML model processor 110, as disclosed herein. In some embodiments, processor 104 may be implemented as, for example, but not limited to, a digital signal processor (DSP), a microprocessor, or a time controller (TCON), etc., but is not limited thereto. The processor may be, for example, but not limited to, one or more of a dedicated processor, a central processing unit (CPU), a microcontroller unit (MCU), a microprocessor unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, etc., or may be defined as one of the foregoing terms. Furthermore, processor 104 may be implemented as a system-on-a-chip (SoC) in which processing algorithms are provided, or may be implemented in the form of a field-programmable gate array (FPGA), etc., but is not limited thereto.

[0087] In some embodiments, decoder circuitry 115 may be configured to convert an encoded representation of the data signaled to the output y 125 generated by ML model modification circuitry 114 into a desired output format for output from VG-ML model processor 110. In some embodiments, VG-ML model processor 110 may be configured to perform the generation, training, and / or modification of large language models (e.g., LLMs supporting AI-based natural language processing applications). Therefore, in the context of a large language model, decoder circuitry 115 may be configured to generate output y 125 as a sequence of words or output tokens based on the input c 121 provided to the model. At multiple iterations or time steps, the large language model implemented by VG-ML model processor 110 may use previous tokens or words to generate output y 125, which is a prediction of the next most likely token in the vocabulary. For example, as Figure 1A As depicted, the output y 125 can also be fed back to the VG-ML model processor 110 during the iteration process to be received as input (to the input embedding circuit 111). As previously described, the output y 125 from the decoder circuit 115 can be presented to the user via the computer device 100 in a format that includes characters, words, phrases, or sentences deemed (e.g., selected as) most appropriate, in order to generate text provided to the user by the application in a coherent format similar to human conversation (e.g., based on the user's sequential word input). In some embodiments, the decoder circuit 115 can be configured to implement multiple layers, including self-attention layers, cross-attention layers, and feedforward neural networks.

[0088] Still referencing Figure 1AThis illustrates an example configuration of the VG-ML model processor 110. Modifications can be made appropriately, for example, based on the intended AI application, without departing from the scope of this disclosure. Figure 1A The configuration shown is illustrated. In some embodiments, elements of the VG-ML model processor 110 may optionally be included and / or removed; for example, for applications not involving large language model processing, the input embedding circuit 111 and decoder circuit 115 may be excluded from the circuitry of the VG-ML model processor 110. As previously described, the VG-ML model processor 110 can be structured (e.g., specifically structured) to generate, train, and / or modify large language models (e.g., transformer-based autoregressive models) and to implement structured pruning tasks of large language models (such as... Figure 1B As shown and described in more detail below.

[0089] Figure 1B This describes an example configuration of a large language model 162 conditioned on intermediate decision parameter d 124, according to some embodiments of the present disclosure. Figure 1A A block diagram of the VG-ML model processor 110 in a computer device.

[0090] Now for reference Figure 1B The configuration of the VG-ML model processor 110 can be structured (e.g., specifically structured) to generate, train, and / or modify large language models, for example, to support AI-based language processing applications (e.g., chatbots, database search engines, etc.). Figure 1B As depicted, in some embodiments, the dashed line shape and / or dashed arrow may be visible only during training time (e.g., the training phase) (e.g., provided). In some embodiments, the decision decoder circuit 113 (such as...) Figure 1B The dynamic structured pruning task (described in the paper) can be implemented. Based on the variational generation aspect disclosed in this paper, the dynamic structured pruning task enables a dynamic and context-aware pruning strategy for large language models that adapts to individual tokens by sampling from a learned latent distribution. As a general description, Figure 1B The circuitry of the VG-ML model processor 110 shown above has the same characteristics as the reference above. Figure 1A The descriptions are basically similar in structure and function. However, Figure 1BA VG-ML model processor with circuitry associated with its transformer functionality is described, which may include: multi-head attention circuits 116 and 131 (e.g., masked multi-head attention circuits) that can be configured to execute multiple attention mechanisms in parallel to process information from the input sequence and then concatenate and / or linearly transform the relevant attention outputs to the desired dimension; and feedforward circuits 117, 118, and 132 that can be configured to pass information in direction (e.g., to sub-units within each encoder, decoder, and machine learning layer).

[0091] As discussed above, the AI ​​and ML fields have experienced recent technological advancements related to language models. Specifically, large-scale language models have set new benchmarks across a variety of natural language processing applications. However, the massive model size and heavy computational costs associated with such large-scale machine learning models can pose significant challenges to model inference, especially in resource-constrained environments. To mitigate the aforementioned and other drawbacks, the VG-ML model processor 110 can be configured to perform different structured pruning tasks based on determined intermediate decision parameters d 124 as a solution to reduce the size and / or complexity of such large-scale language models, which can be aligned with the computational operations of conventional hardware platforms. As disclosed herein, Figure 1B The VG-ML model processor 110 in the document can implement various variational generative aspects, which can improve (e.g., optimize) computational efficiency and reduce (e.g., minimize) the model size associated with generation, training, and / or utilization of large language models. For example, Figure 1B The large language model 162 described in the text can be Figure 1A The type of machine learning model 122 in the text, and Figure 1B The large language model 166 in the pruning process can be Figure 1A The modified machine learning model 126 in the model.

[0092] According to some embodiments of this disclosure, the decision decoder circuit 113 is configured to determine a token-by-token structured pruning task conditioned on latent variables z 123 and intermediate events, such as dynamically adjusting the configuration of the large language model 162 during training and / or inference phases associated with the large-scale model (e.g., pruning the head from the neural network) to generate a pruned large language model 166, thereby enhancing the context awareness of the large language model 162. The intermediate events can be represented by intermediate decision parameters d 124 (e.g., the indirect relationship between output y 125 and latent variables z 123). In some embodiments, the structured pruning task implemented by the decision decoder circuit 113 is performed as an enhancement, in addition to one or more existing pruning strategies applied to the large language model 162, to improve computational complexity. The decision decoder circuit 113 can leverage various variational generative aspects to enable the large language model 162 to be more effectively adapted to domain-specific contexts and can improve (e.g., optimize) model pruning to maintain or even enhance its performance while reducing model complexity. The functionality of the decision decoder circuit 113 (including the structured pruning of the large language model 162) enables (e.g., can achieve) several advantages, such as reducing the complexity of the neural network without compromising its predictive accuracy, and improving the overall performance of the large language model even after pruning.

[0093] In some embodiments, the decision decoder circuit 113 is configured to perform the processes, computations, and / or algorithms described below as functions of implementing the variational generation aspects of this disclosure, including determining intermediate decision parameters d 124 and implementing the structured pruning task disclosed herein. Given a large language model 162 (e.g., M) with parameters θ and a dataset T 135 (e.g., T = {(c, y)}), the pruning decision network D 133 aims to determine a set of attention heads within the large language model 162 (e.g., M) that should be pruned to improve (e.g., optimize) the model performance on the inputs c 121 and corresponding outputs y 125 of the training data (e.g., on each pair of T). The pruning decision network D 133 may generate probability distributions on heads 134 (e.g., attention heads) indicating the likelihood of each head 134 that should be pruned based on an instance of the input c 121. The output of the pruning decision network D133 for each head 134 of the input c121 can be represented as p(h|c; θD), where θD represents the parameters of the pruning decision network D133. In this way, the structured pruning task can be defined as selecting a head 134 for which p(h|c; θD) is ranked among the top K highest values, determined by the number of heads 134 (e.g., N) and the pruning ratio s. This decision-making mechanism ensures that the pruning strategy dynamically adapts to the input c 121, thereby leveraging the predictive power of the pruning decision network D 133 to prune the head 134, while optimizing performance and efficiency by adapting its structure to each data instance of the dataset T 135.

[0094] The resulting pruned large language model 166 can be defined by a subset of the heads 134 retained after pruning, and therefore the optimization for the pruned large language model 166 can be mathematically expressed as:

[0095] (Equation 4)

[0096]

[0097] Where θ h This represents the parameter of the header h, and

[0098] θ' represents the parameter of the pruning model, which is used to remove θ from M. h It was obtained later.

[0099] The criteria for the structured pruning task can be set by a selection function S that selects K heads based on the pruning ratio s. Optimization can be computed to adjust both the parameters θ' of the pruning model and θD of the pruning decision network D133 to maximize the likelihood of the correct output y125 given the input c121.

[0100] Subsequently, the decision decoder circuit 113 can be configured to compute a structured pruning task by solving Equation 4. In some embodiments, this computation involves finding the parameters θ' of the pruning model given the input c 121 and its optimization by maximizing the likelihood of the output y 125 on data pairs in the dataset T 135. In some embodiments, the computation may involve extending Bayes' theorem to include intermediate decision parameters d 124, which can be used to determine one or more tasks of adaptively pruning the large language model 162 based on the computed optimization. Bayes' theorem can be mathematically expressed as:

[0101] (Equation 5)

[0102] Where p(z|y) refers to the posterior probability of z given y.

[0103] p(y|z) refers to the likelihood of y given z, and p(z) refers to the prior probability of z.

[0104] p(y) refers to the marginal likelihood of evidence y occurring.

[0105] The extension of Bayes' theorem in Formula 5 can be extended and expressed mathematically as follows:

[0106] (Equation 6)

[0107] As described above, the decision decoder circuit 113 can be configured to incorporate additional events represented by intermediate decision parameters d 124, which can occur given the latent variable z 123 and conditions indicated by the input c 121. The intermediate decision parameters d 124 can then facilitate the relationship between the latent variable z 123 and the output y 125, conditioned on the input c 121. In this context, given the latent variable z 123 and the input c 121, the occurrence of the event generating the output y 125 can depend on the intermediate decision parameters d 124, and once the intermediate decision parameters d 124 are known, the latent variable z 123 may not provide additional information about the output y 125. Under such conditional independence, the probability p(y|z, c) can be determined using the definition of conditional probability. (For example, the conditional probability of output y125 given intermediate decision parameters d124 and input c121 (conditions) is multiplied by the conditional probability of intermediate decision parameters d124 given latent variable z123 and conditions given by input c121). Therefore, Equation 6 can be reformulated and mathematically expressed as:

[0108] (Equation 7)

[0109] Given the existence of intermediate decision parameters d 124 and input c 121, the relation defined in Equation 7 applies to the conditional independence between output y 125 and latent variable z 123.

[0110] Still referencing Figure 1B The diagram illustrates how the decision encoder circuit 112 can be configured to implement q(z|y, c) to learn to embed the input c 121 and the output y 125 into the latent space 129; and the decision decoder circuit 113 can be configured to implement p(d|z, c) to generate a structured pruning task conditioned on the input c 121 using the sampled latent variables z 123. Furthermore, the ML model modification circuit 114 can be configured to implement p(y|d, c), where the dynamically pruned large language model 166 uses fewer parameters than the previous large language model 162 to predict the output y 125 (e.g., the output token y) specifically for the input c 121, while maintaining context awareness.

[0111] Still referencing Figure 1B In some embodiments, during training time, the decision encoder circuit 112 uses the input (c, y) to generate the mean μ127 and variance σ of the latent space 129 (e.g., the distribution of latent variables).2 128. Then, using the mean μ127 and variance σ... 2 128 By implementation (e.g., by execution) (It can be described as applying probability sampling to the data distribution of the latent space 129 using the latent variable z 123) to sample the latent variable z 123.

[0112] The decision decoder circuit 113 can use the sampled latent variable z 123 and the input c 121 (e.g., input data) to compute the pruning probability p(h|c, z; θD) associated with each structural component in the base model M (e.g., large language model 162). A cluster of neurons with the K highest probabilities can be selected based on the pruning ratio s to configure a temporarily reduced model parameter θ' = θ\θp. A prediction can then be generated using the pruned large language model 166 with parameter θ', which produces the output y 125 of the pruned large language model 166, while considering the mediating (e.g., intermediate) events represented by the intermediate decision parameter d 124 under conditional independence. The updated pruned parameter θ' can then be compared with the unpruned parameter θ. cp The data is then merged back to update the complete large language model 162 for the next iteration. This process, implemented by the decision encoder circuit 112 and the decision decoder circuit 113, can be repeated iteratively, e.g., for each batch across multiple rounds (N), gradually refining the parameters θ. Thus, as disclosed herein, the VG-ML model processor 110 can be configured to determine one or more structured pruning tasks for the large language model 162, which can have some dependencies on the latent variable z123, the intermediate decision parameter d 124, and the input c 121 in a way that introduces intermediate events (e.g., intermediate decision parameters d) to create an indirect dependency between the latent variable z 123 and the output y 125, which will be discussed in further detail below and in Figure 2 As shown in the image.

[0113] exist Figure 1BIn an illustrative example, a large language model 162 processed by a VG-ML model processor 110 can receive an input c 121 (or prompt) shown as input text 142 (e.g., “What is a poodle?”) to be fed into the large language model 162. As a result of the generation process, the pruned large language model 166 can generate new and / or different types of content that can be produced as output y 125 from the VG-ML model processor 110, which can be presented in a user-understandable form. For example, the content may include text (e.g., conversational text), images, videos, etc. According to the variational generation aspect disclosed herein, the decision encoder circuit 112 can capture the contextual meaning of the text of the input prompt by learning to embed the input c 121 and output y 125 (e.g., output token y) into a latent space 129. The decision decoder circuit 113 can use sampled latent variables z 123 to generate a structured pruning task conditioned on the input c 121 and intermediate decision parameters d 124 (determined using the created latent space).

[0114] Therefore, the ML model modification circuit 114 can dynamically prune previous instances of the large language model 162 based on intermediate decision parameters d 124 (e.g., removing multiple parameters from the model's neural network configuration) to transform the large language model 162 (e.g., a first machine learning model) into a pruned large language model 166 (e.g., a modified version of the first machine learning model), which, for example, reduces size and / or complexity through a structured pruning task and specifically predicts output y 125 for input c 121, while having reduced computational overhead and maintaining context awareness. For example, the intermediate decision parameters d 124 can represent intermediate factors, such as indicating that the predicted output for an input cue including the token "poodle" should have a greater probability of generating an output token that is context-dependent with the contextual text 141 (e.g., "miniature poodles") (e.g., intermediate decision parameters d). Subsequently, the VG-ML model processor 110 can leverage this context awareness to dynamically prune the large language model 162, adjusting the model in a way that more effectively controls the generation process to be tuned to intermediate factors (e.g., a greater likelihood of the user intent being "miniature poodle" rather than "royal / giant poodle"). As a result, the pruned large language model 166 can generate output y 125 as new response text 144 to be provided to the user (e.g., "a poodle is a small dog…"). Figure 1BAs shown, the VG-ML model processor 110 can be configured to generate a pruned large language model 166 that predicts output y 125 (for input c 121), which has been conditionally adapted to intermediate decision parameters d 124 in a manner that introduces intermediate factors and creates indirect dependencies between latent variables z 123 and output y 125, resulting in a more flexible generation process for the pruned large language model 166.

[0115] Figure 2 It is a description of some embodiments according to this disclosure and by Figure 1B A diagram illustrating the relationships related to intermediate decisions implemented by the VG-ML model processor.

[0116] refer to Figure 2 Figure 200 illustrates a VG-ML model processor 110 that can, for example, implement aspects of variational generation according to some embodiments of this disclosure. Figure 1A The relationships and / or dependencies between various variables generated are shown in Figure 200. Figure 200 depicts several variables that can be associated with the processing of the machine learning model, including the input c121 (which can correspond to...). Figure 1A The inputs c 121), latent variables 123, intermediate decision parameters d 124, and output y 125 are represented as nodes in the curve graph 200, and the relationships and / or dependencies between them are represented by edges (e.g., lines connecting the nodes). Figure 2An intermediary factor, namely the intermediate decision parameter d124, is shown to be introduced into the process, resulting in a layer of flexibility between the latent variable z123 and the output y125. Variational inference can be achieved by modeling p(d|z, c), which creates indirect relationships (e.g., connected via the intermediate decision parameter d124) between the latent variable z123 and the output y125, and between the input c121 and the output y125. In some embodiments, during the training of the machine learning model, a process (e.g., a generative process) can learn to embed the input c121 and the output y125 into a latent space, and sample that latent space using q(z|y, c) with the latent variable z123, generating a learned relationship between the latent variable z123 and the output y125. Given the latent variable z123, this relationship can then be used to predict the intermediate decision parameter d124. Therefore, given the intermediate decision parameter d 124 and the input c 121, the process can implement conditional independence between the latent variable z 123 and the output y 125. Subsequently, the intermediate decision parameter d 124 can be applied to the final output y 125, as plot 200 shows that the output y 125 indirectly depends on the latent variable z 123 and is conditional on the input c 121 and the intermediate decision parameter d 124 by modeling p(y|d, c). Plot 200 illustrates the indirect relationship between the latent variable z 123 and the output y 125 (e.g., connected via the intermediate decision parameter d 124), which has an additional layer of flexibility, allowing the latent variable z 123 to influence both the intermediate decision parameter d 124 and the output y 125.

[0117] Figure 3 This is a flowchart depicting the operation of a method 3000 for dynamically modifying a machine learning model conditioned on intermediate decision parameters, according to some embodiments of the present disclosure.

[0118] The processor can embed the inputs and outputs of a machine learning model into a latent space (operation 3002). For example, during the training of a machine learning model, input data representing conditions (e.g., input hints in the form of text, images, audio, video, code, etc.) and the outputs generated by predictions from the ML model can be embedded into the latent space. A data distribution can be defined by creating relationships between the model's inputs and outputs within the latent space. In some embodiments, the relationship between input conditions and outputs associated with the embeddings in the latent space can be represented as q(z|y, c), and the data distribution can be defined by the mean μ and variance σ. 2Definition. Method 3000 can adaptively capture the characteristics of each input token by generating embeddings, enabling it to learn latent representations of controlled and diverse outputs conditioned on specific input variables. Therefore, given the conditions associated with the input, the embeddings in the latent space can model the dependencies between latent variables z and data points y. In some embodiments, the machine learning model can be a large language model for supporting AI-based language processing applications, the input can be a cue in text, and the output can be an output token.

[0119] In some embodiments, the processor may determine intermediate decision parameters d based on sampling of latent variables z in a defined latent space (operation 3004). Method 3000 may introduce intermediate events by determining intermediate decision parameters d, which ultimately adapt to the output generated by the machine learning model. Considering intermediate events during the generation, training, and / or utilization of the ML model allows for the addition of mediating factors that can be used to infer the indirect relationship between the latent variables z and the output y, including an additional layer of flexibility provided by the intermediate decision parameters d (e.g., intermediate factors and / or events). For example, intermediate decision parameters d may be determined by variational inference based on latent variables z (e.g., sampling of z in the latent space) and input c. In some embodiments, probabilistic sampling of the latent space may be performed using latent variables z by modeling p(d|z, c). Given latent variables z and input c, the model may then predict intermediate decision parameters d and may ultimately incorporate intermediate control factors that can play a role (e.g., a key role) in the generation process.

[0120] In some embodiments, the processor may determine at least one dynamic and / or structured modification task based on intermediate decision parameters d (operation 3006). Method 3000 may apply the dynamic and / or structured modification task to the ML model while generating, training, and / or utilizing the model, such that the output y from the ML model is generated conditioned on the intermediate decision parameters d, the latent variable z, and the input c. In some embodiments, method 3000 may be used to train a large-scale generative AI model. The already determined (previous operation 3004) intermediate decision parameters d may then be used to generate the dynamic and / or structured modification task, which is applied to the ML model to dynamically modify and / or adapt the generative process of the ML model to predict the output with some dependency on the intermediate decision parameters d. The dynamic and / or structured modification task may involve one or more rules that govern a strategy for dynamically modifying the configuration and / or generative process of the ML model in a structured, organized, and / or mathematically defined manner with respect to the intermediate decision parameters d. Therefore, method 3000 enables the intermediate factor, represented by the intermediate decision parameter d, to dynamically modify the inference relationship between the input c and output y predicted by the model by generating dynamic and / or structured modification tasks to influence the generation process and the final prediction inferred by the ML model. As previously described, method 3000 can be used to process large language models. Thus, in some embodiments, the dynamic and / or structured modification task can be a structured pruning task dynamically applied to a large language model during the training and / or inference phases of the model. Examples of dynamic and / or structured modification tasks may include, but are not limited to: pruning (e.g., unstructured pruning); multimodal output selection (e.g., the output modality can be determined based on conditions and intermediate factors); expert mixing (e.g., selecting one or more experts to contribute to the ML model output); and / or the like.

[0121] In some embodiments, the processor may apply a dynamic and / or structured modification task to the ML model (operation 3008). Method 3000 may involve dynamically modifying and / or adapting the configuration and / or generation process of the ML model according to rules defining a strategy for the dynamic and / or structured modification task. According to embodiments, applying the dynamic and / or structured modification task to the ML model may be included in the training and / or inference phases of the ML model. For example, the rules for the dynamic and / or structured modification task may be dynamically integrated into one or more iterations of the ML model training process in a manner that adapts the model using a structured, organized, and / or mathematically defined relationship with intermediate decision parameters d. In some embodiments, the result of applying the dynamic and / or structured modification task is to generate a modified ML model with a different neural network configuration and / or inference process than a previous ML model (e.g., an ML model trained based on previous training iterations).

[0122] The existence of several embodiments may include processing large language models, where applying dynamic and / or structured modification tasks involves performing structured pruning tasks to dynamically prune the large language model. The structured pruning task can reduce, minimize, and / or optimize the size and / or complexity of a large-scale model based on intermediate decision parameters d, while maintaining context awareness. For example, applying a structured pruning task to a large language model may involve removing one or more parameters, heads, weights, etc., from the model to dynamically adapt the neural network configuration of the large language model in a manner dependent on the intermediate decision parameters d. By applying dynamic and / or structured modification tasks to ML models, method 3000 can generate structured and flexible generative processes that are widely applicable across a wide range of AI- and / or ML-based applications involving the processing of complex and high-dimensional data that can be generated with diversity and specificity. Applications may include: natural language processing; image generation; multimodal generation; autonomous systems; and / or the like. Thus, the modified ML model using method 3000 for dynamic adaptation can receive input (or prompts), such as text, images, audio, video, and code, to generate new and / or different types of content from the input, such as text and chat, images, code, video, and / or embeddings that have been conditionally adapted to the intermediate decision parameter d.

[0123] Figure 4 This is a flowchart depicting the operation of a method 4000 for modifying a machine learning model according to some embodiments of the present disclosure.

[0124] refer to Figure 4 Method 4000 may include one or more of the following operations. Machine learning model 122 (see...) Figure 1A The processor 104 can perform a generation process to predict (e.g., estimate) a first output (operation 4001). The processor 104 can generate a latent space 129 based on the input c 121 to the machine learning model 122 (operation 4002). The processor 104 can determine intermediate decision parameters d 124 based on the latent space 129 and on the latent variable Z 123 (operation 4003). Based on the intermediate decision parameters d 124, the processor 104 can modify the structure of the machine learning model 122 to generate a modified machine learning model 126 to perform a modified generation process conditioned on (e.g., under the condition of) the intermediate decision parameters d 124 (operation 4004). The modified machine learning model 126 can generate a second output y 125 that includes content associated with the input c 121 (operation 4005).

[0125] Therefore, aspects of some embodiments of this disclosure can provide improvements to AI by utilizing variational generation aspects in the processing of machine learning models to achieve: efficient processing; high computational efficiency; reduced latency; reduced power consumption; enhanced flexibility and interpretability; improved accuracy and / or generation quality (e.g., by explicitly modeling intermediate factors affecting the data generation process); and / or similar results. Furthermore, aspects of some embodiments of this disclosure can provide additional structures for improving machine learning models.

[0126] The exemplary embodiments of this disclosure can be extended to the following statements without limitation:

[0127] Statement 1. An example method includes: performing a generation process by a machine learning model to predict a first output, generating a latent space via a processor based on inputs to the machine learning model, determining intermediate decision parameters via the processor based on the latent space, modifying the structure of the machine learning model via the processor based on the intermediate decision parameters to generate a modified machine learning model to perform a modified generation process conditioned on the intermediate decision parameters, and generating a second output by the modified machine learning model including content associated with the inputs.

[0128] Statement 2. An example method includes the method described according to Statement 1, wherein the second output is conditioned on the latent variables and intermediate decision parameters.

[0129] Statement 3. An example method includes the method according to any one of Statements 1 and 2, wherein generating the latent space includes embedding the input and the previously generated output from the machine learning model into the data distribution.

[0130] Statement 4. An example method includes the method described according to Statement 3, wherein intermediate decision parameters are determined based on the input and the relationship inferred from the embedding.

[0131] Statement 5. An example method includes the method according to any one of Statements 1-4, wherein determining intermediate decision parameters includes sampling the latent space based on latent variables.

[0132] Statement 6. An example method includes the method according to any one of Statements 1-5, wherein changing the structure of the machine learning model includes modifying the generation process to infer the indirect relationship between the latent variables and the second output.

[0133] Statement 7. An example method includes the method according to any one of Statements 1-6, wherein the machine learning model is a large language model.

[0134] Statement 8. An example method includes the method according to any one of Statements 1-7, wherein changing the structure of a machine learning model includes performing a structured pruning task on a large language model.

[0135] Statement 9. An example method includes the method according to any one of Statements 1-8, wherein changing the structure of a machine learning model includes applying a structured pruning task to dynamically prune a large language model using rules conditioned on intermediate decision parameters.

[0136] Statement 10. An example method includes the method described in Statement 9, wherein a structured pruning task is applied to dynamically remove at least one of parameters, heads, nodes, edges, or weights from a large language model.

[0137] Statement 11. An example method includes the method according to any one of Statements 9 and 10, wherein a structured pruning task is applied to generate a pruned large language model that is reduced in size from a large language model, and an output is generated conditioned on intermediate decision parameters.

[0138] Statement 12. An example method includes the method according to any one of Statements 8-11, wherein the structured pruning task includes rules conditioned on intermediate decision parameters and latent variables.

[0139] Statement 13. An example method includes the method according to any one of Statements 1-12, and further includes determining a second intermediate decision parameter based on a second latent variable and a second input, and generating a structured modification task conditioned on the second intermediate decision parameter.

[0140] Statement 14. An example method includes the method according to any one of Statements 1-13, wherein the content includes automatically generated images, text, audio, and video based on input.

[0141] Statement 15. An example system for performing the method according to any one of Statements 1-14 includes processing circuitry and a memory storing instructions which, when executed by the processing circuitry, cause the processing circuitry to perform the method according to any one of Statements 1-14.

[0142] Statement 16. An example device for performing the method according to any one of statements 1-14 includes one or more processors configured to perform the method according to any one of statements 1-14.

[0143] While embodiments of the present disclosure have been specifically shown and described with reference to the examples described herein, those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of the disclosure as set forth in the appended claims and their equivalents.

Claims

1. A method for modifying a machine learning model, the method comprising: The generation process is performed by a machine learning model to predict the first output; The latent space is generated by the processor based on the input to the machine learning model; The processor determines the first intermediate decision parameters based on the potential space. Based on the first intermediate decision parameters, the structure of the machine learning model is changed by the processor to generate a modified machine learning model to perform a generation process that modifies the model based on the first intermediate decision parameters. as well as The modified machine learning model generates a second output that includes content associated with the input.

2. The method according to claim 1, wherein, The second output is conditional on the first latent variable and the first intermediate decision parameter.

3. The method according to claim 1, wherein, Generating the latent space involves embedding the input and the output previously generated from the machine learning model into a data distribution.

4. The method according to claim 3, wherein, The first intermediate decision parameter is determined based on the input and the relationship inferred from the embedding.

5. The method according to claim 3, wherein, Determining the first intermediate decision parameter includes sampling the latent space based on the first latent variable.

6. The method according to claim 5, wherein, Changing the structure of the machine learning model includes modifying the generation process to infer the indirect relationship between the first latent variable and the second output.

7. The method according to claim 6, wherein, The machine learning model is a large-scale language model.

8. The method according to claim 7, wherein, Changing the structure of the machine learning model includes performing a structured pruning task on the large language model.

9. The method according to claim 8, wherein, Changing the structure of the machine learning model includes applying the structured pruning task to dynamically prune the large language model using rules conditioned on the first intermediate decision parameter.

10. The method according to claim 9, wherein, The structured pruning task is applied to dynamically remove at least one of parameters, heads, nodes, edges, or weights from the large language model.

11. The method according to claim 10, wherein, The structured pruning task is applied to generate a pruned large language model that is reduced in size from the large language model, and a third output is generated conditioned on the first intermediate decision parameter.

12. The method according to claim 11, wherein, The structured pruning task includes rules conditioned on the first intermediate decision parameter and the first latent variable.

13. The method of claim 12, further comprising: The second intermediate decision parameter is determined based on the second latent variable and the second input. as well as Generate a structured modification task based on the second intermediate decision parameter.

14. The method according to claim 1, wherein, The content includes automatically generated images, text, audio, and video based on the input.

15. An apparatus comprising: One or more processors are configured to execute: The generation process uses a machine learning model to predict the first output; The latent space is generated based on the input to the machine learning model; Intermediate decision parameters are determined based on the potential space; Based on the intermediate decision parameters, the structure of the machine learning model is changed to generate a modified machine learning model to perform a generation process that modifies the model based on the intermediate decision parameters. and Based on the modified machine learning model, a second output is generated that includes content associated with the input.

16. The device according to claim 15, wherein, The one or more processors are configured to execute the generation of the second output conditioned on the intermediate decision parameters.

17. The device according to claim 15, wherein, The one or more processors are also configured to generate the latent space by embedding the input and previously generated output from the machine learning model into a data distribution.

18. The device according to claim 17, wherein, The one or more processors are also configured to perform the determination of the intermediate decision parameters by sampling the latent space based on latent variables.

19. The device according to claim 15, wherein, The one or more processors are also configured to perform a structured pruning task to alter the structure of the machine learning model.

20. A system comprising: Processing circuitry; as well as The memory stores instructions, wherein the instructions are based on the processing circuit being executed by the processing circuit to cause the processing circuit to perform: The generation process uses a machine learning model to predict the first output; The latent space is generated based on the input to the machine learning model; Intermediate decision parameters are determined based on the potential space; Based on the intermediate decision parameters, the structure of the machine learning model is changed to generate a modified machine learning model to perform a generation process that modifies the model based on the intermediate decision parameters. and Based on the modified machine learning model, a second output is generated that includes content associated with the input.

Images