Method, computer system, and computer program (autoregressive generation using a perfect probability distribution)

Abstract

This invention provides a method, system, and program for considering the probability of discarding during subsequent iterations of an autoregressive generation process. [Solution] The method includes performing an autoregressive generation process of a generative machine learning model by generating predictions of a first response token represented by a first probability distribution over V values, where V represents a number greater than 1 of tokens in a predefined dominant set of tokens. The method also includes feeding back the first probability distribution as input to the generative machine learning model for the next iteration of the autoregressive generation process, and in the next iteration generating a next response token represented by a next probability distribution, and performing an output from the generative machine learning model, the output being based on a probability distribution including the first and next probability distributions.

Description

Background Art

[0001] Autoregressive generation is an iterative process for generating data sequences, such as text sequences, image sequences, audio sequences, etc., where each new data element within the sequence is predicted based on previous data elements that the process has already predicted. Autoregressive generation is often performed by machine learning models such as large language models (LLMs).

Summary of the Invention

Problems to be Solved by the Invention

[0002] During subsequent iterations of the autoregressive generation process, the probability of rejection cannot be taken into account.

Means for Solving the Problems

[0003] One exemplary embodiment includes performing an autoregressive generation process of a generative machine learning model by causing a prediction of a first response token represented by a first probability distribution over V values, where V represents a number greater than one of the tokens within a predefined governing set of tokens, where the first probability distribution is fed back as input to the generative machine learning model for a next iteration of the autoregressive generation process, the next iteration causes a next response token represented by a next probability distribution, and the performing step causes an output from the generative machine learning model, the output being based on a probability distribution including the first probability distribution and the next probability distribution, thereby providing a method.

[0004] Another exemplary embodiment provides a computer system comprising: a set of processors; one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media, the program instructions causing the set of processors to perform one or more of the following actions: a procedure for performing an autoregressive generation process of a generative machine learning model, where V represents a number greater than 1 of tokens in a predefined dominant set of tokens, by causing the set of processors to produce predictions of first answer tokens represented by a first probability distribution over V values, where the first probability distribution is fed back as input to the generative machine learning model for the next iteration of the autoregressive generation process, the next iteration to produce a next answer token represented by a next probability distribution, the procedure performed to produce an output from the generative machine learning model, the output being based on a probability distribution including the first and next probability distributions.

[0005] Further exemplary embodiments provide a computer program product comprising one or more computer-readable storage media; and program instructions stored on the one or more computer-readable storage media, the program instructions for performing an operation which may have one or more steps of performing an autoregressive generation process of a generative machine learning model by producing a prediction of a first answer token represented by a first probability distribution over V values, where V represents a number greater than 1 of tokens in a predefined dominant set of tokens, wherein the first probability distribution is fed back as input to the generative machine learning model for the next iteration of the autoregressive generation process, the next iteration producing a next answer token represented by a next probability distribution, the steps performed producing an output from the generative machine learning model, the output being based on a probability distribution including the first probability distribution and the next probability distribution. [Brief explanation of the drawing]

[0006] [Figure 1]This figure illustrates a computing environment according to one embodiment of this solution.

[0007] [Figure 2] This diagram illustrates the process by which a machine learning model performs an autoregressive generation process, based on examples and features of this solution.

[0008] [Figure 3A] This figure shows the process of generating a probability distribution, based on examples and features of this solution.

[0009] [Figure 3B] This figure shows the iterative process of the autoregressive generation process, with examples and features of this solution.

[0010] [Figure 3C] This figure shows the process of the next iteration of the autoregressive generation process, based on the examples and features of this solution.

[0011] [Figure 3D] This figure shows the process of selecting tokens for output by a machine learning model, based on examples and features of this solution.

[0012] [Figure 4A] This flowchart illustrates the method based on examples and features of this solution.

[0013] [Figure 4B] This flowchart illustrates the method with additional examples and features of this solution.

[0014] [Figure 4C] This flowchart illustrates the method with further additional examples and features of this solution.

[0015] [Figure 4D] This flowchart illustrates the method with yet another additional example and features of this solution.

[0016] [Figure 5A] A system diagram showing the integration of an AI model into any decision-making point according to an example and features of this solution.

[0017] [Figure 5B] A diagram showing a process for developing an AI model to support an AI-assisted computer decision-making point according to an example and features of this solution.

[0018] [Figure 5C] A diagram showing a process for using an AI model to support an AI-assisted computer decision-making point according to an example and features of this solution.

Mode for Carrying Out the Invention

[0019] This disclosure includes a detailed description regarding cloud computing, but it should be understood that the implementation of the teachings recited herein is not limited to a cloud computing environment. Rather, embodiments of this solution can be implemented in conjunction with any other type of computing environment known currently or developed in the future.

[0020] According to an exemplary embodiment, a method is provided comprising the steps of: performing an iteration of an autoregressive generation process of a machine learning model over a probability distribution over V values ​​to generate an additional probability distribution for the V values, where V represents the number of tokens in a predefined dominant set of tokens; selecting a value with the highest probability from a first probability distribution for the V values ​​among the probability distributions for the V values; selecting a second value with the highest probability from a second probability distribution for the V values ​​among the probability distributions for the V values; converting the first and second values ​​into a first output token and a second output token, respectively, based on a predefined dominant set of tokens; and generating an output from a machine learning model having a sequence of data including the first output token and the second output token. One of the technical advantages of this system is that, during each iteration of the autoregressive generation process, the complete probability distribution for the tokens in the dominant set of tokens is fed back. By retaining all probability distributions, subsequent iterations of the autoregressive generation process can explore more answers and yield more accurate responses.

[0021] In some embodiments, the method may further comprise the steps of receiving a set of input tokens, generating a set of probability distributions for the set of input tokens, where each probability distribution in the set includes a one-hot coded representation of each input token for V tokens in a predefined dominant set of tokens. The technical advantage of the method is that one-hot coding is used during the first iteration of the autoregressive generation process to adapt the input data into a format that can be processed as assets of probability distributions.

[0022] In some embodiments, the method may further comprise a step of embedding probability distributions for V values ​​into the embeddings based on a linear embedding layer, the linear embedding layer mapping the probability distributions for the V values ​​into embeddings for E values, where E is a different value from V. The V values ​​of the probability distribution are of a different size than the E values ​​of the embeddings. The technical advantage of this step is that the size of the input data is modified to fit the next step of the autoregressive generation process.

[0023] In some embodiments, the method may further comprise a step of transforming the embedding via a transformer stack into a feature embedding that includes additional feature embeddings corresponding to predicted response tokens. The technical advantage of this step is that it predicts tokens (part of the response) based on multiple embeddings rather than just a single embedding, thereby providing the system with more data and consequently improving accuracy.

[0024] In some embodiments, the method may further comprise the steps of: transforming the additional feature embedding into an additional raw probability distribution over V tokens from a dominant set of predefined tokens, based on a linear layer; and performing a softmax function on the additional raw probability distribution over the V tokens to generate an additional probability distribution for V values. The technical effect of this feature is that the softmax function ensures that the sum of the probability distributions for any iteration of the autoregressive generation process is a predefined number (e.g., 1).

[0025] In some embodiments, the method may further comprise a step of generating further additional probability distributions for V values ​​by performing subsequent iterations of the autoregressive generation process on probability distributions for V values ​​and additional probability distributions for V values. The technical advantage of this step is that, while the related process discards the non-maximal probability distribution among the group, it generates additional probability distributions using the entire probability distribution during the autoregressive generation process.

[0026] In some embodiments, the method may further comprise the steps of selecting a third value with the highest probability from an additional probability distribution for V values, and transforming the third value into a third output token based on a dominant set of predefined tokens. The technical effect of this step is that the answer can be revealed finally rather than stepwise, because not only the token with the highest probability but also the entire probability distribution is preserved. Thus, the process generates an answer using more data, resulting in improved accuracy.

[0027] According to an exemplary embodiment, a computer system is provided comprising a set of processors, a set of one or more computer-readable storage media, and program instructions collectively stored in the set of one or more storage media, wherein the program instructions cause the set of processors to perform an iteration of an autoregressive generation process of a machine learning model for a probability distribution over V values ​​to generate an additional probability distribution for the V values, where V represents the number of tokens in a predefined dominant set of tokens; a step of selecting a value with the highest probability from a first probability distribution for the V values ​​among the probability distributions for the V values; a step of selecting a second value with the highest probability from a second probability distribution for the V values ​​among the probability distributions for the V values; a step of converting the first and second values ​​into a first output token and a second output token, respectively, based on a predefined dominant set of tokens; and a step of generating an output from a machine learning model having a sequence of data including the first output token and the second output token. One of the technical advantages of the system is that, during each iteration of the autoregressive generation process, the complete probability distribution for the tokens in the dominant set of tokens is fed back. By retaining all probability distributions, subsequent iterations of the autoregressive generation process can explore more answers, leading to more accurate responses.

[0028] In some embodiments, the processor set may perform an operation comprising the steps of receiving a set of input tokens, generating a set of probability distributions for the set of input tokens, where each probability distribution in the set includes a one-hot coded representation of each input token for V tokens in a predefined dominant set of tokens. The technical advantage of this method is that one-hot coding is used during the first iteration of the autoregressive generation process to adapt the input data into a format that can be processed as assets of probability distributions.

[0029] In some embodiments, the processor set may perform an operation having a procedure of embedding probability distributions for V values ​​into each embedding based on a linear embedding layer, the linear embedding layer mapping the probability distributions for the V values ​​into embeddings for E values, where E is a different value from V. The V values ​​of the probability distribution are of a different size than the E values ​​of the embedding. The technical advantage of this step is that the size of the input data is thereby modified to fit the next step of the autoregressive generation process.

[0030] In some embodiments, the processor set may perform an operation that, via a transformer stack, has a procedure for transforming the embedding into a feature embedding that includes additional feature embeddings corresponding to predicted response tokens. The technical advantage of this step is that it predicts tokens (part of the response) based on multiple embeddings, not just a single embedding, thereby providing the system with more data and consequently improving accuracy.

[0031] In some embodiments, the processor set may perform operations that include, based on a linear layer, transform the additional feature embedding into an additional raw probability distribution over V tokens from a dominant set of predefined tokens, and then run a softmax function on the additional raw probability distribution over the V tokens to generate an additional probability distribution for V values. The technical effect of this feature is that the softmax function ensures that the sum of the probability distributions for any iteration of the autoregressive generation process is a predefined number (e.g., 1).

[0032] In some embodiments, the processor set may perform an operation that involves performing subsequent iterations of the autoregressive generation process on probability distributions for V values ​​and additional probability distributions for V values ​​to generate further additional probability distributions for V values. The technical advantage of this step is that while the related process discards the non-maximum probability distribution in the group, it generates additional probability distributions using the entire probability distribution during the autoregressive generation process.

[0033] In some embodiments, the processor set may perform an operation comprising the steps of selecting a third value with the highest probability from an additional probability distribution for V values, and converting the third value to a third output token based on a dominant set of predefined tokens. The technical effect of this step is that the answer can be revealed finally rather than stepwise, because the entire probability distribution is retained, not just the token with the highest probability. Thus, the process generates an answer using more data, resulting in improved accuracy.

[0034] According to an exemplary embodiment, a computer program product is provided, comprising a set of one or more computer-readable storage media and program instructions collectively stored in the set of one or more computer-readable storage media, wherein the program instructions cause a set of processors to perform a computer operation comprising: a procedure to perform an iteration of an autoregressive generation process of a machine learning model for a probability distribution over V values ​​to generate an additional probability distribution for the V values, where V represents the number of tokens in a predefined dominant set of tokens; a procedure to select a value with the highest probability from a first probability distribution for the V values ​​among the probability distributions for the V values; a procedure to select a second value with the highest probability from a second probability distribution for the V values ​​among the probability distributions for the V values; a procedure to convert the first and second values ​​into a first output token and a second output token, respectively, based on a predefined dominant set of tokens; and a procedure to generate output from a machine learning model having a sequence of data including the first output token and the second output token. One of the technical advantages of this system is that, during each iteration of the autoregressive generation process, the complete probability distribution for the tokens in the dominant set of tokens is fed back. By retaining all probability distributions, subsequent iterations of the autoregressive generation process can explore more answers, leading to more accurate responses.

[0035] In some embodiments, the computer operation may further include a step of receiving a set of input tokens, a step of generating a set of probability distributions for the set of input tokens, where each probability distribution in the set includes a one-hot coded representation of each input token for V tokens in a predefined dominant set of tokens. The technical advantage of this method is that one-hot coding is used during the first iteration of the autoregressive generation process to adapt the input data into a format that can be processed as assets of probability distributions.

[0036] In some embodiments, the computer operation may further include a step of embedding probability distributions for V values ​​into the embeddings based on a linear embedding layer, which maps the probability distributions for the V values ​​into embeddings for E values, where E is a different value from V. The V values ​​of the probability distribution are of a different size than the E values ​​of the embeddings. The technical advantage of this step is that the size of the input data is thus modified to fit the next stage of the autoregressive generation process.

[0037] In some embodiments, the computer operation may further include a step via a transformer stack to transform the embedding into a feature embedding that includes additional feature embeddings corresponding to predicted response tokens. The technical advantage of this step is that it predicts tokens (part of the response) based on multiple embeddings rather than just a single embedding, thereby providing the system with more data and consequently improving accuracy.

[0038] In some embodiments, the computer operation may further include steps of transforming the additional feature embedding into an additional raw probability distribution over V tokens from a dominant set of predefined tokens, based on a linear layer, and running a softmax function on the additional raw probability distribution over the V tokens to generate an additional probability distribution for V values. The technical effect of this feature is that the softmax function ensures that the sum of the probability distributions for any iteration of the autoregressive generation process is a predefined number (e.g., 1).

[0039] In some embodiments, the computer operation may further include a step of generating further additional probability distributions for V values ​​by performing subsequent iterations of the autoregressive generation process on the probability distributions for V values ​​and additional probability distributions for V values. The technical advantage of this step is that, while the related process discards the probability distribution that is not the largest among the group, it generates additional probability distributions using the entire probability distribution during the autoregressive generation process.

[0040] In some embodiments, the computer operation may further include steps for selecting a third value with the highest probability from an additional probability distribution for V values, and for transforming the third value into a third output token based on a dominant set of predefined tokens. The technical effect of this step is that the answer can be revealed finally rather than stepwise, because not only the token with the highest probability but also the entire probability distribution is preserved. Thus, the process generates an answer using more data, resulting in improved accuracy.

[0041] Autoregressive generation is a recursive / iterative process for predicting a sequence of data elements, referred to herein as tokens. For example, given a sequence of tokens from 1 to N, an autoregressive generation process can predict N+1 tokens. Multiple sampling methods can be used for autoregressive generation, including but not limited to greedy decoding, beam search, top-k sampling, random sampling, nuclear sampling or top-p sampling, temperature sampling, and similar methods. One common implementation of the autoregressive generation process utilizes transformer-based neural networks such as LLMs.

[0042] Generative machine learning models often use tokens from a dominant set of tokens as input. For example, a transformer model might assume an index vocabulary as input. The index vocabulary may refer to a vocabulary of data elements, which can be extensive (e.g., hundreds of data elements, thousands of data elements, etc.). A generative model that generates speech may use a dominant set of sounds and / or notes as the basis for its speech generation, with each specific sound or note being assigned to a respective token. A generative model that generates images may use a dominant set of colors, lines, shapes, and / or object outlines as the basis for its image generation, with each specific color, line, shape, etc. being assigned to a respective token. In the case of an autoregressive task that generates words in a language, the vocabulary includes multiple words from one or more languages, for example, one or more words in English and / or one or more words in Spanish, etc. Throughout this disclosure, references to vocabulary are examples of the terminology of a dominant set of tokens. Traditionally, autoregressive generative processes use one of the sampling methods mentioned above to identify a small portion of the data elements (e.g., 1, 3, 5, <=, cr, etc.) as potential matches and discard the remaining data elements from the input to the autoregressive generative process. For example, this model can accept input and generate probabilities for each data element in the vocabulary, which are referred to herein as probability distributions.

[0043] The probability of each data element represents the likelihood that each data element is a solution. This model then selects a few data elements with the highest probability and discards the remaining data elements from consideration during the autoregressive generation process. However, by discarding most of the probabilities, the system loses a considerable amount of information that could lead to better answers in subsequent iterations. Furthermore, the discarded probabilities cannot be considered during subsequent iterations of the autoregressive generation process.

[0044] An exemplary embodiment concerns a novel autoregressive generative process that does not discard probabilities, but instead relies on the complete probability distribution of all elements in the vocabulary for each iteration of the process. This novel autoregressive generative process can be performed by an LLM. For example, the model may use the entire probability distribution as input to the process, rather than selecting a small number of data elements from an initial probability distribution, and then consider all data elements in the vocabulary with respect to the predicted token. Furthermore, the result of the autoregressive generative process is an additional probability distribution for the next token in the sequence, which includes an additional probability distribution across the entire vocabulary of data elements. This process is repeated iteratively until all tokens in the sequence have been predicted. In each iteration, the entire probability distribution from all previous iterations is preserved, allowing the autoregressive generative process to consider far more data and possible answers.

[0045] Some of the advantages of the exemplary embodiment include improved accuracy of machine learning models performing autoregressive generative processes. By preserving the full probability distribution for the vocabulary, the process can consider more information in each iteration and identify possible solutions that would normally have been discarded. In addition, since the full / perfect probability distribution is preserved and used in each subsequent iteration, no information is lost throughout each iteration of the process, enabling improved accuracy. Furthermore, models that accept and generate a perfect probability distribution can be combined to form cycle GANs, for example, without disrupting the gradient flow in backpropagation by avoiding non-differentiable operations such as "max". Thus, this embodiment also includes technical improvements to training generative machine learning models by improving gradient propagation and subsequent training of the generative model. Gradient propagation can be improved by computing the loss on the softmax of the logit for the one-hot encoded golden index.

[0046] The features, structures, or characteristics described throughout this specification may be combined or omitted in any preferred manner in one or more embodiments. For example, the use of the phrases “exemplary embodiments,” “some embodiments,” or other similar language means that certain features, structures, or characteristics described throughout this specification in relation to embodiments may be included in at least one embodiment. Therefore, the appearance of the phrases “exemplary embodiments,” “some embodiments,” “other embodiments,” or other similar language throughout this specification does not necessarily refer to the same group of embodiments, and the features, structures, or characteristics described may be combined or omitted in any preferred manner in one or more embodiments. Furthermore, any connection between elements in the drawings may allow unidirectional and / or bidirectional communication, even if the indicated connection is a unidirectional or bidirectional arrow. Also, any device shown in the drawings may be a different device. For example, if it is shown that a mobile device is transmitting information, it may also be possible to transmit information using a wired device.

[0047] Various aspects of this disclosure are described by explanatory text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of computer program products (CPPs). With respect to any flowchart, depending on the technology involved, operations may be performed in a different order than those shown in a given flowchart. For example, also depending on the technology involved, two operations shown in consecutive blocks of a flowchart may be performed in reverse order, as a single integrated stage, simultaneously, or with at least partial time overlap.

[0048] Embodiments of a computer program product ("CPP Embodiment" or "CPP") are terms used in this disclosure to describe any set of one or more storage media (also called "mediums") that collectively comprise a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. "Storage device" is any tangible device capable of holding and storing instructions for use by a computer processor. Computer-readable storage media may, but are not limited to, electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any preferred combination thereof. Some known types of storage devices, including these media, include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices (such as pits / lands formed on the main surface of a punch card or disk), or any suitable combination of those described above. When the term "computer-readable storage medium" is used in this disclosure, it shall not be interpreted as storage in the form of a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses passing through optical fiber cables, electrical signals communicated through wires, and / or other transmission media.As will be understood by those skilled in the art, data is typically moved at several intermittent points during the normal operation of a storage device, such as during access, defragmentation, or garbage collection; however, since data is not transient while it is stored, this does not mean that the storage device is transient.

[0049] Referring to Figure 1, the computing environment 100 includes an example of an environment for executing at least a portion of the computer code involved in performing the method of the present invention, such as an autoregressive generative system (block 200) using a perfect probability distribution. In addition to block 200, the computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, the computer 101 has a processor set 110 (including processing circuits 120 and a cache 121), a communication fabric 111, volatile memory 112, persistent storage 113 (including the operating system 122 and block 200 identified above), a peripheral device set 114 (including a user interface (UI) device set 123, storage 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. The remote server 104 includes a remote database 130. The public cloud 105 includes a gateway 140, a cloud orchestration module 141, a host physical machine set 142, a virtual machine set 143, and a container set 144.

[0050] Computer 101 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device currently known or to be developed in the future that is capable of running programs, accessing networks, or querying databases such as remote database 130. As is well understood in the technical field of computer technology, and depending on the technology, the execution of a computer implementation method may be distributed among multiple computers and / or across multiple locations. On the other hand, in this presentation of computing environment 100, in order to keep the presentation as simple as possible, the detailed discussion focuses on a single computer, specifically computer 101. Although computer 101 is not shown in the cloud in Figure 1, it may be located in the cloud. On the other hand, computer 101 is not required to be located in the cloud, except to any extent that can be definitively shown.

[0051] The processor set 110 includes one or more computer processors of any type currently known or to be developed in the future. The processing circuitry 120 may be distributed across multiple packages, for example, multiple coordinated integrated circuit chips. The processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. The cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for high-speed access by threads or cores running on the processor set 110. The cache memory is typically organized into multiple levels depending on its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set may be located "off-chip". In some computing environments, the processor set 110 may operate with qubits and be designed to perform quantum computing.

[0052] Computer-readable program instructions are typically loaded onto computer 101, causing the processor set 110 of computer 101 to execute a series of operational steps, thereby realizing a computer implementation method. As a result, the instructions thus executed instantiate the methods specified in the flowcharts and / or descriptions of the computer implementation methods contained herein (collectively referred to as the "Methods of the Invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the Methods of the Invention. In computing environment 100, at least some of the instructions for executing the Methods of the Invention may be stored in blocks 200 in persistent storage 113.

[0053] The communication fabric 111 is a signal conduction path that enables various components of the computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive paths, such as buses, bridges, physical input / output ports, and similar components. Other types of signal communication paths, such as optical fiber communication paths and / or wireless communication paths, may be used.

[0054] The volatile memory 112 is any type of volatile memory currently known or to be developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, the volatile memory 112 is characterized by random access, but this is not required unless explicitly stated. In computer 101, the volatile memory 112 is located in a single package and resides inside computer 101, but alternatively or additionally, the volatile memory may be distributed across multiple packages and / or located externally to computer 101.

[0055] The persistent storage 113 is any form of non-volatile storage for a computer, currently known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether the computer 101 and / or the persistent storage 113 are directly powered. The persistent storage 113 may be read-only memory (ROM), but typically at least a portion of the persistent storage allows for writing, deleting, and rewriting of data. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. The operating system 122 can take multiple forms, such as various known proprietary operating systems or open-source portable operating system interface type operating systems employing a kernel. The code contained in block 200 typically includes at least a portion of computer code involved in performing the method of the present invention.

[0056] The peripheral device set 114 includes a set of peripheral devices for the computer 101. Data communication connections between the computer 101's peripheral devices and other components can be implemented in various ways, including Bluetooth® connections, Near-Field Communication (NFC) connections, connections made by cables (such as universal serial bus (USB) type cables), insert-type connections (e.g., secure digital (SD) cards), connections made via local area communication networks, and even connections made via wide area networks such as the internet. In various embodiments, the UI device set 123 may include components such as a display screen, speaker, microphone, wearable devices (such as goggles and smartwatches), keyboard, mouse, printer, touchpad, game controller, and haptic device. Storage 124 is external storage such as an external hard drive, or insertable storage such as an SD card. Storage 124 may be persistent and / or volatile. In some embodiments, storage 124 may take the form of a quantum computing memory device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (for example, computer 101 locally stores and manages a large database), this storage may be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another may be a motion detector.

[0057] The network module 115 is a collection of computer software, hardware, and firmware that enables computer 101 to communicate with other computers via the WAN 102. The network module 115 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for transmission over a communication network, and / or web browser software for communicating data over the internet. In some embodiments, the network control and network forwarding functions of the network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control and forwarding functions of the network module 115 are performed on physically separate devices so that the control function manages multiple different network hardware devices. Computer-readable program instructions for performing the methods of the present invention can typically be downloaded from an external computer or external storage device to computer 101 via a network adapter card or network interface included in the network module 115.

[0058] WAN102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances by any currently known or future-developed technology for transmitting computer data. In some embodiments, WAN102 may be replaced and / or complemented by a local area network (LAN), such as a Wi-Fi network, designed to transmit data between devices located in a local area. WANs and / or LANs typically include computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0059] An end-user device (EUD) 103 is any computer system used and controlled by an end-user (e.g., a customer of the company operating computer 101), and can take any of the forms discussed above in relation to computer 101. EUD 103 typically receives useful and valuable data from the operation of computer 101. For example, in a hypothetical case where computer 101 is designed to provide recommendations to an end-user, these recommendations would typically be communicated from computer 101's network module 115 to EUD 103 via WAN 102. In this way, EUD 103 can display or otherwise present the recommendations to the end-user. In some embodiments, EUD 103 may be a client device such as a thin client, heavy client, mainframe computer, desktop computer, and similar.

[0060] The remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. The remote server 104 may be controlled and used by the same entity that operates computer 101. The remote server 104 represents a machine that collects and stores useful and valuable data for use by other computers, such as computer 101. For example, in a hypothetical case where computer 101 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to computer 101 from the remote database 130 of the remote server 104.

[0061] The public cloud 105 is any computer system available for use by multiple entities, providing on-demand availability of computer system resources and / or other computer functions, particularly data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct active management of the computing resources of the public cloud 105 is performed by the computer hardware and / or software of the cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments running on various computers that make up the host physical machine set 142, which is the universe of physical computers available in and / or to the public cloud 105. The virtual computing environment (VCE) typically takes the form of virtual machines from the virtual machine set 143 and / or containers from the container set 144. These VCEs may be stored as images and are understood to be transferable either as images or after instantiation of the VCE, among and between various physical machine hosts. The cloud orchestration module 141 manages the transfer and storage of images, deploys new VCE instances, and manages active instances of the VCE deployment. The gateway 140 is a collection of computer software, hardware, and firmware that enables the public cloud 105 to communicate over the WAN 102.

[0062] Here, some further explanation of virtualized computing environments (VCEs) is provided. A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from an image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature in which the kernel allows for the existence of multiple isolated user-space instances called containers. These isolated user-space instances typically behave as actual computers from the perspective of the programs running in them. Computer programs running on a normal operating system can utilize all the resources of that computer, including connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running inside a container can only use the contents of the container and the devices allocated to that container; this feature is known as containerization.

[0063] The private cloud 106 is similar to the public cloud 105, except that its computing resources are available only for use by a single enterprise. While the private cloud 106 is shown as communicating with the WAN 102, in other embodiments, the private cloud may be completely isolated from the internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. Each of the multiple clouds remains a separate and independent entity, but the larger hybrid cloud architecture is coupled together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the multiple configuration clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0064] Cloud computing services and / or microservices (not shown separately in Figure 1): Private and public clouds 106 are programmed and configured to deliver cloud computing services and / or microservices (unless otherwise indicated, the term “microservices” should be interpreted as including larger “services” of any size). Cloud services are typically infrastructure, platforms, or software hosted by a third-party provider and made available to users over the internet. Cloud services facilitate the flow of user data from front-end clients (e.g., user-side servers, tablets, desktops, laptops) to the provider’s systems over the internet and vice versa. In some embodiments, cloud services may be configured and orchestrated according to an “as a service” technology paradigm, where something is presented to internal or external customers in the form of cloud computing services. An as a service offer typically provides endpoints to which various customers interface. These endpoints are typically based on a set of APIs. One category of as-a-service offers is Platform as a Service (PaaS), where a service provider provisions, instantiates, runs, and manages a modular bundle of code that a customer can use to instantiate a computing platform and one or more applications, typically without the complexity of building and maintaining the infrastructure associated with them. Another category is Software as a Service (SaaS), where software is centrally hosted and allocated on a subscription basis. SaaS is also known as on-demand software, web-based software, or web-hosted software.The four technical sub-areas involved in cloud services are deployment, integration, on-demand, and virtual private networks.

[0065] Various embodiments provide a novel type of autoregressive generation system for machine learning models. By making appropriate modifications to the model, it is possible to create a transformer that takes a complete probability distribution across the entire vocabulary as input and produces an additional complete probability distribution as output. Autoregressive generation can be performed using these complete probability distributions. Some of the advantages of this process include no information loss (except for rounding errors) because the output from previous stages is not discarded. Furthermore, the system can eliminate the non-differentiable stage of the top k sampling (max⇔top-1), resulting in reduced information loss during backpropagation. This allows for more accurate loss calculations. Here, the transformer model produces results that are not "collapsed" to the same extent as one-hot coding.

[0066] The system described herein may be particularly useful in situations where there are multiple tokens that are highly likely to be the answer. In this case, the system can retain all information about all tokens, thereby allowing for the consideration of paths for multiple additional tokens that would not normally be considered by an autoregressive generation process. The use of a total probability distribution reduces information loss and leads to more accurate predictions for each token in the sequence.

[0067] Figure 2 illustrates process 201 in which the machine learning model 222 performs an autoregressive generation process, based on an example and features of this solution. The machine learning model 222 may be an AI model, a transformer model, and / or similar. For example, the machine learning model 222 may be a transformer neural network such as an LLM. Referring to Figure 2, the host platform 220 may host a software application (not shown) containing the machine learning model 222. The host platform 220 may be a cloud platform, a web server, an on-premises server, a distributed system, a hybrid system, and similar. A user may operate a user device to connect to the host platform 220. In response, the host platform 220 may output a graphical user interface 210 on the display screen of the user device. The user may input data and commands to the machine learning model 222 via the graphical user interface 210.

[0068] In this example, the user provides input 212, which is a question posed to the machine learning model 222. Input 212 may include an input sequence of words mapped to tokens in a predefined vocabulary. In response, the machine learning model 222 may generate a response to input 212 using autoregressive generation. The response may include a sequence of tokens output / predicted by the machine learning model 222, including a first predicted token 230, a second predicted token 231, a third predicted token 232, a fourth predicted token 233, a fifth predicted token 234, a sixth predicted token 235, and a seventh predicted token 236, which are predicted by the machine learning model 222 during multiple iterations of the autoregressive generation process.

[0069] For example, the machine learning model 222 may receive input 212 from the graphical user interface 210, retrieve additional content such as a predefined token vocabulary, any additional data to be taken in (e.g., documents, images, audio, video, etc.), and similar from data sources 224 and 226, and generate a predicted output. Here, the machine learning model 222 may iteratively execute an autoregressive generation process. During the first iteration of the autoregressive generation process, the machine learning model 222 may predict a first predicted token 230 based on the input 212 and the predefined token vocabulary. During the second iteration, the autoregressive generation process may generate a second predicted token 231 based on the first predicted token 230 and the predefined token vocabulary.

[0070] This process may continue for multiple additional iterations until a stopping point, such as a limiting criterion, is reached or the answer is completely determined. In some cases, the limiting criterion is reaching the maximum number of tokens the system can handle, or the sequence end token is generated. In this example, the third iteration of the autoregressive generation process generates a third prediction token 232 from a predefined token vocabulary including the first token 230 and the second token 231; the fourth iteration of the autoregressive generation process generates a fourth prediction token 233 from a predefined token vocabulary including the first token 230, the second token 231, and the third token 232; and the fifth iteration of the autoregressive generation process generates a predefined token vocabulary including the first token 230, the second token 231, the third token 232, and the fourth token 233. The sixth iteration of the autoregressive generation process generates a sixth prediction token 235 from a predefined token vocabulary including the first token 230, the second token 231, the third token 232, the fourth token 233, and the fifth token 234, and the seventh iteration of the autoregressive generation process generates a seventh prediction token 236 from a predefined token vocabulary including the first token 230, the second token 231, the third token 232, the fourth token 233, the fifth token 234, and the sixth token 235. The prediction tokens can be sequentially concatenated by the machine learning model 222 and output as a string of tokens. This output can be displayed on the graphical user interface 210.

[0071] Figures 3A to 3D show examples of autoregressive generation processes that may be performed by the machine learning model 222 during each iteration. For example, Figure 3A shows process 300A for generating a probability distribution, with an example and features of this solution. Referring to Figure 3A, the host platform 310 hosts the software application 312 coupled to the machine learning model 320.

[0072] In this example, machine learning model 320 may correspond to machine learning model 222 shown in the example in Figure 2. To initiate the prediction process, software application 312 may receive an input sequence "define machine learning" which may be asked for or otherwise entered by the user of software application 312. In this example, the input sequence includes three tokens 302, 304, and 306, which represent "define," "machine," and "learning," respectively. Software application 312 may generate a probability distribution for each of the three input tokens across a predefined vocabulary from a vocabulary index (not shown), resulting in a sequence of probability distributions 314, which are referred to herein as N probability distributions, where N is the number of probability distributions in the sequence. Here, the sequence includes three probability distributions (i.e., one probability distribution for each input token). The vocabulary index may be referred to as the dominant set of tokens. Each token in the index may receive a probability value of its likelihood of being the next token in the answer / response.

[0073] Each probability distribution in the sequence of N probability distributions 314 will contain a slot for each token in a vocabulary containing V tokens. For example, V tokens can be a large number such as 1,000, 10,000, or 50,000 tokens. However, the only slot that will contain a value is the slot corresponding to the input token. For example, the probability distribution for the input token "define" will contain "1" in the slot of the probability distribution corresponding to the token "define" from the vocabulary, while the other slots in the probability distributions corresponding to other tokens in the vocabulary will have the value zero (0). This is called one-hot coding. Similar one-hot coding can be generated for each input token, resulting in three one-hot coded probability distributions.

[0074] In some embodiments, a sequence of probability distributions 314 may be referred to as the “initial” sequence of the probability distribution, and these are obtained by tokenizing “prompts,” which may be questions or tasks to be completed. Tokenization divides the input sequence into units (pieces) in which lexical entries exist, thereby creating a sequence of lexical indices. The “one-hot” representation of the lexical indices is a pseudo-probability distribution.

[0075] A sequence of probability distributions 314 is input to a machine learning model 320, which can be used to predict the probability distribution for a first output token (i.e., the first token in the response to the input sequence "define machine learning"). Here, the machine learning model 320 includes a linear ExV layer 322, a transformer stack 324, a linear layer 326, and a softmax layer 328. The linear ExV layer 322 is configured to map a V-dimensional vector (a probability distribution across the lexicon) to an internal embedding of dimension E. Thus, the linear ExV layer 322 refers to the size of a matrix, which has E rows and V columns. The linear ExV layer 322 maps a vector of dimension V to a vector of dimension E. The transformer stack 324 passes the internal embedding through a series of layers, each of which is a transformer self-attention block, ultimately producing a "feature vector" of internal dimension E. The linear layer 326 remaps the feature vector to a V-dimensional vector, i.e., a logit distribution across the lexicon. Next, the softmax layer 328 transforms the logit distribution into a probability distribution corresponding to the first output token. This probability distribution includes probability values ​​(e.g., numerical values) indicating the probability that each generated output token should be a corresponding member of the vocabulary, e.g., a word. If the vocabulary contains 10,000 words, the probability distribution includes the probability values ​​(10,000 probability values) that each word should be the next part of the sequence in which it is generated.

[0076] Figure 3B shows a process 300B that utilizes a sequence of probability distributions (e.g., probability distribution 314) during an iteration of the autoregressive generation process, according to an example and features of this solution. Referring to Figure 3B, the machine learning model 320 can transform the probability distributions into embeddings by running a linear ExV layer 322 on the sequence of probability distributions. In the example of Figure 3B, the sequence of N probability distributions 314 shown in Figure 3A includes three probability distributions 330, 340, and 350 corresponding to three input tokens "define", "machine", and "learning". During the first iteration of the autoregressive generation process, a fourth probability distribution 360 is generated. The fourth probability distribution 360 corresponds to a first output token (i.e., answer, response, etc.) for the input tokens. The fourth probability distribution 360 is referred to herein as the N+1 probability distribution.

[0077] Here, the number of data values ​​(V) in probability distributions 330, 340, and 350 can be significantly larger than the number of data values ​​(E) that can fit into the embedding vectors generated by the linear ExV layer 322. The linear ExV layer can use matrices to transform probability distributions 330, 340, and 350, each having V values ​​(dimension V), into embeddings of dimension E, including embeddings 331, 341, and 351. Embeddings 331, 341, and 351 are referred to herein as embeddings that embed V values ​​from lexical probability distributions into a size of E values, where E is less than V.

[0078] The machine learning model 320 may run the transformer stack 324 on embeddings 331, 341, and 351 to generate feature embeddings 332, 342, 352, and 362, also referred to herein as embedding N+1. The transformer stack 324 includes a self-attention layer that predicts a feature embedding 362 corresponding to a first output token. In some embodiments, feature embeddings 332, 342, and 352 are equal to embeddings 331, 341, and 351, respectively, so that the transformer stack 324 does not adjust these embeddings, but instead analyzes the relationships between tokens in one or more input sets of embeddings 331, 341, and 351 to make a prediction about the next token for the answer / response, which is represented by the feature embedding 362. In some embodiments, the transformer stack 324 itself includes another encoder, and as a result, the transformer stack 324 performs predictions about the next token and also performs an additional transformation of the input embeddings to a smaller dimensional space than the dimensional space used for embeddings 331, 341, and 351. Thus, in these embodiments, feature embeddings 332, 342, and 352 are different from embeddings 331, 341, and 351.

[0079] Next, the machine learning model 320 runs a linear layer 326 over N+1 feature embeddings 332, 342, 352, and 362 to generate N+1 logits across the vocabulary 333, 343, 353, and 363. Logits, sometimes also referred to as activations, refer to the raw data outputs or raw scores from the final layer of the machine learning model. The N+1 logit scores include the raw score for each member of the dominant set of tokens, e.g., each member of the vocabulary, and thus have a size of V raw scores. The softmax layer 328 then transforms the N+1 logits across the vocabulary 333, 343, 353, and 363 into a probability distribution 360, which is referred to herein as probability distribution N+1. The additional probability distribution 360 includes the probability for each data value V (V tokens) in the vocabulary, which indicates the likelihood that the token is the output token for this iteration, i.e., the prediction of the next token following the input sequence. In this case, the additional probability distribution 360 is not a one-hot coded representation like the three probability distributions 330, 340, and 350 are. Instead, each slot in the additional probability distribution 360 stores a decimal value representing the probability that the corresponding token from the vocabulary is a correct token. The softmax layer 328 ensures that the sum of the probabilities in the additional probability distribution 360 is 1. In at least some embodiments, this additional probability distribution 360 is a vector of floating-point values, and the only information that might be lost for each iteration is a floating-point rounding error on the probability values ​​in the vector.

[0080] Traditionally, an additional probability distribution 360 was analyzed to identify the most likely probability of the most probable token (e.g., top 1 token, top 3 tokens, etc.), and the remaining probabilities from the additional probability distribution 360 were dropped or otherwise discarded. However, in an exemplary embodiment, the entire additional probability distribution 360 is fed back for the next iteration of the autoregressive generation process, thereby preventing information loss and allowing the machine learning model 320 to explore more solutions. Thus, if multiple tokens are very similar / close, the machine learning model can explore multiple answer paths in each iteration compared to a conventional autoregressive generation process that discards this extraneous information.

[0081] Figure 3C shows process 300C, which performs the next iteration of the autoregressive generation process according to an example and features of this solution. Referring to Figure 3C, the machine learning model 320 can perform a linear ExV layer 322 on N+1 probability distributions (e.g., probability distributions 330, 340, 350, and 360 from Figure 3B) to transform the N+1 probability distributions into N+1 embeddings, including an embedding 361 corresponding to an additional probability distribution 360. As previously mentioned, the number of data values ​​V in each probability distribution can be significantly larger than the number of data values ​​E that can fit into the embedding vectors generated by the linear ExV layer 322. The ExV layer can use matrices to transform a probability distribution with V data values ​​into an embedding with E data values ​​in a vector space, where E is less than V.

[0082] The machine learning model 320 can run a transformer stack 324 on N+1 embeddings (including embedding 361) to generate N+2 feature embeddings, including a feature embedding 362 corresponding to the next predicted output token (N+2). Next, the machine learning model 320 runs a linear layer 326 on the N+2 feature embeddings (including feature embedding 362) to generate N+2 logits across the vocabulary for the N+2 embeddings, the embeddings including a logit across the vocabulary 363 corresponding to the next predicted output token. Next, a softmax layer 328 transforms the N+2 logits across the vocabulary (including the logit across the vocabulary 363) into an N+2 probability distribution including an additional probability distribution 370, referred herein to as probability distribution N+2. The additional probability distribution 370 includes the probability that each data value in the vocabulary is a second output token for the response to the initial query input. In at least some embodiments, in additional iterations such as the iterations described in Figure 3C, some of the embeddings, logits, and probability distributions are preserved from one or more previous iterations and do not need to be recalculated. Nevertheless, the various components naturally analyze the received information anew and calculate N+X embeddings, N+X+1 embeddings including the new token prediction set embeddings, N+X+1 logits, and N+X+1 probability distributions anew.

[0083] This same process, shown in Figures 3B and 3C, can be repeated until all predicted output tokens are generated, where in each iteration, additional predicted tokens are added based on the previous predicted tokens, utilizing the full distribution value of the vocabulary.

[0084] Figure 3D illustrates the process 300D for selecting tokens for output by a machine learning model, based on an example and features of this solution. Traditionally, machine learning models select predictive tokens after each iteration of an autoregressive generation process. However, in the exemplary embodiment, it is not necessary to select predictive tokens after each iteration, as all probabilities from all probability distributions are preserved and can be analyzed at the end of the iterative process (e.g., after the last iteration).

[0085] Referring to Figure 3D, the machine learning model 320 may analyze each of the probability distributions iteratively generated by the autoregressive generative process, select tokens from each probability distribution, generate a sequence of predicted tokens from the selected tokens, and output the sequence of predicted tokens. For example, the machine learning model 320 may select token 381 as the first predicted token from the N+1 probability distribution 360, where token 381 may be the token with the highest probability value. The machine learning model 320 may also select token 382 from the N+2 probability distribution 370, where token 382 may be the token with the highest probability in the N+2 probability distribution 370. The selected tokens (e.g., token 381, token 382, ​​etc.) may be concatenated and output by the machine learning model 320 as a response to the sequence of input tokens. In at least some embodiments, this analysis of a sequence of probability distributions to find the most likely token for each stage of the output sequence is considered greedy decoding, but greedy coding is performed after all predictions have been made by the transformer stack and / or after all probability distributions (one for each response / answer token) have been generated.

[0086] Here, the machine learning model 320 may wait to generate prediction tokens until all iterations of the autoregressive generation process have been completed.

[0087] In at least some embodiments, generative machine learning models used to perform generative tasks using iterative passback of perfect probability distributions described herein undergo augmented training by using ground truth probability distributions that yield the correct golden sequence with respect to metrics such as L1 or L2 (as opposed to one-hot coding) as labeled training data. The correct golden sequence is also referred to as the ground truth output, e.g., the correct response / answer to a query. In at least some embodiments, the ground truth probability distribution refers to one or more probability distributions within a certain predefined margin from the one-hot coding of the correct golden sequence. The target of such predicted probability distributions may be replaced by improved predictions as the model improves.

[0088] Figure 4A shows a flowchart of Method 400 according to an exemplary embodiment. Referring to Figure 4A, in 401, the Method may comprise a step of performing an autoregressive generation process of a generative machine learning model by producing a prediction of a first answer token represented by a first probability distribution over V values, where V is a number greater than 1 of tokens in a predefined dominant set of tokens. In 402, the first probability distribution is fed back as input to the generative machine learning model for the next iteration of the autoregressive generation process. In 403, the next iteration produces the next answer token represented by the next probability distribution. In 404, the step of performing produces an output from the generative machine learning model, which is based on a probability distribution including the first and next probability distributions.

[0089] Figure 4B shows a flowchart of Method 410 according to an exemplary embodiment. Referring to Figure 4B, in 411, the Method may further have the steps of: selecting a first value having the highest probability from among V values ​​of a first probability distribution; selecting a next value having the highest probability from among V values ​​of a next probability distribution; and converting the first value and the next value into a first output token and a second output token, respectively, based on a dominant set of predefined tokens, the output being a sequence of data containing the first output token and the second output token. In 412, the autoregressive generation process continues to provide feedback to generate additional response tokens until a limit criterion is reached, where the steps of selecting the first value, selecting the second value, and converting are performed in response to the reaching of the limit criterion and after the autoregressive generation process has generated the last probability distribution for the last token for the output.

[0090] In 413, for each iteration of the autoregressive generation process, the set of all probability distributions, including the first and subsequent probability distributions, is fed back as input to the generative machine learning model for the next iteration of the autoregressive generation process. In 414, the autoregressive generation process continues to provide feedback to generate additional response tokens until a limit criterion is reached, where output generation occurs in response to reaching the limit criterion.

[0091] In 415, the execution steps include: a step in which a generative machine learning model receives a text input; a step in which it generates input tokens from the text input; and a step in which it generates an input probability distribution for each input token, where each input probability distribution includes a one-hot coded representation of each input token for V tokens in a predefined dominant set of tokens, where the generative machine learning model produces a prediction of a first probability distribution by performing an execution on the input probability distribution. In 416, the method includes a step in which it embeds the input probability distributions for V values ​​of the input into embeddings using a linear embedding layer, where the linear embedding layer maps the probability distributions for V values ​​into embeddings for E values, where E is a value different from V. In 417, the value of E is a value smaller than V.

[0092] Figure 4C shows a flowchart of Method 420 according to an exemplary embodiment. Referring to Figure 4C, in 421, the value of E is less than the value of V. In 422, the generative machine learning model has a linear embedding layer for the autoregressive generative process that maps a probability distribution for V values ​​to embeddings for each of E values, where E is a different value from V. In 423, the generative machine learning model further has a transformer stack for the autoregressive generative process that receives embeddings from the linear embedding layer and in response generates next-token feature embeddings that represent predictions for the next token of the output.

[0093] In 424, the generative machine learning model further comprises a linear layer and a softmax function, where the linear layer receives an embedding containing a next-token feature embedding and in response generates V raw output scores, the V raw output scores are input to a softmax function which in response generates a first probability distribution. In 425, the generative machine learning model further comprises a transformer stack for an autoregressive generative process which receives an embedding and in response generates a next-token feature embedding representing a prediction about the next token of the output.

[0094] Figure 4D shows a flowchart of method 430 according to an exemplary embodiment. Referring to Figure 4D, in 431, the generative machine learning model has a linear layer and a softmax function, where, for the autoregressive generative process, the linear layer receives an embedding containing the next token feature embedding, in response to which it produces V raw output scores, the V raw output scores are input to a softmax function, which in response it produces a first probability distribution.

[0095] In 432, the method comprises the step of training a generative machine learning model to perform an autoregressive generative process by using one or more ground truth probability distributions for supervised learning, where one or more ground truth probability distributions are based on a correct golden sequence of ground truth answers to a query. In 433, one or more ground truth probability distributions are within a specific predefined margin from the one-hot coding of the correct golden sequence.

[0096] Detailed descriptions of training and running machine learning models are further explained and illustrated herein. The training and running of machine learning models described in the examples in Figures 5A–5C may be performed within a confidential machine learning computing environment, as described in the examples herein.

[0097] Figure 5A shows an AI network supporting an artificial intelligence (AI)-assisted decision point in a software service running on a computer. As an example, the AI ​​models trained in the examples herein may refer to AI models for any of the tasks performed herein, including machine learning models, neural networks, large language models (LLMs), and similar models. While the exemplary solutions presented utilize machine learning models such as neural networks, a type of machine learning model (ML), other branches of AI, including but not limited to computer vision, fuzzy logic, expert systems, deep learning, generative AI, and natural language processing, may be employed in the development of the AI ​​models in these solutions. Furthermore, the AI ​​models included in these examples and features of the solutions are not limited to specific AI algorithms. Any algorithm, or combination of algorithms, relating to supervised, unsupervised, and reinforcement learning may be employed.

[0098] The AI ​​models, ML models, neural networks, and other branches of AI described and / or illustrated herein are built upon the fundamental principles of prior art, forming the foundation for all future technological advancements in artificial intelligence. The AI ​​classification system describes the stages of progress and development of AI. The first classification is known as "reactive machines," followed by the modern AI classification of "limited memory machines" (also known as "artificial narrow intelligence"), then evolving to "theory of mind" (also known as "artificial general intelligence"), and finally reaching the AI ​​classification of "self-aware" (also known as "artificial superintelligence"). Modern limited memory machines are an expanded group of AI models built upon their predecessors, reactive machines. Reactive machines mimic human responses to stimuli; however, their capabilities are limited because they typically cannot learn from previous experiences. As AI models demonstrated their learning capabilities, their classification evolved into that of limited-memory machines. In this modern classification, AI models inherited all the capabilities of reactive machines while learning from large amounts of data, detecting patterns, solving problems, generating and predicting data, and similar tasks.

[0099] Examples of AI models classified as limited memory machines include, but are not limited to, chatbots, virtual assistants, machine learning, neural networks, deep learning, natural language processing, generative AI models, and any future AI models yet to be developed that possess the characteristics of a limited memory machine.

[0100] For example, neural networks are a type of machine learning model that learns associations and connections based on training data, improving their accuracy to perform fast data classification, clustering, and other data analyses. Such neural network capabilities form the foundation of current deep learning models and also serve as the basic building blocks for models yet to be developed.

[0101] For example, generative AI models combine limited memory machine learning and deep learning with limited memory machine learning techniques to form the foundational building blocks for future AI models. For instance, theory of mind is the next advancement in AI, enabling AI models to perceive, connect, and react by generating appropriate responses in response to the entities they interact with; all of these theory of mind capabilities rely on the fundamental principles of generative AI. Furthermore, in the evolution to self-aware classification, AI models will be able to understand and evoke emotions in the entities they interact with, as well as possess their own emotions, beliefs, and desires, all of which rely on the fundamental principles of generative AI, which learn from experience and generate and derive conclusions about themselves and their surroundings.

[0102] An AI model may include, but is not limited to, at least one machine learning model, neural network model, deep learning model, generative AI model, or any combination of models from the lineage of AI. AI models are essential and core to future artificial intelligence models. As used herein, AI models refer to both modern and future AI models.

[0103] Artificial intelligence systems are built and trained to perform various tasks in an automated manner. For example, AI systems can receive and understand oral and / or written dialogue and function as digital assistants, speech-to-text programs, etc. Other AI systems are trained with different types of information, enabling the trained system to generate content such as new works of art based on observed styles or new compound ideas based on the history of chemical research.

[0104] A foundational model is a type of artificial intelligence system that is trained on a broad set of unlabeled data and can be used for different tasks with minimal fine-tuning. Unlabeled data, in some cases, includes images and / or language. In response to a short prompt input to the foundational model, the system generates an output, such as an entire essay or a complex image, based on the parameters indicated in the input prompt. The foundational model can produce an output that attempts to satisfy the parameters, even if it has never been trained with specific training data containing those exact parameters, for example, regarding that exact argument or to generate an image in that manner.

[0105] By using self-supervised learning and transfer learning, foundational models can apply the information they have learned about one situation to another. For example, just as a person who learns how to drive a certain car can learn how to drive other types of vehicles, such as other cars, trucks, or buses, without excessive effort, foundational models can similarly be used to achieve proficiency in several new domains without needing to be trained entirely from scratch. Foundational models appear to possess inherent creativity in performing tasks such as piecing together a coherent argument or creating entirely original works of art. Foundational models are well-established in natural language processing techniques. One example of how useful foundational models are is that, with respect to previous generations of AI techniques, if one wanted to build an AI model capable of summarizing the body of text for oneself, tens of thousands of labeled examples would have been needed just for the summarization use case. By using pre-trained foundational models, the amount of labeled data required is dramatically reduced. Firstly, foundational models are fine-tuned using domain-specific unlabeled corpora to create domain-specific foundational models. Next, a foundational model for summarization is trained using a much smaller amount of labeled data, potentially as few as a thousand labeled examples. Unlike traditional techniques that require building a model from scratch for each use case, domain-specific foundational models are usable for multiple tasks. The foundational model can also be applied to areas such as coding analysis, generation, and repair in computer programming.

[0106] Some foundational models are used for sentiment analysis. By using pre-trained foundational models, sentiment analysis for new languages ​​can be trained using only a few thousand sentences, reducing the number of annotations required to one-hundredth of that for conventional models. Reducing labeling requirements makes implementation in various technical domains significantly easier. Systems that perform specific tasks within a single domain are being replaced by broader AI that learns more generally and works across multiple domains and problems. Foundational models trained on large, unlabeled datasets and fine-tuned for various applications are driving this shift.

[0107] Large-scale language models (LLMs) are a category of foundational models trained on massive amounts of data, capable of understanding and generating natural language and other types of content to perform a wide range of tasks. LLMs are implemented at different levels to enhance their natural language understanding (NLU) and natural language processing (NLP) capabilities. This development of LLMs has occurred in parallel with the development of machine learning, machine learning models, algorithms, neural networks, and transformer models that provide architectures for these AI systems.

[0108] LLM is a type of foundational model trained on vast amounts of data to provide the necessary foundational capabilities to drive multiple use cases and applications, and to solve numerous tasks. The concept of LLM is in stark contrast to the idea of ​​individually building and training domain-specific models for each of these use cases, which is prohivitive under several criteria (most importantly cost and infrastructure), hinders synergies, and can even lead to performance degradation.

[0109] LLM represents a significant breakthrough in NLP and artificial intelligence. LLM is accessible through interfaces such as OpenAI's Chat GPT-3 and GPT-4, which have received support from Microsoft®. Other examples include Meta's Llama model, Google's Transformer-based bidirectional encoder representations (BERT / RoBERTa), and the PaLM model. IBM has also recently launched its Granite model series on watsonx.ai, which serves as the backbone of generative AI for other IBM products such as watsonx Assistant and watsonx Orchestrate.

[0110] In short, LLMs are designed to understand and generate text similar to that produced by humans, in addition to other forms of content, based on the vast amount of data used to train them. They have the ability to infer from context, generate consistent and contextual responses, translate into languages ​​other than English, summarize text, answer questions (general conversations and FAQs), and even assist with creative document creation or code generation tasks. LLMs can perform some or all of these tasks thanks to the many, for example, billions of parameters that enable them to capture complex patterns in language and perform a wide range of language-related tasks. LLMs are transforming applications in various fields, from chatbots and virtual assistants to content generation, research support, and language translation.

[0111] LLMs operate by leveraging deep learning techniques and vast amounts of text data. These models are typically based on transformer architectures, such as generative pre-trained transformers, which excel at handling sequential data like text input. An LLM consists of multiple layers of neural networks, each with fine-tunable parameters during training, which are further enhanced by layers known as attention mechanisms that focus on specific parts of the dataset.

[0112] During the training process, these models learn to predict the next word in a sentence based on the context provided by the preceding word. This is done by assigning probability scores to the recurrence of tokenized words, i.e., words broken down into smaller sequences of characters. These tokens are then converted into embeddings, which are numerical representations of this context.

[0113] To ensure accuracy, this process involves training the LLM on a large text corpus (e.g., billions of pages), allowing the LLM to learn grammar, semantics, and conceptual relationships through zero-shot learning and self-supervised learning. Once trained on this training data, the LLM can autonomously predict the next word based on the input it receives and generate text by utilizing the patterns and knowledge it has acquired. This results in consistent and contextual language generation that can be used for a wide range of NLU and content generation tasks.

[0114] Model performance can also be improved through prompt engineering, prompt tuning, fine-tuning, and other methods such as Reinforcement Learning with Human Feedback (RLHF), which removes bias, hate speech, and inaccurate responses, often a byproduct of training on a large amount of unstructured data, known as "hallucination." LLM augments conversational AI in chatbots and virtual assistants, improving conversations to provide context-aware responses that mimic interactions with human agents.

[0115] LLM excels in content generation, automating the creation of blog posts, explanatory materials, and other document creation tasks. LLM accelerates knowledge discovery by assisting in the summarization and extraction of information from vast datasets. LLM also plays a crucial role in language translation, breaking down language barriers by providing accurate and contextual translations. Furthermore, LLM can be used for writing code or for "translating" between programming languages. LLM contributes to accessibility by assisting individuals with disabilities, including text-to-speech applications and content generation in user-friendly formats.

[0116] LLM often possesses the following capabilities: • Text generation: The ability to generate language in response to prompts, creating, refining, and editing documents such as emails, blog posts, or other medium-to-long-length content. A good example is Retrieval-Augmented Generation (RAG). • Content Summarization: Summarizes long articles, news articles, research reports, corporate documents, and even dialogue histories into detailed text with length adjusted according to the output format. • AI Assistant: A chatbot that answers queries, performs backend tasks, and provides detailed information in natural language as part of an integrated self-service solution for handling inquiries. • Code generation: Assists developers in building applications, detecting errors in code, identifying security issues across multiple programming languages, and even "translating" between them. • Sentiment Analysis: Analyze text to determine user tone in order to understand large-scale user feedback and support brand reputation management. • Language Translation: Fluent translation and multilingual capabilities provide broad coverage to organizations across multiple languages ​​and regions.

[0117] A software service 504 (see Figure 5A) running on a host platform 502 (see Figure 5A) may provide one or more application programming interfaces (APIs) 520 that enable interaction with other software components through a set of data definitions and protocols. In some examples and features of this solution, the provided APIs may employ Simple Object Access Protocol (SOAP), Remote Procedure Calls (RPC), and Representational State Transfer (REST) ​​techniques. In some examples and features of this solution, multiple APIs 520 send data to one or more decision subsystems 524 of the software service 504 to support decision-making. In some examples and features of this solution, the software service 504 stores data contained in API requests or data generated during processing of API requests in one or more databases 506 (see Figure 5A).

[0118] The software service 504 may provide one or more user interfaces (UIs) 522, such as a graphical user interface (GUI) hosted on the server side. In some examples and features of this solution, the provided UIs 522 employ template-based frameworks, component-based frameworks, etc. In some examples and features of this solution, these UIs 522 send data to one or more decision subsystems 524 of the software service 504 to support decision-making. In some examples and features of this solution, the software service 504 stores data included in UI requests or data generated during the processing of UI requests in one or more databases 506.

[0119] The software service 504 may include one or more decision subsystems 524 that drive the decision-making process of the software service 504. In some examples and features of this solution, the decision subsystem 524 receives data from one or more APIs 520 as input to the decision-making process. In some examples and features of this solution, the decision subsystem 524 may receive data from one or more UIs 522 as input to the decision-making process. The decision subsystem 524 may collect service configuration or historical execution data from one or more databases 506 to assist in the decision-making process. The decision subsystem 524 may provide feedback to APIs 520 or UIs 522.

[0120] The AI ​​production system 530 may be used to support its decision-making process by a decision subsystem 524 within a software service 504. The AI ​​production system 530 includes one or more AI models 532, which are executed to generate responses such as prediction, categorization, and UI prompts, but are not limited to these. In some examples and features of this solution, the AI ​​production system 530 is hosted on a server. In some examples and features of this solution, the AI ​​production system 530 is hosted in the cloud. In some examples and features of this solution, the AI ​​production system 530 is deployed in a distributed multi-node architecture.

[0121] The AI ​​development system 540 creates one or more AI models 532. In some examples and features of this solution, the AI ​​development system 540 develops and trains one or more AI models 532 using data from one or more data sources 550. The data sources 550 may be local or third-party data sources. Furthermore, the data provided by the data sources may be real-world or synthetic. In some examples and features of this solution, the AI ​​development system 540 utilizes feedback data from one or more AI production systems 530 for the development of new models and / or the retraining of existing models. In some examples and features of this solution, the AI ​​development system 540 resides on a server and runs on a server. In some examples and features of this solution, the AI ​​development system 540 is hosted in the cloud. In some examples and features of this solution, the AI ​​development system 540 is deployed in a distributed multi-node architecture. In some examples and features of this solution, the AI ​​development system 540 utilizes a distributed data pipeline / analysis engine.

[0122] Once trained and validated in the AI ​​development system 540, the AI ​​model 532 may be stored in the AI ​​model registry 560 for retrieval by either the AI ​​development system 540 or one or more AI production systems 530. In one example of this solution, the AI ​​model registry 560 resides on a dedicated server. In some examples and features of this solution, the AI ​​model registry 560 is hosted in the cloud. In some examples and features of this solution, the AI ​​model registry 560 resides within the AI ​​production system 530. In some examples and features of this solution, the AI ​​model registry 560 is a distributed database.

[0123] Figure 5B shows a process 500B for developing one or more AI models to support AI-assisted decision-making points. The AI ​​development system 540 performs a step to develop an AI model 532, which begins with data extraction 541, where data is loaded and ingested from one or more data sources 550. In some examples and features of this solution, historical model feedback data is extracted from one or more AI production systems 530.

[0124] Once data is extracted during data extraction 541, the data undergoes data preparation 542 for model training. In some examples and features of this solution, this stage involves performing statistical tests to verify how well the data reflects real-world events, including its distribution and the diversity of data within the dataset. The results of these statistical tests may lead to the adoption of one or more data transformations to normalize one or more values ​​within the dataset. In some examples and features of this solution, data deemed noisy is cleaned. Noisy datasets include, but are not limited to, null values ​​and long string values, and other values ​​that do not contribute to training. Data preparation 542 may be a manual or automated process using one or more elements and / or functions described and / or illustrated herein.

[0125] Data features are identified and extracted during the feature extraction stage 543. In some examples and features of this solution, a data feature lies within the prepared data from the data preparation stage 542. In some examples and features of this solution, a data feature requires that a portion of the prepared data from the data preparation stage 542 be enhanced with data from another data source so as to be useful in developing the AI ​​model 532. In some examples and features of this solution, identifying relevant features (relevant attributes) for model training is performed via an automated process using one or more of the elements and / or functions described and / or illustrated herein. Once features are identified, their values ​​are collected into the dataset used to develop the AI ​​model 532.

[0126] The dataset output from the feature extraction stage 543 is split into a training dataset and a validation dataset 544. The training dataset is used to train the AI ​​model 532, and the validation dataset is used to evaluate the performance of the AI ​​model 532 on unseen data.

[0127] The AI ​​model 532 is trained and tuned using the training dataset from the data partitioning stage 544 545. In this stage, the training dataset is provided with an initial set of algorithm parameters that can be automatically determined based on the AI ​​algorithm and the interdependencies between relevant attributes determined according to various embodiments. The performance of the AI ​​model 532 is then tested within the AI ​​development system 540 using the validation dataset from stage 544. These stages may be repeated with adjustments to one or more algorithm parameters until the model's performance is acceptable based on various goals and / or results.

[0128] The AI ​​model 532 is evaluated in a staging environment (not shown) similar to the target AI production system 530 546. This evaluation uses a validation dataset to ensure that the performance in the AI ​​production system 530 matches or exceeds expectations. In some examples and features of this solution, the validation dataset from stage 544 is used. In some examples and features of this solution, one or more previously unseen validation datasets are used. In some examples and features of this solution, the staging environment is part of the AI ​​development system 540 and is managed separately from the AI ​​development system 540. Once the AI ​​model 532 is validated, it is stored in the AI ​​model registry 560, where it is available for deployment and future updates. In some examples and features of this solution, the model evaluation stage 546 may be a manual or automated process using one or more of the elements and / or functions described and / or illustrated herein.

[0129] In some examples and features of this solution, the AI ​​development system includes a user interface (not shown). The user interface may be used to manage the development system infrastructure, stages 541-548 within the development system, intermediate data transmitted between the various stages 541-548, and the data source 550.

[0130] Once the AI ​​model 532 is validated and published to the AI ​​model registry 560, it may be deployed to one or more AI production systems 530 during the model deployment phase 547. In some examples and features of this solution, the performance of the deployed AI model 532 is monitored by the AI ​​development system 540 548. In some examples and features of this solution, feedback data of the AI ​​model 532 is provided by the AI ​​production system 530 to enable model performance monitoring 548, and the AI ​​development system 540 periodically requests feedback data for model performance monitoring 548, and such feedback data includes one or more triggers that result in the AI ​​model 532 being updated by repeating phases 541-548 with updated data from one or more data sources 550.

[0131] Figure 5C illustrates process 500C for utilizing an AI model to support AI-assisted decision-making points. As previously stated, the AI ​​model utilization process illustrated herein reflects ML, a specific branch of AI, but this solution is not limited to ML, nor is it limited to any AI algorithm or combination of algorithms.

[0132] Referring to Figure 5C, the AI ​​production system 530 may be used by the decision-making subsystem 524 in the software service 504 to support its decision-making process. The AI ​​production system 530 provides an API 534 executed by an AI server process 536 that can make requests. In some examples and features of this solution, a request may include an identifier for an AI model 532 that will be executed based on the type of request. In some examples and features of this solution, a data payload (e.g., input to the AI ​​model during execution) may be included in the request. The data payload may include API 520 data from the software service 504, UI 522 data from the software service 504, or data (not shown) from other software service 504 subsystems.

[0133] Upon receiving an API 534 request, the AI ​​server process 536 may transform the data payload or a portion of the data payload into valid feature values ​​for the AI ​​model 532 537. The data transformation 537 may include, but is not limited to, combining data values, normalizing data values, and augmenting the input data with data from other data sources 550. Once the data transformation is complete, the AI ​​server process 536 runs the appropriate AI model 532 using the transformed input data. Upon receiving the execution result, the AI ​​server process 536 responds to the API requester, which is the decision subsystem 524 of the software service 504. In some examples and features of this solution, the response may result in an update of the UI 522 in the software service 504. In some examples and features of this solution, the response may include a request identifier that can be later used by the software service 504 to provide feedback on the performance of the AI ​​model 532. In some examples and features of this solution, a model feedback record may be added to the model feedback data 538 by the AI ​​server process 536.

[0134] In some examples and features of this solution, API 534 includes an interface for providing AI model 532 feedback after the execution response of AI model 532 has been processed. This mechanism allows the requester to provide feedback on the accuracy of the results of AI model 532. In some examples and features of this solution, the feedback interface includes an identifier for the initial request so that it can be used to associate the feedback with the request. Upon receiving a call to the feedback interface of API 534, the AI ​​server process 536 creates a model feedback record and adds it to model feedback data 538, which holds past model feedback records. In some examples and features of this solution, the records in this model feedback data 538 are provided to model performance monitoring 548 in the AI ​​development system 540. This model feedback data may be streamed to the AI ​​development system 540 or provided on request. In some examples and features of this solution, the model feedback records in model feedback data 538 are used as input for retraining AI model 532.

[0135] In some examples and features of this solution, the AI ​​production system 530 includes a user interface (not shown). The user interface may be used to manage the production system infrastructure, the components of production systems 530-538, and the operation of the AI ​​production systems and their components.

[0136] The embodiments described above may be implemented in hardware, in computer programs executed by a processor, in firmware, or in a combination thereof. Computer programs may be implemented on computer-readable media such as storage media. For example, computer programs may reside in random access memory ("RAM"), flash memory, read-only memory ("ROM"), erasable programmable read-only memory ("EPROM"), electrically erasable programmable read-only memory ("EEPROM"), registers, hard disks, removable disks, compact disk read-only memory ("CD-ROM"), or any other form of storage media known in the art.

[0137] An exemplary storage medium may be coupled to a processor so that the processor can read information from and write information to the storage medium. In an alternative configuration, the storage medium may be integrated with the processor. The processor and storage medium may reside within an application-specific integrated circuit ("ASIC"). In an alternative configuration, the processor and storage medium may exist as separate components.

Claims

1. The step of performing the autoregressive generative process of a generative machine learning model by generating predictions of a first response token represented by a first probability distribution over V values, where V represents a number greater than 1 of tokens in a predefined dominant set of tokens. Equipped with, Here, the first probability distribution is fed back as input to the generative machine learning model for the next iteration of the autoregressive generation process. Here, the following iteration produces the following response token, which is represented by the following probability distribution: Here, the execution step produces an output from the generative machine learning model, the output being based on a probability distribution including the first probability distribution and the next probability distribution. method.

2. The steps to be performed are: A step of selecting a first value from among the V values ​​of the first probability distribution that has the highest probability; The step of selecting the next value having the highest probability from among the V values ​​of the following probability distribution; and A step of converting the first value and the next value into a first output token and a second output token, respectively, based on the dominant set of predefined tokens; It further possesses, The output is a sequence of data including the first output token and the second output token. The method according to claim 1.

3. The method according to claim 2, wherein the autoregressive generation process continues the feedback to generate additional response tokens until a limit criterion is reached, wherein the steps of selecting the first value, selecting the next value, and transforming are performed in response to the reaching of the limit criterion and after the autoregressive generation process has generated a final probability distribution for the last token for the output.

4. The method according to any one of claims 1 to 3, wherein for each iteration of the autoregressive generation process, a set of all probability distributions including the first probability distribution and the next probability distribution is fed back as the input to the generative machine learning model for the next iteration of the autoregressive generation process.

5. The method according to any one of claims 1 to 3, wherein the autoregressive generation process continues the feedback to generate additional response tokens until a limit criterion is reached, wherein the generation of the output is performed in response to the reaching of the limit criterion.

6. The aforementioned execution step is when the generative machine learning model: The stage where text input is received; The step of generating an input token from the aforementioned text input; and A step of generating an input probability distribution for each of the input tokens, wherein each input probability distribution includes a one-hot coded representation of each input token for the V tokens in the dominant set of predefined tokens; The process is initiated via, where the generative machine learning model, by being executed on the input probability distribution, produces the prediction of the first probability distribution. The method according to any one of claims 1 to 3.

7. The method according to claim 6, wherein the action performed on the input probability distribution comprises the step of embedding the input probability distribution for the V values ​​of the input into an embedding using a linear embedding layer, where the linear embedding layer maps the probability distribution for the V values ​​into an embedding of E values, where E is a different value from V.

8. The method according to claim 7, wherein E is a value smaller than V.

9. The method according to any one of claims 1 to 3, wherein the generative machine learning model has a linear embedding layer that maps a probability distribution for V values ​​to embeddings for each of E values ​​for the autoregressive generative process, where E is a value different from V.

10. The method according to claim 9, wherein the generative machine learning model further comprises a transformer stack that receives the embeddings from the linear embedding layer for the autoregressive generative process and, in response, generates next-token feature embeddings representing predictions for the next token of the output.

11. The method according to claim 10, wherein the generative machine learning model further comprises a linear layer and a softmax function, wherein the linear layer receives the embedding, including the next token feature embedding, and in response generates V raw output scores, the V raw output scores are input to the softmax function, and the softmax function in response generates the first probability distribution.

12. The method according to any one of claims 1 to 3, wherein the generative machine learning model has a transformer stack that receives an embedding for the autoregressive generative process and in response generates a next-token feature embedding that represents a prediction about the next token of the output.

13. The generative machine learning model has a linear layer and a softmax function, where for the autoregressive generative process: The method according to any one of claims 1 to 3, wherein the linear layer receives an embedding including a next token feature embedding, and in response generates V raw output scores, the V raw output scores are input to the softmax function, and the softmax function in response generates the first probability distribution.

14. The method according to any one of claims 1 to 3, further comprising the step of training a generative machine learning model for performing the autoregressive generative process by using one or more ground truth probability distributions for supervised learning, wherein the one or more ground truth probability distributions are based on a correct golden sequence of ground truth answers to a query.

15. The method according to claim 14, wherein the one or more ground truth probability distributions are within a specific predefined margin from the one-hot coding of the correct golden sequence.

16. Processor set; One or more computer-readable storage media; and Program instructions stored on one or more computer-readable storage media The program instructions are provided to the processor set: A procedure for performing the autoregressive generative process of a generative machine learning model by generating predictions of a first response token represented by a first probability distribution over V values, where V represents a number greater than 1 of tokens in a predefined dominant set of tokens. Perform an action that has the following characteristics: Here, the first probability distribution is fed back as input to the generative machine learning model for the next iteration of the autoregressive generation process. Here, the following iteration produces the following response token, which is represented by the following probability distribution: Here, the procedure to be performed produces an output from the generative machine learning model, the output being based on a probability distribution including the first probability distribution and the next probability distribution. Computer system.

17. The steps to be performed are: A procedure for selecting a first value from among the V values ​​of the first probability distribution that has the highest probability; A procedure for selecting the next value having the highest probability from among the V values ​​of the following probability distribution; and A procedure for converting the first value and the next value into a first output token and a second output token, respectively, based on the dominant set of predefined tokens; It further includes, The output is a sequence of data including the first output token and the second output token. The computer system according to claim 16.

18. The computer system according to claim 17, wherein the autoregressive generation process continues the feedback to generate additional response tokens until a limit criterion is reached, wherein the steps of selecting the first value, selecting the next value, and transforming are performed in response to the reaching of the limit criterion and after the autoregressive generation process has generated a final probability distribution for the last token for the output.

19. The computer system according to any one of claims 16 to 18, wherein for each iteration of the autoregressive generation process, a set of all probability distributions, including the first probability distribution and the next probability distribution, is fed back as the input to the generative machine learning model for the next iteration of the autoregressive generation process.

20. to the computer A procedure for performing the autoregressive generative process of a generative machine learning model by generating predictions of a first response token represented by a first probability distribution over V values, where V represents a number greater than 1 of tokens in a predefined dominant set of tokens. A computer program for executing, Here, the first probability distribution is fed back as input to the generative machine learning model for the next iteration of the autoregressive generation process. Here, the following iteration produces the following response token, which is represented by the following probability distribution: Here, the procedure to be performed produces an output from the generative machine learning model, the output being based on a probability distribution including the first probability distribution and the next probability distribution. Computer program.

Images