Computer implementation method, computer program, and computer system (prompt session optimization)

Abstract

To understand and control a temperature parameter of an MLM which is one of important parameters in this field.SOLUTION: A machine learning model ("MLM") is set to a first temperature state, a baseline prompt is issued to the MLM at the first temperature state, and a first response to the baseline prompt is received from the MLM at the first temperature state. The MLM is set to a second temperature state, the baseline prompt is issued to the MLM at the second temperature state, and a second response to the baseline prompt is received from the MLM at the second temperature state. A selected baseline response ("SBR") is selected from the first and second responses to the baseline prompt. The SBR is supplied as a baseline action to a reinforcement learning model ("RLM") that is configured to compute a reward in response to the baseline action and to compute a predicted temperature state based on the reward.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates generally to optimizing prompt sessions with machine learning models ("MLM"), and more particularly, but not by way of limitation, to accurately and reliably receiving responses to prompt queries with a desired degree of linguistic determinism by applying reinforcement learning during prompt sessions to adjust MLM temperature. [Background technology]

[0002] Large-scale language models, such as MLMs, have revolutionized natural language processing by generating coherent and contextually relevant text responses to user prompts. Summary of the Invention [Problem to be solved by the invention]

[0003] One important parameter in this field is understanding and controlling the temperature parameter of the MLM. Temperature plays a fundamental role in controlling the randomness and creativity of the generated output. Temperature determines the degree to which the MLM explores alternative word choices, thereby introducing variability in its responses to user prompts. [Means for solving the problem]

[0004] According to one embodiment, a computer-implemented method is provided for optimizing a prompt session with a machine learning model ("MLM") trained on a training dataset. The computer-implemented method includes setting the MLM to a first temperature state, issuing a baseline prompt to the MLM in the first temperature state, and receiving a first response to the baseline prompt from the MLM in the first temperature state. The method also includes setting the MLM to a second temperature state, issuing the baseline prompt to the MLM in the second temperature state, and receiving a second response to the baseline prompt from the MLM in the second temperature state. A selected baseline response ("SBR") is selected from the first and second responses to the baseline prompt. In some embodiments, the SBR is provided as a baseline behavior to a reinforcement learning model ("RLM"), the RLM configured to: calculate a reward in response to the baseline behavior; and calculate a predicted temperature state based on the reward.

[0005] In one embodiment, a computer program product is provided for optimizing a prompt session with a machine learning model ("MLM") trained on a set of training data. The computer program product comprises a computer-readable storage medium having program instructions embodied thereon. The program instructions are executable by a processor to cause a computing device to set the MLM to a first temperature state, issue a baseline prompt to the MLM in the first temperature state, and receive a first response to the baseline prompt from the MLM in the first temperature state. The program instructions further cause the computing device to set the MLM to a second temperature state, issue the baseline prompt to the MLM in the second temperature state, and receive a second response to the baseline prompt from the MLM in the second temperature state. A selected baseline response ("SBR") is selected from the first and second responses to the baseline prompt. In some embodiments, the SBR is fed as a baseline behavior to a reinforcement learning model (“RLM”), which is configured to calculate a reward in response to the baseline behavior and to calculate a predicted temperature state based on the reward.

[0006] According to one embodiment, a computer system is provided for optimizing a prompt session with a machine learning model ("MLM") trained on a set of training data. The computer system comprises a processor, a computer-readable memory, a computer-readable tangible storage device, and program instructions stored on the computer-readable storage device for execution by the processor via the computer-readable memory, wherein the execution of the program instructions causes the computer system to perform a method. The method includes setting the MLM to a first temperature state, issuing a baseline prompt to the MLM in the first temperature state, and receiving a first response to the baseline prompt from the MLM in the first temperature state. The method also includes setting the MLM to a second temperature state, issuing the baseline prompt to the MLM in the second temperature state, and receiving a second response to the baseline prompt from the MLM in the second temperature state. A selected baseline response ("SBR") is selected from the first and second responses to the baseline prompt. The SBR is fed as a baseline behavior to a reinforcement learning model (“RLM”), which is configured to calculate a reward in response to the baseline behavior and to calculate a predicted temperature state based on the reward.

[0007] The techniques described herein may be implemented in a number of ways. Example implementations are provided below with reference to the following figures. [Brief explanation of the drawings]

[0008] The drawings are of exemplary embodiments. They do not depict all embodiments. Other embodiments may be used in addition or instead. To save space or for a more effective illustration, details that may be obvious or unnecessary may be omitted. Some embodiments may be practiced with additional components or steps and / or without all of the components or steps shown. When the same number appears in different drawings, it refers to the same or similar components or steps.

[0009] [Figure 1] 1 is a block diagram of a computer hardware platform including a prompt session optimization ("PSO") engine for efficiently and reliably optimizing prompt sessions with MLMs consistent with an exemplary embodiment.

[0010] [Figure 2] 2 is a block diagram of a reinforcement learning computing system including the PSO engine of FIG. 1 consistent with an exemplary embodiment.

[0011] [Figure 3] FIG. 10 is a graphical illustration of similarity for multidimensional vectors in the embedding space of an MLM, consistent with an example embodiment.

[0012] [Figure 4] FIG. 10 is a block diagram of a workflow for encoding search vectors and data vectors into an MLM embedding space and receiving responses to the search vectors from an MLM embedding spacer, consistent with an example embodiment.

[0013] [Figure 5] FIG. 4 is a diagram similar to FIG. 3 but illustrating similarities for search vectors associated with multiple data vectors embedded in an MLM embedding space, consistent with an exemplary embodiment.

[0014] [Figure 6] FIG. 1 is a three-dimensional view of three clusters of data vectors in an MLM embedding space, consistent with an example embodiment.

[0015] [Figure 7] FIG. 10 illustrates hot and cold responses to the prompt "what is the meaning of life?" consistent with an illustrative embodiment.

[0016] [Figure 8a] FIG. 10 illustrates a relatively low temperature sampling distribution for a sentence transformer encoder consistent with an exemplary embodiment.

[0017] [Figure 8b] FIG. 8b illustrates a relatively high temperature sampling distribution for the sentence transformer encoder of FIG. 8a, consistent with an exemplary embodiment.

[0018] [Figure 9a] FIG. 7 graphically illustrates search vectors in the embedding space of FIG. 6 with the MLM temperature set to 0, consistent with an example embodiment.

[0019] [Figure 9b] FIG. 9b is similar to FIG. 9a, but with the MLM temperature set to less than 0.5.

[0020] [Figure 9c] FIG. 9a is similar to FIG. 9a, but with the MLM temperature set to greater than 0.5.

[0021] [Figure 10] 1 is a flowchart illustrating a method for prompt session optimization consistent with exemplary embodiments.

[0022] [Figure 11]FIG. 11 illustrates an algorithm in the strategy block of FIG. 10 consistent with an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0023] In the following detailed description, by way of example, numerous specific details are set forth in order to provide a thorough understanding of the relevant teachings. It will be apparent, however, that the present teachings may be practiced without such details. In other instances, well-known methods, procedures, components, and / or circuits have been described at a relatively high-level, without detail, to avoid unnecessarily obscuring aspects of the present teachings.

[0024] According to one aspect of the present disclosure, a computer-implemented method for optimizing a prompt session with a machine learning model ("MLM") trained on a training dataset is provided. The computer-implemented method includes setting the MLM to a first temperature state, issuing a baseline prompt to the MLM in the first temperature state, and receiving a first response to the baseline prompt from the MLM in the first temperature state. The computer-implemented method further includes setting the MLM to a second temperature state, issuing the baseline prompt to the MLM in the second temperature state, and receiving a second response to the baseline prompt from the MLM in the second temperature state. A baseline response ("SBR") is selected from the first and second responses to the baseline prompt. The SBR is fed as a baseline behavior to a reinforcement learning model ("RLM"), the RLM configured to calculate a reward in response to the baseline behavior and to calculate a predicted temperature state based on the reward. A technical feature of this method is an improved user experience during the prompt session by adjusting the MLM for desired linguistic determinism in the MLM's responses to user prompts. A technical advantage of this method is improved temperature control that does not require parametric knowledge of the temperature settings and associated sampling probability distributions.

[0025] In one embodiment, the method further includes setting the MLM to a recursive temperature state corresponding to the temperature state of the SBR, issuing recursive prompts to the MLM in the recursive temperature state, and receiving a first response to the recursive prompt from the MLM in the recursive temperature state. The method further includes setting the MLM to the predicted temperature state, issuing the recursive prompt to the MLM in the predicted temperature state, and receiving a second response to the recursive prompt from the MLM in the predicted temperature state. A selected iterative response ("SIR") is selected from the first and second responses to the recursive prompt. The method further determines whether the SIR is satisfactory. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for the computing device.

[0026] In one embodiment, when the method determines that the SIR is satisfactory, it sets the MLM to a temperature state corresponding to the temperature state of the SIR. When the method determines that the SIR is unsatisfactory, it provides the SIR as a recurring action to the RLM, which is configured to recalculate the reward in response to the recurring action and recalculate the predicted temperature state based on the recalculated reward. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device.

[0027] In one embodiment, the method further includes setting the MLM to a subsequent iterative temperature state corresponding to the temperature state of the SIR, issuing subsequent iterative prompts to the MLM at the subsequent iterative temperature state, and receiving a first response to the subsequent iterative prompt from the MLM at the subsequent iterative temperature state. The method further includes setting the MLM to the recalculated predicted temperature state, issuing subsequent iterative prompts to the MLM at the recalculated predicted temperature state, and receiving a second response to the subsequent iterative prompt from the MLM at the recalculated predicted temperature state. A subsequent SIR is selected from the first and second responses to the subsequent iterative prompts, and determining whether the subsequent SIR is satisfactory. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's responses to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for the computing device.

[0028] In one embodiment, if the method determines that the subsequent SIR is satisfactory, it sets the MLM to a temperature state corresponding to the temperature state of the subsequent SIR. If the method determines that the subsequent SIR is unsatisfactory, it provides the subsequent SIR as a subsequent recurring action to the RLM, which is configured to recalculate the reward in response to the subsequent recurring action and recalculate the predicted temperature state based on the most recently recalculated reward. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device.

[0029] In one embodiment, the training dataset includes a plurality of data vectors in the embedding space of the MLM. A technical feature of this method is an improved user experience during a prompting session by tuning an MLM for desired linguistic determinism in its responses to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for computing devices.

[0030] In one embodiment, the baseline prompt, the repetitive prompt, and the subsequent repetitive prompt each include a search vector in the embedding space of the MLM. A technical feature of this method is an improved user experience during a prompting session by adjusting the MLM for desired linguistic determinism in the MLM's responses to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device.

[0031] In one embodiment, the iterative temperature state and the subsequent iterative temperature state each include a probability distribution around one of the search vectors in the embedding space of the MLM. A technical feature of this method is an improved user experience during a prompt session by adjusting an MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device.

[0032] In one embodiment, the iterative temperature state includes a probability distribution around the SBR and the subsequent iterative temperature state includes a probability distribution around the SIR. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device.

[0033] According to one aspect of the present disclosure, a computer program product is provided for optimizing a prompt session with a machine learning model ("MLM") trained on a set of training data. The computer program product comprises a computer-readable storage medium having program instructions embodied thereon. The program instructions are executable by a processor to cause a computing device to set the MLM to a first temperature state, issue a baseline prompt to the MLM in the first temperature state, and receive a first response to the baseline prompt from the MLM in the first temperature state. The program instructions further cause the computing device to set the MLM to a second temperature state, issue the baseline prompt to the MLM in the second temperature state, and receive a second response to the baseline prompt from the MLM in the second temperature state. A selected baseline response ("SBR") is selected from the first and second responses to the baseline prompt. The SBR is fed as a baseline behavior to a reinforcement learning model ("RLM"), which is configured to calculate a reward in response to the baseline behavior and to calculate a predicted temperature state based on the reward. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device. Another technical advantage of this method is improved processing power and reduced processing overhead for a computing device.

[0034] In one embodiment, the program instructions executable by the processor further cause the computing device to: set the MLM to a repetitive temperature state corresponding to the temperature state of the SBR; issue a repetitive prompt to the MLM at the repetitive temperature state; and receive a first response to the repetitive prompt from the MLM at the repetitive temperature state. The program instructions further cause the computing device to set the MLM to the predicted temperature state; issue the repetitive prompt to the MLM at the predicted temperature state; and receive a second response to the repetitive prompt from the MLM at the predicted temperature state. A selected repetitive response ("SIR") is selected from the first and second responses to the repetitive prompt, and it is determined whether the SIR is satisfactory. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions.

[0035] In one embodiment, the program instructions executable by the processor further cause the computing device to, upon determining that the SIR is satisfactory, set the MLM to a temperature state corresponding to the temperature state of the SIR. Upon determining that the SIR is unsatisfactory, the computing device provides the SIR as a recursive action to the RLM, which is configured to recalculate the reward in response to the recursive action and recalculate the predicted temperature state based on the recalculated reward. A technical advantage of this apparatus is improved processing power and reduced processing overhead for a computing device.

[0036] In one embodiment, the program instructions executable by the processor further cause the computing device to: set the MLM to a subsequent iterative temperature state corresponding to the temperature state of the SIR; issue a subsequent iterative prompt to the MLM at the subsequent iterative temperature state; and receive a first response to the subsequent iterative prompt from the MLM at the subsequent iterative temperature state. The computing device further sets the MLM to the recalculated predicted temperature state, issues the subsequent iterative prompt to the MLM at the recalculated predicted temperature state, and receives a second response to the subsequent iterative prompt from the MLM at the recalculated predicted temperature state. A subsequent SIR is selected from the first and second responses to the subsequent iterative prompt, and it is determined whether the subsequent SIR is satisfactory. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's responses to user prompts. A technical advantage of this method is improved temperature control that does not require parametric knowledge of the temperature settings and associated sampling probability distributions.

[0037] In one embodiment, the program instructions executable by the processor further cause the computing device to, upon determining that the subsequent SIR is satisfactory, set the MLM to a temperature state corresponding to the temperature state of the subsequent SIR. Upon determining that the subsequent SIR is unsatisfactory, the computing device provides the subsequent SIR as a subsequent repetitive action to the RLM, which is configured to recalculate the reward in response to the subsequent repetitive action and recalculate the predicted temperature state based on the most recent recalculated reward. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions.

[0038] In one embodiment, the training data set includes a plurality of data vectors in the embedding space of the MLM. The baseline prompt, the recursive prompt, and the subsequent recursive prompt each include a search vector in the embedding space of the MLM. The recursive temperature states and the subsequent recursive temperature states each include a probability distribution around one of the search vectors in the embedding space of the MLM. A technical feature of this method is an improved user experience during a prompting session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions.

[0039] According to one aspect of the present disclosure, a computer system for optimizing a prompt session with a machine learning model ("MLM") trained on a set of training data is provided. The computer system includes a processor, a computer-readable memory, a computer-readable tangible storage device, and program instructions stored on the storage device for execution by the processor via the computer-readable memory. Execution of the program instructions causes the computer system to perform a method comprising: setting the MLM to a first temperature state; issuing a baseline prompt to the MLM at the first temperature state; and receiving a first response to the baseline prompt from the MLM at the first temperature state. The method further includes setting the MLM to a second temperature state; issuing the baseline prompt to the MLM at the second temperature state; and receiving a second response to the baseline prompt from the MLM at the second temperature state. A selected baseline response ("SBR") is selected from the first and second responses to the baseline prompt. The SBR is fed as a baseline behavior to a reinforcement learning model ("RLM"), which is configured to calculate a reward in response to the baseline behavior and to calculate a predicted temperature state based on the reward. A technical feature of this system is improved computational speed and reduced processing overhead. A technical feature of this method is an improved user experience during prompt sessions by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions.

[0040] In one embodiment, the system includes setting the MLM to a repetitive temperature state corresponding to the temperature state of the SBR, issuing repetitive prompts to the MLM at the repetitive temperature state, and receiving a first response to the repetitive prompt from the MLM at the repetitive temperature state. The system further sets the MLM to the predicted temperature state, issues the repetitive prompt to the MLM at the predicted temperature state, and receives a second response to the repetitive prompt from the MLM at the predicted temperature state. A selected repetitive response ("SIR") is selected from the first and second responses to the repetitive prompt, and it is determined whether the SIR is satisfactory. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions.

[0041] In one embodiment, when the system determines that the SIR is satisfactory, it sets the MLM to a temperature state corresponding to the temperature state of the SIR. When the system determines that the SIR is not satisfactory, it provides the SIR as a recursive action to the RLM, which is configured to recalculate the reward in response to the recursive action and recalculate the predicted temperature state based on the recalculated reward. The system further sets the MLM to a subsequent recursive temperature state corresponding to the temperature state of the SIR, issues a subsequent recursive prompt to the MLM at the subsequent recursive temperature state, and receives a first response to the subsequent recursive prompt from the MLM at the subsequent recursive temperature state. The system further sets the MLM to the recalculated predicted temperature state, issues the subsequent recursive prompt to the MLM at the recalculated predicted temperature state, and receives a second response to the subsequent recursive prompt from the MLM at the recalculated predicted temperature state. A subsequent SIR is selected from the first and second responses to the subsequent repetitive prompts. It is determined whether the subsequent SIR is satisfactory. If the subsequent SIR is determined to be satisfactory, the system further sets the MLM to a temperature state corresponding to the temperature state of the subsequent SIR. If the subsequent SIR is determined to be unsatisfactory, the system provides the subsequent SIR as a subsequent repetitive action to the RLM, which is configured to recalculate the reward in response to the subsequent repetitive action and recalculate the predicted temperature state based on the most recently recalculated reward. A technical advantage of this system is improved processing power and reduced processing overhead for a computing device. A technical feature of this method is an improved user experience during a prompt session by adjusting the MLM for desired linguistic determinism in the MLM's response to user prompts. A technical advantage of this method is improved temperature control without requiring parametric knowledge of temperature settings and associated sampling probability distributions.

[0042] Although terms such as first, second, third, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element could be referred to as a second element, and similarly, a second element could be referred to as a first element, without departing from the scope of the example embodiments. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0043] It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the spirit and scope as defined by the claims. The description of the embodiments is not intended to be limiting. In particular, elements of the embodiments described below may be combined with elements of different embodiments.

[0044] Various aspects of the present disclosure are described through text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. For any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, again depending on the technology involved, two operations shown in successive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in an at least partially overlapping manner.

[0045] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any set of one or more storage media (also referred to as "media") collectively included in a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices (such as punch cards or pits / lands formed on a major surface of a disk), or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in this disclosure, is not to be construed as storage in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals communicated through wires, and / or other transmission media.As will be appreciated by those skilled in the art, data is typically moved at some infrequent time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but the above does not make the storage device temporary, as the data is not temporary while it is stored.

[0046] 1 , environment 100 includes an example of an environment for the execution of at least a portion of the computer code involved in performing the methods of the present invention discussed herein, including a prompt session optimization (“PSO”) engine 180. In addition to PSO engine 180, computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, computer 101 includes a set of processors 110 (including processing circuitry 120 and cache 121), a communications fabric 111, volatile memory 112, persistent storage 113 (including an operating system 122 and PSO engine 180 as identified above), a set of peripheral devices 114 (including a set of user interface (UI) devices 123, storage 124, and a set of Internet of Things (IoT) sensors 125), and a network module 115. The remote server 104 includes a remote database 130. The public cloud 105 includes a gateway 140, a cloud orchestration module 141, a set of host physical machines 142, a set of virtual machines 143, and a set of containers 144.

[0047] Computer 101 may take the form of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch or other wearable computer, a mainframe computer, a quantum computer, or any other form of computer or mobile device now known or later developed that is capable of executing programs, accessing a network, or querying a database, such as remote database 130. As is well understood in the art of computer technology, and depending on the technology, execution of a computer-implemented method may be distributed among multiple computers and / or among multiple locations. While in this presentation of computing environment 100, to keep the presentation as concise as possible, the detailed discussion focuses on a single computer, specifically computer 101. Although computer 101 is not depicted in the cloud of FIG. 1 , it may be located in a cloud. However, computer 101 is not required to reside within a cloud except to any extent that may be expressly indicated.

[0048] Processor set 110 includes one or more computer processors of any type now known or later developed. Processing circuitry 120 may be distributed across multiple packages, e.g., multiple coordinated integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by threads or cores executing on processor set 110. Cache memory is typically organized into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all caches for a processor set may be located “off-chip.” In some computing environments, processor set 110 may be designed to operate with qubits and perform quantum computing.

[0049] Computer-readable program instructions are typically loaded onto computer 101 to cause processor set 110 of computer 101 to execute a series of operational steps, thereby realizing a computer-implemented method, whereby the instructions so executed instantiate the method specified in the flowcharts and / or descriptions of the computer-implemented method contained herein (collectively referred to as the "methods of the present invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by processor set 110 to control and direct the execution of the methods of the present invention. In computing environment 100, at least some of the instructions for executing the methods of the present invention may be stored in PSO engine 180 in persistent storage 113.

[0050] Communications fabric 111 is the signal-conducting pathway that allows various components of computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive pathways, such as those that make up buses, bridges, physical input / output ports, and the like. Other types of signal communication pathways may be used, such as fiber optic and / or wireless communication pathways.

[0051] Volatile memory 112 may be any type of volatile memory now known or later developed. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, volatile memory 112 is characterized by random access, although this is not required unless expressly indicated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 101.

[0052] Persistent storage 113 is any form of non-volatile storage for a computer, now known or later developed. The non-volatility of this storage means that stored data remains regardless of whether power is supplied to computer 101 and / or to persistent storage 113 directly. Persistent storage 113 may be read-only memory (ROM), but typically at least a portion of persistent storage allows data to be written, data to be deleted, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 122 may take several forms, such as various known proprietary operating systems or open-source Portable Operating System Interface-type operating systems employing a kernel. The code included in PSO engine 180 typically includes at least a portion of the computer code involved in performing the methods of the present invention.

[0053] The peripheral device set 114 includes a set of peripheral devices of the computer 101. Data communication connections between the peripheral devices and other components of the computer 101 may be implemented in various ways, such as Bluetooth® connections, Near-Field Communication (NFC) connections, connections made by cable (such as a universal serial bus (USB)-type cable), insertion-type connections (e.g., a secure digital (SD) card), connections made over a local area communication network, and even connections made over a wide area network such as the Internet. In various embodiments, the UI device set 123 may include components such as a display screen, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. The storage 124 may be external storage, such as an external hard drive, or insertable storage, such as an SD card. The storage 124 may be persistent and / or volatile. In some embodiments, storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (e.g., computer 101 locally stores and manages large databases), this storage may be provided by a peripheral storage device designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. IoT sensor set 125 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0054] Network module 115 is a collection of computer software, hardware, and firmware that enables computer 101 to communicate with other computers over WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for implementing the methods of the present invention can be downloaded to computer 101 from an external computer or external storage device, typically through a network adapter card or network interface included in network module 115.

[0055] WAN 102 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology for communicating computer data, now known or later developed. In some embodiments, WAN 102 may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include copper transmission cables, optical fiber transmissions, wireless transmissions, and computer hardware such as routers, firewalls, switches, gateway computers, and edge servers.

[0056] End-user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and may take any of the forms discussed above in connection with computer 101. EUD 103 typically receives useful and useful data from the operation of computer 101. For example, in the hypothetical case where computer 101 is designed to provide recommendations to the end user, the recommendations would typically be communicated from network module 115 of computer 101 over WAN 102 to EUD 103. In this manner, EUD 103 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 103 may be a client device such as a thin client, a heavy client, a mainframe computer, a desktop computer, etc.

[0057] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 may be controlled and used by the same entity that operates computer 101. Remote server 104 represents a machine that collects and stores useful and useful data for use by other computers, such as computer 101. For example, in the hypothetical case where computer 101 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to computer 101 from remote database 130 of remote server 104.

[0058] Public cloud 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer functionality, particularly data storage (cloud storage) and computing power, without direct active management by users. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct active management of public cloud 105's computing resources is performed by computer hardware and / or software in cloud orchestration module 141. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments (VCEs) running on various computers that comprise host physical machine set 142, the universe of physical computers in and / or available to public cloud 105. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It is understood that these VCEs may be stored as images and can be transferred among and between various physical machine hosts, either as images or after instantiation of the VCEs. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCE, and manages active instantiations of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that enables public cloud 105 to communicate over WAN 102.

[0059] Some further description of a virtualized computing environment (VCE) is now provided. A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from the image. Two well-known types of VCE are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows the existence of multiple isolated user space instances called containers. These isolated user space instances typically behave as actual computers from the perspective of programs running in them. A computer program running on a normal operating system can utilize all of the computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and of the devices assigned to the container; this feature is known as containerization.

[0060] A private cloud 106 is similar to a public cloud 105, except that the computing resources are available only for use by a single enterprise. While the private cloud 106 is shown as communicating with the WAN 102, in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., private, community, or public cloud types), often each implemented by a different vendor. While each of the multiple clouds remains a separate, discrete entity, the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the constituent clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0061] Thus, a computing system generally facilitates signal processing in accordance with one or more embodiments illustratively described herein. For example, the signal processing may be related to an artificial neural network system, an artificial intelligence system, a collaborative filtering system, a recommendation system, a signal processing system, a word embedding system, a topic model system, an image processing system, a data analysis system, a media content system, a video streaming service system, an audio streaming service system, an e-commerce system, a social network system, an internet search system, an online advertising system, a medical system, an industrial system, a manufacturing system, and / or another digital system. The system may employ hardware and / or software to solve problems that are highly technical in nature, not abstract, and cannot be performed as a set of mental acts by a human.

[0062] For simplicity of explanation, specialized computer-implemented methods are depicted and described as a series of acts. It should be understood and appreciated that the subject innovation is not limited by the depicted acts and / or by the order of acts. That is, for example, acts may be performed in various orders and / or concurrently, with other acts not shown and described herein. Moreover, not all explicitly disclosed acts may be required to implement a computer-implemented method in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that a computer-implemented method may alternatively be represented as a series of interrelated states via a state diagram or events. In addition, it should be further appreciated that the computer-implemented methods disclosed hereinafter and throughout this specification can be stored on an article of manufacture to facilitate transporting and transferring such computer-implemented methods to a computer. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from a computer-readable device or storage medium.

[0063] The system may employ hardware and / or software to solve problems that are highly technical in nature, not abstract, and cannot be implemented as a set of mental acts by a human. One or more embodiments of the system may also provide technical improvements to computer processing units associated with control signal processing by improving the processing performance of the computer processing units, reducing computational errors and computational bottlenecks in the computer processing units, improving the processing efficiency of the computer processing units, and / or reducing the amount of time for the computer processing units to execute computer processes.

[0064] In an exemplary embodiment of this disclosure, Figure 2 diagrammatically illustrates a reinforcement learning computing environment 200 in which a PSO engine 180 (Figure 1) is configured to automatically guide a user in optimally adjusting the temperature of a machine learning model (MLM) 202, such as, but not limited to, a foundation model or a large-scale language model, during a prompt session with the MLM 202. While the disclosed embodiments are directed to applying machine learning to natural language processing, alternative equivalent embodiments may be directed to other types of machine learning, such as, but not limited to, those directed to image classification, computer vision, gaming, and the like.

[0065] The computing environment 200 can include collecting and conditioning datasets from sample data 204 stored in computer memory. The quality of training of the MLM 202 depends on the quantity and quality of the sample data 204. On one side, the machine learning (“ML”) pipeline 206 can function to parse training datasets from the stored sample data 204, such as for rapid parallel training trials on any desired number of MLMs 202. On the other side, the ML pipeline 206 can function to feed training datasets to rapid parallel training trials running on one or more MLMs 202. In between, the data pipeline 206 can function to preprocess the datasets so that they are in a suitable format for reliable execution on the MLM models 202.

[0066] In one aspect, the sample data 204 can be stored in one or more computer memories. Extracting a dataset from the sample data 204 can involve many formatting operations, such as concatenating data tables together and the like. Preprocessing the dataset can involve many transformation operations, such as resizing images, decoding video, augmenting data, and the like. Preprocessing can include multiplexing the feature data stream and the label data stream into a combined, combined data stream for training trials. In one example, the features include video images, the labels can be cross-identification for the images and the like. This label processing can further include converting integer values ​​to tensor values ​​for performing classification modeling. Duplicate data can be discarded, and incomplete or erroneous data can be supplemented and / or corrected. The sample data 204 can also be randomized before parsing it to reduce the adverse effects of sampling variation. The sample data 204 can also be split into multiple mutually exclusive portions. The largest portion is typically for the training data set, while smaller portions can be used for the test data set, the adjustment data set, and the like.

[0067] MLM 202 can be any of several different machine models that can be used with machine learning. Generally, machine models well suited for natural language processing include bidirectional encoder representations from transformers (“BERT”) and generative pre-trained transformers (“GPT”). Other ML resources in computing environment 200 can employ any suitable ML-based, statistical-based, and / or probability-based techniques. For example, ML resources can employ expert systems, fuzzy logic, support vector machines (“SVMs”), hidden Markov models (HMMs), greedy search algorithms, rule-based systems, Bayesian models (e.g., Bayesian networks), neural networks, other nonlinear training techniques, data fusion, utility-based analysis systems, and the like. For example, the ML resources may execute a set of clustering ML calculations, e.g., k nearest neighbor ("kNN") and / or approximate nearest neighbor ("ANN") calculation blocks, a set of linear and / or logistic regression ML calculations, a set of decision tree ML calculations, a set of random forest ML calculations, a set of regression tree ML calculations, a set of least squares ML calculations, a set of instance-based ML calculations, a set of support vector regression ML calculations, a set of k-means ML calculations, a set of spectral clustering ML calculations, a Gaussian mixture model ML calculation, a set of regularization ML calculations, a set of rule ML calculations, a set of Bayesian ML calculations, a set of deep Boltzmann calculations, a set of deep belief network calculations, a set of convolutional neural network calculations, a set of stacked autoencoder calculations, and / or a set of different ML calculations.

[0068] The hyperparameters of the MLM 202 can be adjusted to optimize performance. One such hyperparameter, specifically the temperature hyperparameter of the MLM 202, is addressed in this disclosure. Other hyperparameters can be adjusted as well, including, for example, the number of hidden layers in the neural network, the learning rate, and the like.

[0069] The MLM 202 can have an encoder 208 to convert sample data 204 in many forms, such as documents, text, classifications and subclassifications, and the like, into data vectors in the Euclidean embedding space 212 of the MLM 202. The encoder 208 can include multiple sentence transformers configured to preserve the semantic content of the sample data 204 in the complex multi-dimensional data vectors.

[0070] Figure 2 shows a simplified two-dimensional representation of several encoded data vectors in an embedding space 212. The closer adjacent data vectors are to each other in the embedding space 212, the more similar the linguistic content of the documents they represent. For example, the vector embedding may indicate that the linguistic content of doc1 compared to doc2 is more similar than that of doc1 compared to doc8. Figure 3 is a simplified illustration of different similarities between three vectors in a multidimensional embedding space 212. The similarity of two vectors is related to the dot product of the angles between them, which can be calculated in terms of cosine similarity, for example.

[0071] Returning to Figure 2, groups of similar documents form data vector clusters. In particular, in this example, five data vectors form a first cluster (doc1, doc2, doc3, doc4, doc5), and three data vectors form a second cluster (doc6, doc7, doc8). The overall proximity of all data vectors in a cluster collectively defines the cluster density. That is, the first cluster (doc1-doc5) has a lower cluster density than the second cluster (doc6-doc8).

[0072] The vector store 213 can be used to identify, store, and retrieve data vectors so that they need to be calculated only once for a computation. This reduces computational overhead and increases data throughput. The vector store 213 can also perform valuable resource functions such as calculating distances between data vectors, approximate nearest neighbor ("ANN") calculations on data vectors, and the like. In more complex applications, the vector store 213 can be a vector database, such as commercially available vector databases sold under the trademarks Pinecone®, Weaviate®, Chroma®, and others. In less complex applications, the vector store 213 can be a vector library, such as commercially available vector libraries sold under the trademarks FAISS®, ScaNN®, ANNOY®, and others.

[0073] The PSO engine 180 may include a human interface 214 through which a user may input prompts to query the MLM 202 for information. The PSO engine 180 may also include a reinforcement learning ("RL") model 216, sometimes referred to as a reward model. In the context of reinforcement learning, the MLM 202 is sometimes referred to as an agent. The RL model 216 passes a reward and state to the MLM 202, which in return returns an action to the RL model 216. The state refers to the current configuration of the embedding space (or environment) 212. The reward provides feedback information from the embedding space 212 and is used to evaluate actions taken by the MLM 202. Generally, the RL model 216 generates a positive reward to inform the MLM 202 that it has taken actions in accordance with the desired objective of the RL model 216. Ultimately, the goal of the RL model 216 is to guide the MLM 202 in taking actions that maximize a reward function.

[0074] An example of an RL model algorithm that can be used is a Markov decision process (MDP), as generally shown in Figure 2. Other RL techniques include Monte Carlo, temporal difference learning, Q-learning, deep Q network (DQN), state-action-reward-state-action (SARSA), distributed cluster-based multi-agent bidding solution (DCMAB), and the like.

[0075] The computing environment 200 is therefore particularly configured to provide technical improvements to data systems, machine learning systems, artificial intelligence systems, and data analysis systems, such as, but not limited to, data classification systems, data regression systems, data batch processing and clustering systems, and the like. The prompt session optimization of the present disclosure may further provide one or more inferences, provide one or more predictions, and / or determine one or more relationships between data. For example, optimization as described herein may model one or more inferences and / or predictions and / or determine one or more relationships between variables analyzed in the data. The machine learning predicts outputs, e.g., probabilities, from historical data. Such optimized machine learning aids in downstream decision-making, even automating such downstream decision-making.

[0076] Thus, the computing environment 200 generally facilitates optimizing machine learning in accordance with one or more embodiments illustratively described herein. For example, the optimization may relate to a rapid parallel training trial system, an artificial intelligence system, a collaborative filtering system, a recommendation system, a signal processing system, a word embedding system, a topic model system, an image processing system, a data analysis system, a media content system, a video streaming service system, an audio streaming service system, an e-commerce system, a social network system, an internet search system, an online advertising system, a medical system, an industrial system, a manufacturing system, and / or another digital system. The system may employ hardware and / or software to solve problems that are highly technical in nature, not abstract, and cannot be implemented as a set of mental acts by a human.

[0077] For simplicity of explanation, specialized computer-implemented methods and computer program products are depicted and described as a series of acts. It is to be understood and appreciated that the subject innovation is not limited by the depicted acts and / or by the order of acts. That is, for example, acts may be performed in various orders and / or simultaneously with other acts not shown and described herein. Moreover, not all explicitly disclosed acts are necessarily required to implement computer-implemented methods and products in accordance with the disclosed subject matter. In addition, those skilled in the art will understand and appreciate that computer-implemented methods and products may alternatively be represented as a series of interrelated states via state diagrams or events. In addition, it should be further appreciated that the computer-implemented methods disclosed herein and throughout this specification can be stored on an article of manufacture to facilitate transporting and transferring such computer-implemented methods to a computer. The term article of manufacture, as used herein, is intended to encompass a computer program accessible from a computer-readable device or storage medium.

[0078] The system may employ hardware and / or software to solve problems that are highly technical in nature, not abstract, and cannot be implemented as a set of mental acts by a human. One or more embodiments of the system may also provide technical improvements to a computer processing unit associated with an ML process by improving the processing performance of the computer processing unit, reducing computational bottlenecks in the computer processing unit, improving the processing efficiency of the computer processing unit, and / or reducing the amount of time it takes the computer processing unit to perform the ML process.

[0079] Figure 4 is a high-level block diagram of a workflow for generating vector embeddings and exploring them. The ML pipeline 206 supplies the MLM 202 with preprocessed sample data 204 (Figure 2) in many forms, such as documents, text, categories, and the like. The MLM 202 encodes the sample data 204 to generate data vectors 302 that can be identified and referenced in an index 304. The index 304 is a data structure that stores metadata for the data vectors 302 in computer memory, such as random access memory. In this manner, the vector store 213 is configured to enable fast, reliable, and low-overhead computations using the stored data vectors 302.

[0080] After the data vector 302 is stored in the index 304, a user can input a search prompt to query the MLM 202 via the human interface 214. The MLM 202 encodes the user's prompt into a search vector in a manner similar to how it encodes the data vector 302. The vector store 212 can then execute a search algorithm with the aid of resources 308 stored in computer memory, such as calculated vector distances, ANN values, and the like, to generate a response 310 to the user's prompt. FIG. 5 is similar to FIG. 3 but further illustrates the MLM 202 encoding the user prompt "that is a joyful {blank}" to generate a search vector 502 that is more closely aligned with the first cluster of data vectors (doc1-doc5 in FIG. 2) than with the second cluster of data vectors (doc6-doc8 in FIG. 2).

[0081] 6 is a more complex and realistic three-dimensional view of the embedding space 212 (FIG. 2) containing three vector clusters 602, 604, 606 of data vectors, each represented by a triangle, a square, and a circle, respectively. The density of different clusters in this embedding space 212 can leverage the temperature hyperparameter of the MLM 202. Generally, the temperature value can be varied to alter the degree of linguistic determinism in the response from the MLM 202 to user prompts. Temperature can be set to a desired value generally within the range of 0 to 1, and in some cases, 0 to 2. For the purposes of this discussion, the former range will be used.

[0082] Higher temperature settings generally result in less deterministic responses from the MLM 202 by making the responses more random and creative. This is preferable in use cases involving creative endeavors such as story writing, poetry writing, title or tagline generation, and the like. Conversely, lower temperature settings generally result in more deterministic responses from the MLM 202 by making the responses more predictable and repeatable. This is preferable in use cases involving more fact-based endeavors such as answering questions, researching historical facts, writing technical instructions, and the like. Figure 7 shows different deterministic responses from the MLM 202 in response to the same user prompt, "What is the meaning of life?" Less deterministic responses were obtained at a higher MLM temperature of 0.9, and more deterministic responses were obtained at a lower MLM temperature of 0.3.

[0083] Varying the MLM temperature results in a modification of the probability distribution of samples in the embedding space 212 when the MLM 202 is encoding the search vector. In some embodiments, for example, the temperature parameter T modifies the Softmax sampling function as the denominator of the exponent in a logarithmic function:

[0084]

number

[0085] Figures 8a and 8b show how this temperature-dependent function probabilistically affects the sampling distribution during vector encoding. At the low temperature setting in Figure 8a, the MLM's sentence transformer has a 30% chance of selecting the best token "sunny" for the next word, and a 5% chance of selecting the second-best token "rainy." However, at the high temperature setting in Figure 8b, "sunny" is only 10% likely, and "rainy" is only 2% less likely. Also, at the high temperature in Figure 8b, there will be more tokens in the sampling pool because they all eventually sum to 1.0.

[0086] 9a, 9b, and 9c are simplified two-dimensional diagrams showing search vectors 306 centered around a region of embedding space 212 (FIG. 6) in which three vector clusters 602, 604, and 606 are also embedded. These diagrams show the different probability sampling that results from setting the temperature value to zero in FIG. 9a, to a value less than 0.5 in FIG. 9b, and to a value greater than 0.5 in FIG. 9c. In reality, the probability distribution is spherical in embedding space 212, but the two-dimensional diagrams in FIGS. 9a, 9b, and 9c usefully help the reader understand how different temperature settings affect the sampling distribution.

[0087] A temperature setting of zero in FIG. 9a typically serves as the baseline for this comparison in that it will result in sampling only the most probable token, i.e., the nearest neighbor to the search vector 306, which in this example is data vector 6041. Increasing the temperature in FIG. 9b to a lower temperature value, e.g., a value less than 0.5, increases the sampling distribution to include four data vectors 6041, 6042, 6043, and 6044. In this example, all data vectors in the sampling distribution in FIG. 9b are members of the same vector cluster 604. This makes the response less deterministic than in FIG. 9a, but only to the extent that the response becomes limited to the linguistic content of only one vector cluster 604. Increasing the temperature in FIG. 9c to a higher temperature value, e.g., a value greater than 0.5, increases the sampling distribution even further to include 13 data vectors. In this example, the data vectors in the high temperature sampling distribution of Figure 9c include all members of three vector clusters 6021-6024, 6041-6046, and 6061-6063, which makes the response relatively less deterministic and more creative.

[0088] 10 is a flowchart illustrating an exemplary inventive method 1000 for optimizing prompt sessions with an MLM after it has been trained on a training dataset. Method 1000 begins by block 1002, which issues baseline prompts to the MLM in each of a first temperature state and a second temperature state. SBR block 1004 sets the MLM to the first temperature state. The baseline prompts are issued to the MLM in the first temperature state. A first response to the baseline prompt is received from the MLM in the first temperature state. Block 1006 sets the MLM to the second temperature state. The baseline prompt is issued to the MLM in the second temperature state. A second response to the baseline prompt is received from the MLM in the second temperature state.

[0089] The SBR block 1004 selects a selected baseline response ("SBR") from the first and second responses to the baseline prompt. This selection can be made by a human user via a computer interface, or the selection can be performed automatically. The RL model block 1010 provides the SBR as a baseline behavior for a reinforcement learning ("RL") model.

[0090] The RL model controls the policy block 1012 to calculate a reward in response to the baseline behavior. A worthy objective of the RL model is to predict the temperature in a way that is satisfactory to the user and as easy and fast as possible. While any algorithm that operates according to that objective can be used, FIG. 11 illustrates a strategy block 1012 that calculates a reward R in response to the baseline behavior and predicts the predicted temperature state T based on the reward. p 11 shows an exemplary computer method 1100 for calculating

[0091] Method 1100 begins with LSR block 1104, which presents the user's last selected response ("LSR"). The LSR corresponds to the temperature state when the MLM generated the user-selected response. In FIG. 10, it is labeled "Selected Temperature," and in FIG. 11, it is labeled "T" based on that correspondence. s The method is labeled T s is the existing temperature T prior to LSR. e At this point in the method 1000, which is the first pass through the RL model 1010, the LSR 1104 may be the SBR 1004 (FIG. 10). Block 1106 compares the LSR block 1004 with a low existing temperature value T e rather than a high temperature value T s If it was selected, reward block 1108 may set reward R to 1. For this first pass, T ecan be set to any desired reference value, such as, but not limited to, a grid temperature of 0.5. Similarly, block 1110 determines whether LSR block 1104 has a high existing temperature value T e Instead of a lower temperature value T s If so, reward block 1112 may set reward R to −1. Block 1114 determines whether LSR block 1104 has selected an existing temperature value T e The temperature value T is equal to s If so, the reward block 1116 may set the reward R to 0.

[0092] The bias function block 1118 is s Appropriate reward R blocks 1108, 1112, 1116 can be applied to incrementally adjust T. For example, and without limitation, the reward can be s It can act as a weighting function applied to a predetermined increment for adjusting T. That is, in this example, if the predetermined increment is 0.1 and the reward is 1, then T s The incremental adjustment to T may be 0.1. Conversely, if the reward is -1, the incremental adjustment may be -0.1, and if the reward is 0, the incremental adjustment may be 0. Block 1120 applies the bias function block 1118 to T s , and the predicted temperature T p is derived.

[0093] Returning to FIG. 10, strategy block 1012 determines two iterative temperature states: predicted temperature state T p and the selected temperature state T s Block 1016 controls block 1014 in issuing repetitive prompts in each of the s , which is the temperature state corresponding to the temperature state of the SBR 1004 at this point in the method. s The first response is issued to the MLM at the iterative temperature state T sBlock 1018 receives the temperature data from the MLM at the predicted temperature state T p A recursive prompt is issued to the MLM in the predicted temperature state. A second response to the recursive prompt is received from the MLM in the predicted temperature state.

[0094] Block 1020 selects a selected iterative response ("SIR") from the first and second responses to the iterative prompt. This selection can be made by a human user, or the selection can be automated using any desired logic. Control then passes to block 1022, which selects the temperature state T corresponding to SIR 1020. s Determine whether the is satisfactory to the user.

[0095] Temperature state T s is satisfied, then block 1024 sets the MLM temperature to T s Block 1024 may be used to set T for later reference or recall, e.g., to apply Ts in different prompt sessions and / or with different MLMs. s can also be stored in computer memory.

[0096] Conversely, if block 1022 determines that SIR 1020 is not satisfactory, then counter 1026 increments the environment for a subsequent pass through RL model 1010. Counter block 1026 can provide SIR 1020 to RL model 1010 in a feedback loop as an iterative action. In this and all other subsequent passes, the most recent SIR 1020 populates LSR block 1104 in FIG. 11. Similarly, the temperature state T corresponding to the most recent SIR 1020 spopulates the selected temperature block 1016 in Figure 10. In other words, in this first subsequent pass, the SIR 1020 replaces the SBR 1004, and in all subsequent passes, the most recent SIR 1020 is adopted. This causes the RL model 1010 to continually recalculate the reward in response to the most recent repetitive behavior and to select the predicted temperature T based on the most recent recalculated reward. p Iterative feedback is provided to recalculate

[0097] In this first subsequent pass through the RL model 1010, the strategy block 1012 generates two subsequent iterative temperature states: the recalculated predicted temperature state T p and the subsequent iterative temperature state T corresponding to the most recent SIR1020 temperature state. s At each of the subsequent iterations, block 1014 controls the issuance of subsequent iteration prompts. Block 1016 sets the MLM to the subsequent iteration temperature state corresponding to the most recent SIR 1020 temperature state. s Subsequent repeated prompts may be issued to the MLM at the subsequent repeated temperature state T s The MLM then receives a first response to subsequent repeated prompts from the MLM at 1004. The MLM then receives a recalculated predicted temperature state T based on the SIR 1020 instead of the SBR 1004. p Subsequent repeated prompts will be set to the recalculated predicted temperature state T p The recalculated predicted temperature state T p A second response to the subsequent repetitive prompt is received from the MLM at The SIR selects a subsequent SIR from the first and second responses to the subsequent repetitive prompt. Block 1022 can then determine whether the subsequent SIR 1022 is satisfactory to the user.

[0098] Upon determining that the subsequent SIR 1020 is satisfactory, block 1024 updates the MLM temperature state to the most recent T corresponding to the satisfactory subsequent SIR 1020. sAt this point, method 1000 has successfully optimized the temperature parameters for continuing the prompt session with the MLM. Block 1024 can store the optimized temperature in computer memory for later reference or use.

[0099] Conversely, if the user determines that the most recent subsequent SIR 1020 is not satisfactory, the counter 1026 can again increment in a feedback loop that provides the SIR 1020 as a repetitive action to the RL model 1010. By connecting the SIR 1020 to the LSR block 1104 of FIG. 11, the RL model 1010 recalculates the reward in response to the repetitive action and calculates the predicted temperature T based on the most recent recalculated reward. p This iterative method can be repeated as many times as necessary until the most recent SIR 1020 is determined to be satisfactory to the user. At that point, the temperature state T corresponding to the most recent subsequent SIR 1020 is calculated. s can define optimized temperature parameters for the MLM.

[0100] The description of various embodiments of the present teachings is presented for illustrative purposes and is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the spirit and scope of the described embodiments. The terminology used herein has been selected to best explain the principles, practical applications, or technical improvements of the embodiments over techniques found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0101] While the foregoing describes what is considered to be the best mode and / or other examples, it is understood that various modifications may be made therein and that the subject matter disclosed herein may be implemented in various forms and examples, and that the teachings may be applied in numerous applications, only some of which are described herein. It is intended by the following claims to claim any and all applications, modifications, and variations that fall within the true scope of the present teachings. The components, steps, features, objects, benefits, and advantages discussed herein are merely exemplary. Neither they nor any discussion thereof is intended to limit the scope of protection. While various advantages have been discussed herein, it will be understood that not all embodiments necessarily include all advantages. Unless otherwise stated, all measurements, values, ratings, positions, dimensions, sizes, and other specifications set forth in this specification, including the following claims, are approximate and not exact. They are not intended to have a reasonable range consistent with the functions to which they relate and with those customary in the art to which they pertain. Numerous other embodiments are also contemplated. These include embodiments having fewer, additional, and / or different components, steps, features, objects, benefits, and advantages. These also include embodiments in which the components and / or steps are configured and / or ordered differently.

[0102] Aspects of the present disclosure are described herein with reference to call flow diagrams and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each step in the flowchart diagrams and / or block diagrams, and combinations of blocks in the call flow diagrams and / or block diagrams, can be implemented by computer-readable program instructions.

[0103] These computer-readable program instructions may be provided to a processor of a computer, special purpose computer, or other programmable data processing apparatus to produce a machine, whereby the instructions, executed via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more blocks of the call flow processes and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium, whereby the instructions can instruct a computer, programmable data processing apparatus, and / or other device to function in a particular manner, whereby a computer-readable storage medium having instructions stored thereon comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the call flows and / or block diagrams.

[0104] Furthermore, the computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device, and a series of operational steps executed on the computer, other programmable apparatus, or other device to generate a computer-implemented process, whereby the instructions executing on the computer, other programmable apparatus, or other device implement the functions / acts specified in one or more blocks of the call flow process and / or block diagram.

[0105] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a call flow process or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or call flow diagrams, and combinations of blocks in the block diagrams and / or call flow diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or acts or executes a combination of dedicated hardware and computer instructions.

[0106] It should be recognized that a computer system (e.g., dedicated computer 101, PSO engine 180, and / or processing resources) performs actions involving quantile regression of heteroscedastic error distributions that cannot be performed by a human (e.g., beyond the capabilities of a single human mind). For example, the amount of data processed, the rate at which the data is processed, and / or the data types of data processed over a particular period of time may be greater, faster, and different than the amount, rate, and data types that can be processed by a single human mind over the same period of time. A computer system may also perform the above-referenced signal processing functions while being fully operable to perform one or more other functions. Moreover, the signal processing output generated by a computer system may include information that is impossible to manually obtain by a user. For example, the amount of information contained in the signal processing output and / or the variety of information contained in the signal processing output may be more complex than information manually obtained by a user.

[0107] Moreover, because at least the conditional conformal prediction intervals of the present disclosure are established from a combination of electrical and mechanical components and circuits, a human being cannot replicate or perform the processing performed by the computer system (e.g., special-purpose computer 101, PSO engine 180, resources) disclosed herein. For example, a human being cannot communicate and / or process data associated with PSO engine 180 for a given downstream task. Additionally, special-purpose computer 101 significantly improves the operating efficiency of the computer system by accurately and reliably eliminating deleterious signal amplitude instability and noise.

[0108] While the foregoing has been described in conjunction with exemplary embodiments, it is understood that the term "exemplary" is intended as an example only, and not as best or optimal. Except as immediately noted, nothing mentioned or shown is intended to, or should be construed to, convey to the public any component, step, feature, object, benefit, advantage, or equivalent, whether or not it is recited in the claims.

[0109] It will be understood that the terms and phrases used herein have the ordinary meanings ascribed to such terms and phrases in relation to the corresponding respective fields of investigation and study, unless a specific meaning is otherwise stated herein. Relative terms such as first and second and the like may be used solely to distinguish one entity or action from another, without necessarily requiring or implying any actual relationship or order between such entities or actions. The terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements may include not only those elements, but also other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by "a" or "an" does not, in the absence of further constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that includes that element.

[0110] This Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Additionally, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments have more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as separately claimed subject matter. [Other possible items] [Item 1] 1. A computer-implemented method for optimizing a prompt session with a machine learning model (“MLM”) trained on a training dataset, comprising: setting the MLM to a first temperature state; issuing a baseline prompt to the MLM at the first temperature condition; receiving a first response to the baseline prompt from the MLM at the first temperature condition; setting the MLM to a second temperature state; issuing the baseline prompt to the MLM in the second temperature state; receiving a second response to the baseline prompt from the MLM at the second temperature condition; selecting a selected baseline response ("SBR") from the first and second responses to the baseline prompt; and feeding the SBR as a baseline behavior to a reinforcement learning model (“RLM”), wherein the RLM: calculating a reward in response to said baseline behavior; and calculating a predicted temperature state based on said reward; configured to: A computer-implemented method comprising: [Item 2] setting the MLM to a cycling temperature condition corresponding to the temperature condition of the SBR; issuing repeated prompts to the MLM in the repeated temperature conditions; receiving a first response to the repetitive prompt from the MLM in the repetitive temperature state; setting the MLM to the predicted temperature state; issuing the repetitive prompt to the MLM at the predicted temperature condition; receiving a second response to the repetitive prompt from the MLM at the predicted temperature state; selecting a selected repetitive response ("SIR") from the first and second responses to the repetitive prompt; and determining whether the SIR is satisfactory Item 1. The computer-implemented method of item 1, further comprising: [Item 3] upon determining that the SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the SIR; and determining that the SIR is not satisfactory, providing the SIR to the RLM as a recursive action; recalculating the reward in response to the repetitive behavior; and recalculating the predicted temperature state based on the recalculated reward. configured to: Item 3. The computer-implemented method of item 2, further comprising: [Item 4] setting the MLM to a subsequent iterative temperature state corresponding to the temperature state of the SIR; issuing subsequent repeat prompts to the MLM in the subsequent repeat temperature state; receiving a first response to the subsequent repetitive prompt from the MLM at the subsequent repetitive temperature state; setting the MLM to the recalculated predicted temperature state; issuing the subsequent recursive prompts to the MLM at the recalculated predicted temperature state; receiving a second response to the subsequent repetitive prompts from the MLM at the recalculated predicted temperature state; selecting a subsequent SIR from the first and second responses to the subsequent repetitive prompts; determining whether the subsequent SIR is satisfactory; Item 4. The computer-implemented method of item 3, further comprising: [Item 5] upon determining that the subsequent SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the subsequent SIR; and if it determines that the subsequent SIR is not satisfactory, providing the subsequent SIR to the RLM as a subsequent iterative action, the RLM recalculating the reward in response to the subsequent repetitive behavior; and recalculating the predicted temperature state based on the most recently recalculated reward; configured to: Item 5. The computer-implemented method of item 4, further comprising: [Item 6] Item 10. The computer-implemented method of item 1, wherein the training dataset comprises a plurality of data vectors in the embedding space of the MLM. [Item 7] 5. The computer-implemented method of claim 4, wherein the baseline prompt, the iterative prompt, and the subsequent iterative prompt each comprise a search vector in an embedding space of the MLM. [Item 8] 8. The computer-implemented method of claim 7, wherein the iterative temperature state and the subsequent iterative temperature state each include a probability distribution around one of the search vectors in the embedding space of the MLM. [Item 9] The repeated temperature conditions include: comprising a probability distribution around the SBR; Item 5. The computer-implemented method of item 4, wherein the subsequent iterative temperature states include a probability distribution around the SIR. [Item 10] 1. A computer program product for optimizing a prompt session with a machine learning model (“MLM”) trained on a set of training data, the computer program product comprising: a computer-readable storage medium having program instructions embodied thereon, the program instructions configuring a computing device to: setting the MLM to a first temperature state; issuing a baseline prompt to the MLM in the first temperature state; receiving a first response to the baseline prompt from the MLM at the first temperature condition; setting the MLM to a second temperature state; issuing the baseline prompt to the MLM in the second temperature state; receiving a second response to the baseline prompt from the MLM at the second temperature condition; selecting a selected baseline response ("SBR") from the first and second responses to the baseline prompt; and feeding the SBR as a baseline behavior to a reinforcement learning model (“RLM”), wherein the RLM: calculating a reward in response to said baseline behavior; and calculating a predicted temperature state based on said reward; configured to: a computer program product executable by a processor to cause the [Item 11] The program instructions executable by the processor cause the computing device to: setting the MLM to a repeating temperature state corresponding to the temperature state of the SBR; issuing a repetitive prompt to the MLM in the repetitive temperature state; receiving a first response to the repetitive prompt from the MLM in the repetitive temperature state; setting the MLM to the predicted temperature state; issuing the repetitive prompt to the MLM at the predicted temperature state; receiving a second response to the repetitive prompt from the MLM at the predicted temperature state; selecting a Selected Iterative Response ("SIR") from the first and second responses to the iterative prompt; and Procedures for determining whether the SIR is satisfactory Item 11. The computer program product according to item 10, further comprising: [Item 12] The program instructions executable by the processor cause the computing device to: upon determining that the SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the SIR; and if it is determined that the SIR is not satisfactory, providing the SIR to the RLM as a recursive action, the RLM comprising: recalculating the reward in response to the repetitive behavior; and recalculating the predicted temperature state based on the recalculated reward. configured to: Item 12. The computer program product according to item 11, further comprising: [Item 13] The program instructions executable by the processor cause the computing device to: setting the MLM to a subsequent iterative temperature state corresponding to the temperature state of the SIR; issuing subsequent iterative prompts to the MLM in the subsequent iterative temperature states; receiving a first response to the subsequent repetitive prompt from the MLM in the subsequent repetitive temperature state; setting the MLM to the recalculated predicted temperature state; issuing the subsequent iterative prompts to the MLM at the recalculated predicted temperature state; receiving a second response to the subsequent repetitive prompts from the MLM at the recalculated predicted temperature state; selecting a subsequent SIR from the first and second responses to the subsequent repetitive prompts; and and determining whether the subsequent SIR is satisfactory. Item 13. The computer program product according to item 12, further comprising: [Item 14] The program instructions executable by the processor cause the computing device to: upon determining that the subsequent SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the subsequent SIR; and if it is determined that the subsequent SIR is not satisfactory, providing the subsequent SIR to the RLM as a subsequent iterative action, the RLM comprising: recalculating the reward in response to the subsequent repetitive behavior; and recalculating the predicted temperature state based on the most recently recalculated reward; configured to: Item 14. The computer program product according to item 13, further comprising: [Item 15] Item 11. The computer program product of item 10, wherein the training data comprises a plurality of data vectors in the embedding space of the MLM. [Item 16] Item 14. The computer program product of item 13, wherein the baseline prompt, the repetitive prompt, and the subsequent repetitive prompt each comprise a search vector in an embedding space of the MLM. [Item 17] Item 17. The computer program product of item 16, wherein the iterative temperature state and the subsequent iterative temperature state each include a probability distribution around one of the search vectors in the embedding space of the MLM. [Item 18] 1. A computer system for optimizing a prompt session with a machine learning model (“MLM”) trained on a set of training data, the computer system comprising: a processor, a computer-readable memory, a computer-readable tangible storage device, and program instructions stored on the storage device for execution by the processor via the computer-readable memory, wherein the execution of the program instructions provides the computer system with: setting the MLM to a first temperature state; issuing a baseline prompt to the MLM at the first temperature condition; receiving a first response to the baseline prompt from the MLM at the first temperature condition; setting the MLM to a second temperature state; issuing the baseline prompt to the MLM in the second temperature state; receiving a second response to the baseline prompt from the MLM at the second temperature condition; selecting a selected baseline response ("SBR") from the first and second responses to the baseline prompt; and feeding the SBR as a baseline behavior to a reinforcement learning model (“RLM”), wherein the RLM: calculating a reward in response to said baseline behavior; and calculating a predicted temperature state based on said reward; configured to: A computer system configured to perform the method comprising: [Item 19] The method comprises: setting the MLM to a cycling temperature condition corresponding to the temperature condition of the SBR; issuing repeated prompts to the MLM in the repeated temperature conditions; receiving a first response to the repetitive prompt from the MLM in the repetitive temperature state; setting the MLM to the predicted temperature state; issuing the repetitive prompt to the MLM at the predicted temperature condition; receiving a second response to the repetitive prompt from the MLM at the predicted temperature state; selecting a selected repetitive response ("SIR") from the first and second responses to the repetitive prompt; and determining whether the SIR is satisfactory Item 19. The computer system of item 18, further comprising: [Item 20] The method comprises: upon determining that the SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the SIR; and determining that the SIR is not satisfactory, providing the SIR to the RLM as a recursive action; recalculating the reward in response to the repetitive behavior; and recalculating the predicted temperature state based on the recalculated reward. configured to: setting the MLM to a subsequent iterative temperature state corresponding to the temperature state of the SIR; issuing subsequent repeat prompts to the MLM in the subsequent repeat temperature state; receiving a first response to the subsequent repetitive prompt from the MLM at the subsequent repetitive temperature state; setting the MLM to the recalculated predicted temperature state; issuing the subsequent recursive prompts to the MLM at the recalculated predicted temperature state; receiving a second response to the subsequent repetitive prompts from the MLM at the recalculated predicted temperature state; selecting a subsequent SIR from the first and second responses to the subsequent repetitive prompts; determining whether the subsequent SIR is satisfactory; upon determining that the subsequent SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the subsequent SIR; and if it determines that the subsequent SIR is not satisfactory, providing the subsequent SIR to the RLM as a subsequent iterative action, the RLM recalculating the reward in response to the subsequent repetitive behavior; and recalculating the predicted temperature state based on the most recently recalculated reward; configured to: 20. The computer system of claim 19, further comprising:

Claims

1. 1. A computer-implemented method for optimizing a prompt session with a machine learning model (“MLM”) trained on a training dataset, comprising: setting the MLM to a first temperature state; issuing a baseline prompt to the MLM in the first temperature state; receiving a first response to the baseline prompt from the MLM at the first temperature condition; setting the MLM to a second temperature state; issuing the baseline prompt to the MLM in the second temperature state; receiving a second response to the baseline prompt from the MLM at the second temperature condition; selecting a selected baseline response ("SBR") from the first response and the second response to the baseline prompt; and feeding the SBR as a baseline behavior to a reinforcement learning model (“RLM”), the RLM calculating a reward in response to said baseline behavior; and calculating a predicted temperature state based on said reward; configured to: A computer-implemented method comprising:

2. setting the MLM to a repeating temperature state corresponding to the temperature state of the SBR; issuing repeated prompts to the MLM in the repeated temperature conditions; receiving a first response to the repetitive prompt from the MLM in the repetitive temperature state; setting the MLM to the predicted temperature state; issuing said repetitive prompt to said MLM at said predicted temperature state; receiving a second response to the repetitive prompt from the MLM at the predicted temperature state; selecting a selected repetitive response ("SIR") from the first response and the second response to the repetitive prompt; and determining whether the SIR is satisfactory; The computer-implemented method of claim 1 further comprising:

3. upon determining that the SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the SIR; and determining that the SIR is not satisfactory, providing the SIR to the RLM as a recursive action; recalculating the reward in response to the repetitive behavior; and recalculating the predicted temperature state based on the recalculated reward. configured to: The computer-implemented method of claim 2 further comprising:

4. setting the MLM to a subsequent iterative temperature state corresponding to the temperature state of the SIR; issuing subsequent iterative prompts to the MLM at the subsequent iterative temperature states; receiving a first response to the subsequent iterative prompt from the MLM at the subsequent iterative temperature state; setting the MLM to the recalculated predicted temperature state; issuing the subsequent recursive prompts to the MLM at the recalculated predicted temperature state; receiving a second response to the subsequent repetitive prompts from the MLM at the recalculated predicted temperature state; selecting a subsequent SIR from the first response and the second response to the subsequent repetitive prompt; determining whether the subsequent SIR is satisfactory; The computer-implemented method of claim 3 further comprising:

5. determining that the subsequent SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the subsequent SIR; and determining that the subsequent SIR is not satisfactory, providing the subsequent SIR to the RLM as a subsequent iterative action; recalculating the reward in response to the subsequent repetitive behavior; and recalculating the predicted temperature state based on the most recently recalculated reward; configured to: The computer-implemented method of claim 4 further comprising:

6. The computer-implemented method of claim 1 , wherein the training data set comprises a plurality of data vectors in an embedding space of the MLM.

7. The computer-implemented method of claim 4 , wherein the baseline prompt, the iterative prompt, and the subsequent iterative prompt each comprise a search vector in an embedding space of the MLM.

8. The computer-implemented method of claim 7 , wherein the iterative temperature state and the subsequent iterative temperature state each comprise a probability distribution around one of the search vectors in the embedding space of the MLM.

9. The repeated temperature conditions include: a probability distribution around said SBR; The computer-implemented method of claim 4 , wherein the subsequent iterative temperature states include a probability distribution around the SIR.

10. 1. A computer program for optimizing a prompt session with a machine learning model (“MLM”) trained on a set of training data, the computer program comprising: setting the MLM to a first temperature state; issuing a baseline prompt to the MLM in the first temperature state; receiving a first response to the baseline prompt from the MLM at the first temperature condition; setting the MLM to a second temperature state; issuing the baseline prompt to the MLM in the second temperature state; receiving a second response to the baseline prompt from the MLM at the second temperature condition; selecting a selected baseline response ("SBR") from the first response and the second response to the baseline prompt; and feeding the SBR as a baseline behavior to a reinforcement learning model (“RLM”), wherein the RLM: calculating a reward in response to said baseline behavior; and calculating a predicted temperature state based on said reward; configured to: A computer program for executing

11. the computing device; setting the MLM to a repeating temperature state corresponding to the temperature state of the SBR; issuing a repeating prompt to the MLM in the repeating temperature state; receiving a first response to the repetitive prompt from the MLM in the repetitive temperature state; setting the MLM to the predicted temperature state; issuing said repeating prompt to said MLM at said predicted temperature state; receiving a second response to the repetitive prompt from the MLM at the predicted temperature state; selecting a selected repetitive response ("SIR") from the first response and the second response to the repetitive prompt; and a procedure for determining whether the SIR is satisfactory; The computer program of claim 10, further comprising:

12. the computing device; upon determining that the SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the SIR; and If it is determined that the SIR is not satisfactory, the RLM is provided with the SIR as an iterative action, the RLM comprising: recalculating the reward in response to the repetitive behavior; and recalculating the predicted temperature state based on the recalculated reward. configured to: The computer program of claim 11 further comprising:

13. the computing device; setting the MLM to a subsequent iterative temperature state corresponding to the temperature state of the SIR; issuing a subsequent iterative prompt to the MLM in the subsequent iterative temperature state; receiving a first response to the subsequent iterative prompt from the MLM at the subsequent iterative temperature state; setting the MLM to the recalculated predicted temperature state; issuing the subsequent iterative prompts to the MLM at the recalculated predicted temperature state; receiving a second response to the subsequent repetitive prompts from the MLM at the recalculated predicted temperature state; selecting a subsequent SIR from the first response and the second response to the subsequent repetitive prompts; and determining whether the subsequent SIR is satisfactory; The computer program of claim 12 , further comprising:

14. the computing device; upon determining that the subsequent SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the subsequent SIR; and if it is determined that the subsequent SIR is not satisfactory, providing the subsequent SIR to the RLM as a subsequent iterative action, the RLM comprising: recalculating the reward in response to the subsequent repetitive behavior; and recalculating the predicted temperature state based on the most recently recalculated reward; configured to: The computer program of claim 13 , further comprising:

15. The computer program of claim 10 , wherein the training data comprises a plurality of data vectors in an embedding space of the MLM.

16. 14. The computer program of claim 13, wherein the baseline prompt, the repetitive prompt, and the subsequent repetitive prompt each comprise a search vector in an embedding space of the MLM.

17. 17. The computer program of claim 16, wherein the iterative temperature state and the subsequent iterative temperature state each comprise a probability distribution around one of the search vectors in the embedding space of the MLM.

18. 1. A computer system for optimizing a prompt session with a machine learning model (“MLM”) trained on a set of training data, the computer system comprising: a processor, a computer-readable memory, a computer-readable tangible storage device, and program instructions stored on the computer-readable tangible storage device for execution by the processor via the computer-readable memory, wherein the execution of the program instructions provides the computer system with: setting the MLM to a first temperature state; issuing a baseline prompt to the MLM in the first temperature state; receiving a first response to the baseline prompt from the MLM at the first temperature condition; setting the MLM to a second temperature state; issuing the baseline prompt to the MLM in the second temperature state; receiving a second response to the baseline prompt from the MLM at the second temperature condition; selecting a selected baseline response ("SBR") from the first response and the second response to the baseline prompt; and feeding the SBR as a baseline behavior to a reinforcement learning model (“RLM”), the RLM calculating a reward in response to said baseline behavior; and calculating a predicted temperature state based on said reward; configured to: A computer system configured to perform the method comprising:

19. The method comprises: setting the MLM to a repeating temperature state corresponding to the temperature state of the SBR; issuing repeated prompts to the MLM in the repeated temperature conditions; receiving a first response to the repetitive prompt from the MLM in the repetitive temperature state; setting the MLM to the predicted temperature state; issuing said repetitive prompt to said MLM at said predicted temperature state; receiving a second response to the repetitive prompt from the MLM at the predicted temperature state; selecting a selected repetitive response ("SIR") from the first response and the second response to the repetitive prompt; and determining whether the SIR is satisfactory; 20. The computer system of claim 18, further comprising:

20. The method comprises: upon determining that the SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the SIR; and determining that the SIR is not satisfactory, providing the SIR to the RLM as a recursive action; recalculating the reward in response to the repetitive behavior; and recalculating the predicted temperature state based on the recalculated reward. configured to: setting the MLM to a subsequent iterative temperature state corresponding to the temperature state of the SIR; issuing subsequent iterative prompts to the MLM at the subsequent iterative temperature states; receiving a first response to the subsequent iterative prompt from the MLM at the subsequent iterative temperature state; setting the MLM to the recalculated predicted temperature state; issuing the subsequent recursive prompts to the MLM at the recalculated predicted temperature state; receiving a second response to the subsequent repetitive prompts from the MLM at the recalculated predicted temperature state; selecting a subsequent SIR from the first response and the second response to the subsequent repetitive prompt; determining whether the subsequent SIR is satisfactory; determining that the subsequent SIR is satisfactory, setting the MLM to a temperature state corresponding to the temperature state of the subsequent SIR; and determining that the subsequent SIR is not satisfactory, providing the subsequent SIR to the RLM as a subsequent iterative action; recalculating the reward in response to the subsequent repetitive behavior; and recalculating the predicted temperature state based on the most recently recalculated reward; configured to:

20. The computer system of claim 19, further comprising:

Images