Computer-based method, computer system and computer program (combinatorial prompting for large language models)

Abstract

To provide methods, systems and programs for identifying hard prompts for increasing capabilities of LLMs while keeping explainability at the token level.SOLUTION: A method comprises: selecting a predetermined number of examples from a training dataset; concatenating each of the selected examples with a current prompt of a target model to obtain a set of candidate prompts; for each individual candidate prompt in the set of candidate prompts, calculating a loss value over a validation dataset; and replacing the current prompt with the individual candidate prompt having a lowest calculated loss value that is less than or equal to an original loss value over the validation set for the current prompt to obtain an updated prompt.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE This application relates generally to computer processing, and more specifically to generating and employing progressively optimized combinatorial prompts for tuning a subject model. [Background technology]

[0002] Large language models (LLMs) are deep neural networks that are pre-trained on a diverse and large text corpus. LLMs are currently commonly deployed to perform multiple natural language processing tasks. Successfully redeploying pre-trained LLMs for specialized downstream tasks typically required fine-tuning to train the model on the specific dataset for the specialized downstream task. Recently, as LLMs have grown significantly in size (making fine-tuning less attractive due to costs), manual prompt design for tuning the model has become increasingly popular, where instead of fine-tuning weights for each task, only the LLM prompts are changed to fit each task. Thus, many companies employing LLMs are constantly striving to adopt the most effective prompt-based tuning techniques to increase model accuracy while maintaining overall model performance. Summary of the Invention [Problem to be solved by the invention]

[0003] For example, when designing hard prompts, in contrast to soft prompts, considering their future potential for enhancing the power of LLMs while maintaining token-level explainability, it is difficult to identify the best possible hard prompts when the only constraint is the vocabulary (tokens) of the LLM. [Means for solving the problem]

[0004] According to one embodiment, a method, computer system, and computer program product are provided for generating and employing progressively optimized combination prompts for tuning a target model. The embodiment may comprise selecting a predetermined number of examples from a training dataset. The embodiment may also comprise concatenating each of the selected examples with a current prompt of the target model to obtain a set of candidate prompts. The embodiment may further comprise calculating a loss value for each individual candidate prompt in the set of candidate prompts against a validation dataset. The embodiment may also comprise replacing the current prompt with the individual candidate prompt having the lowest calculated loss value that is lower than or equal to the original loss value against the validation set for the current prompt to obtain an updated prompt. [Brief description of the drawings]

[0005] These and other objects, features, and advantages of the present disclosure will become apparent from the following detailed description of illustrative embodiments, which is to be read in connection with the accompanying drawings, in which various features of the drawings are not drawn to scale, as the illustrations are for clarity in facilitating those skilled in the art to understand the invention together with the detailed description.

[0006] [Figure 1] 1 illustrates an exemplary networked computing environment according to at least one embodiment.

[0007] [Diagram 2] 1 illustrates an operational flowchart for an exemplary process for generating and employing progressively optimized combination prompts for tuning a target model, according to at least one embodiment.

[0008] [Diagram 3] 3 shows a diagram of the example process of FIG. 2.

[0009] [Figure 4] 1 illustrates an illustrative algorithm that may be employed to perform an exemplary process for generating and employing progressively optimized combination prompts for tuning a target model, according to at least one embodiment.

[0010] [Diagram 5] A series of tables illustrating comparative results (F-1 scores) for entity matching and lexicon matching tasks for an exemplary question answering model tuned according to the described embodiment and results from a comparison model for the same task tuned using either random sampling or manual selection are shown. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Detailed embodiments of the claimed structures and methods are disclosed herein; however, it can be understood that the disclosed embodiments are merely exemplary of the claimed structures and methods, which may be embodied in various forms. The present disclosure, however, may be embodied in many different forms and should not be construed as being limited to the exemplary embodiments described herein. In the description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0012] The singular forms "a," "an," and "the" should be understood to include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to a "component surface" includes a reference to one or more of such surfaces unless the context clearly dictates otherwise.

[0013] According to one aspect, the disclosure relates to a computer-based method of generating and employing progressively optimized combination prompts for tuning a target model, the method comprising: selecting a predetermined number of examples from a training dataset, concatenating each of the selected examples with a current prompt of the target model to obtain a set of candidate prompts, calculating a loss value against a validation dataset for each individual candidate prompt in the set of candidate prompts, and replacing the current prompt with the individual candidate prompt having the lowest calculated loss value that is lower than or equal to the original loss value against the validation set for the current prompt to obtain an updated prompt. Thus, the described method iteratively adds prompt components by leveraging examples from the training dataset that result in the minimum loss, progressively increasing the specificity and relevance of the prompts to elicit more accurate responses from the target model.

[0014] In embodiments, iteratively generating and employing progressively optimized combination prompts to tune the subject model may further comprise: processing subsequent batches of examples from the training dataset in an amount corresponding to a predetermined number of examples until employing the progressively optimized combination prompts produces a minimum loss value that is higher than or equal to the comparison loss value for the validation set associated with the most recently updated prompt. Thus, described embodiments provide improved tuning of the subject model by employing progressively optimized combination prompts by processing subsequent batches of examples from the training dataset until there is no discernible benefit from updating the most recently updated prompts for the calculated minimum loss value or given validation set.

[0015] In embodiments, the target model may be a question answering system that is capable of performing various tasks, such as entity matching and lexicon matching, with improved accuracy as a result of the question answering system being tuned using the described combination prompts.

[0016] In another embodiment, iteratively generating and employing progressively optimized combination prompts for tuning the target model may comprise processing subsequent batches of examples from the training dataset in an amount corresponding to a predetermined number of examples until a predetermined threshold number of examples have been incorporated into the prompts, allowing a user to configure when tuning of the target model is complete depending on the number of examples incorporated into the prompts, rather than continuing indefinitely until there is no discernible gain in terms of loss on the validation set.

[0017] In another embodiment, selecting the predetermined number of examples from the training dataset further comprises performing a centroid-based selection of the predetermined number of examples based on geometric techniques, which provides an improved selection of the predetermined number of examples from the training dataset when compared to conventional manual or random selection methods.

[0018] In yet another embodiment, selecting the predetermined number of examples from the training dataset further comprises performing a random search of the training dataset for hyperparameters that include the predetermined number of examples, thereby providing yet another improved method of selecting the predetermined number of examples from the training dataset when compared to traditional manual or random selection methods.

[0019] In embodiments, the described methods may further comprise continually calculating a minimum loss value for the most recent prompts to provide a baseline loss value. Thus, the described embodiments may progressively optimize the generated and employed combination prompts based on the calculated minimum loss value, which allows a system employing the described embodiments to determine when incorporating a given candidate combination prompt for tuning a given subject model would not provide a discernible benefit.

[0020] In another aspect, the disclosure relates to a computer system comprising one or more processors, one or more computer readable memories, one or more computer readable tangible storage media, and program instructions stored in at least one of the one or more computer readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer readable memories, the computer system may execute a method comprising: selecting a predetermined number of examples from a training dataset, concatenating each of the selected examples with a current prompt of a subject model to obtain a set of candidate prompts, calculating a loss value against a validation dataset for each candidate prompt in the set of candidate prompts, and replacing the current prompt with the individual candidate prompt having the lowest calculated loss value that is lower than or equal to the original loss value against the validation set for the current prompt to obtain an updated prompt. Thus, the described computer system may iteratively add prompt components by leveraging examples from the training dataset that result in the minimum loss, progressively increasing the specificity and relevance of the prompts to elicit more accurate responses from the subject model.

[0021] In yet another aspect, the disclosure relates to a computer program product comprising one or more computer-readable tangible storage media and program instructions stored on at least one of the one or more computer-readable tangible storage media, the program instructions being executable by a processor capable of performing a method comprising: selecting a predetermined number of examples from a training dataset; concatenating each of the selected examples with a current prompt of a target model to obtain a set of candidate prompts; calculating a loss value for each candidate prompt in the set of candidate prompts against a validation dataset; and replacing the current prompt with the respective candidate prompt having a lowest calculated loss value that is lower than or equal to an original loss value against the validation set for the current prompt to obtain an updated prompt. Thus, the described computer program product may iteratively add prompt components by leveraging examples from the training dataset that result in the minimum loss, progressively increasing the specificity and relevance of the prompts to elicit more accurate responses from the target model.

[0022] FIELD OF THE DISCLOSURE Embodiments of the present application relate generally to computer processing, and more particularly to generating and employing progressively optimized combination prompts for tuning a target model. Exemplary embodiments described below provide, among other things, a system, method, and program product for selecting a predetermined number of examples from a training dataset, concatenating each of the selected examples with a current prompt of a target model to obtain a set of candidate prompts, calculating a loss value against a validation dataset for each individual candidate prompt in the set of candidate prompts, and replacing the current prompt with the individual candidate prompt having the lowest calculated loss value that is lower than or equal to the original loss value against the validation set for the current prompt to obtain an updated prompt.

[0023] As explained earlier, large-scale language models (LLMs) are deep neural networks that are pre-trained on a diverse and large text corpus. LLMs are currently commonly deployed to perform multiple natural language processing tasks. To successfully redeploy pre-trained LLMs for specialized downstream tasks, fine-tuning was usually required to train the model on the specific dataset for the specialized downstream task. Recently, as the scale of LLMs has grown significantly (making fine-tuning less attractive due to costs), manual prompt design for tuning the model has become increasingly popular, where instead of fine-tuning weights for each task, only the LLM prompts are changed to fit each task. Thus, many companies employing LLMs are constantly striving to adopt the most effective prompt-based tuning techniques to increase model accuracy while maintaining overall model performance.

[0024] However, there are many challenges when designing prompts for tuning large language models. For example, when designing hard prompts, in contrast to soft prompts, it is difficult to identify the best possible hard prompt when the only constraint is the vocabulary (tokens) of the LLM, given their future potential for enhancing the power of the LLM while maintaining token-level explainability. This is because the probability space (number of possible permutations) can be impractically large. Alternative prompt design methods therefore rely on crude solutions such as random sampling procedures or leveraging human intuition to perform manual selection. These approaches can limit the performance of a given model and result in the inclusion of inefficient examples in the prompt. Thus, if the challenge of generating shorter prompts by including only the most efficient examples were overcome, it would be possible to explore the text space more efficiently and systematically, thereby improving accuracy and speed when performing various tasks.

[0025] Accordingly, a method, computer system, and computer program product for generating and employing progressively optimized combination prompts for tuning a target model would be advantageous. The method, system, and computer program product may select a predetermined number of examples from a training dataset. The method, system, and computer program product may concatenate each of the selected examples with a current prompt of the target model to obtain a set of candidate prompts. The method, system, and computer program product may then calculate a loss value for each individual candidate prompt in the set of candidate prompts against a validation dataset. The method, system, and computer program product may then replace the current prompt with the individual candidate prompt having the lowest calculated loss value that is lower than or equal to the original loss value for the validation set for the current prompt to obtain an updated prompt. As a result, the method, system, and computer program product provide improved generation and employment of progressively optimized combination prompts for tuning a target model. The described embodiments provide an incremental prompt design method that gradually improves prompts while ensuring reliable performance. The described embodiments iteratively add prompt components by leveraging examples from the training dataset that yield the minimum loss, progressively increasing the specificity and relevance of the prompts to elicit more accurate responses from the subject model. The described embodiments may iteratively generate and employ progressively optimized combination prompts for tuning the subject model by processing subsequent batches of examples from the training dataset until there is no discernible benefit from updating the most recently updated prompts for the computed minimum loss value or given validation set.

[0026] The present invention may be a system, method, and / or computer program product integrated at any possible level of technical detail. The computer program product may include a computer-readable storage medium or media having computer-readable program instructions for causing a processor to carry out aspects of the present invention.

[0027] Various aspects of the present disclosure are described by text, flow charts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of computer program products (CPPs). For any flow chart, depending on the technology involved, operations may be performed in a different order than that shown in a given flow chart. For example, again depending on the technology involved, two operations shown in successive flow chart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in an at least partially overlapping manner in time.

[0028] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any set of one or more storage media (also referred to as "media") collectively contained in a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices (such as punch cards or pits / lands formed on a major surface of a disk), or any suitable combination of the above. Computer-readable storage media, as the term is used in this disclosure, is not to be construed as storage in the form of a transitory signal per se, such as electric waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals communicated through wires, and / or other transmission media. As will be appreciated by those skilled in the art, data is typically moved at some occasional time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but this does not make the storage device temporary, as the data is not temporary while it is stored.

[0029] 1, a computing environment 100 includes an example of an environment for executing at least a portion of computer code involved in performing the methodology of the present invention, such as a prompt optimization program / code 150. The computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106, in addition to the prompt optimization code 150. In this embodiment, the computer 101 includes a processor set 110 (including processing circuitry 120 and cache 121), a communication fabric 111, a volatile memory 112, a persistent storage 113 (including an operating system 122 and the prompt optimization code 150, as identified above), a peripheral device set 114 (including a user interface (UI) device set 123, storage 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. The remote server 104 includes a remote database 130. The public cloud 105 includes a gateway 140, a cloud orchestration module 141, a set of host physical machines 142, a set of virtual machines 143, and a set of containers 144.

[0030] Computer 101 may take the form of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch or other wearable computer, a mainframe computer, a quantum computer, or any other form of computer or mobile device now known or later developed that is capable of executing programs, accessing a network, or querying a database, such as remote database 130. As is well understood in the field of computer technology, and depending on the technology, execution of a computer-implemented method may be distributed among multiple computers and / or among multiple locations. However, in this description of computing environment 100, for purposes of brevity as much as possible, the detailed discussion focuses on a single computer, specifically computer 101. Although computer 101 is not shown in the cloud in FIG. 1, it may be located in the cloud. However, computer 101 is not required to be in the cloud except to any extent that may be categorically indicated.

[0031] Processor set 110 includes one or more computer processors of any type now known or developed in the future. Processing circuitry 120 may be distributed across multiple packages, e.g., multiple tailored integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by threads or cores executing on processor set 110. Cache memory is typically organized into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all of the caches for a processor set may be located "off-chip." In some computing environments, processor set 110 may be designed to operate with qubits and perform quantum computing.

[0032] Computer readable program instructions are typically loaded onto the computer 101 to cause a sequence of operational steps to be executed by the processor set 110 of the computer 101, thereby realizing a computer-implemented method, such that the instructions so executed instantiate the method specified in the flowcharts and / or descriptions of the computer-implemented methods contained herein (collectively, the "methods of the present invention"). These computer readable program instructions are stored in various types of computer readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the methods of the present invention. In the computing environment 100, at least a portion of the instructions for executing the methods of the present invention may be stored in prompt optimization code 150 in persistent storage 113.

[0033] Communications fabric 111 is the signal-conducting pathways that allow the various components of computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive pathways, such as switches and conductive pathways that make up buses, bridges, physical input / output ports, etc. Other types of signal communication pathways may be used, such as fiber optic and / or wireless communication pathways.

[0034] Volatile memory 112 is any type of volatile memory now known or developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory is characterized by random access, although this is not required unless expressly indicated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101, although alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 101.

[0035] The persistent storage 113 is any form of non-volatile storage for a computer, now known or later developed. The non-volatility of this storage means that the stored data is maintained regardless of whether power is provided to the computer 101 and / or to the persistent storage 113 directly. The persistent storage 113 can be a read-only memory (ROM), but typically at least a portion of the persistent storage allows data to be written, data to be deleted, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. The operating system 122 can take a number of forms, such as various known proprietary operating systems or open source portable operating system interface type operating systems that employ a kernel. The code included in the prompt optimization program 150 typically includes at least a portion of the computer code involved in carrying out the method of the present invention.

[0036] The peripheral device set 114 includes a set of peripheral devices of the computer 101. Data communication connections between the peripheral devices and other components of the computer 101 may be implemented in various ways, such as Bluetooth connections, Near Field Communication (NFC) connections, connections made by cables (such as Universal Serial Bus (USB) type cables), insertion type connections (e.g., Secure Digital (SD) cards), connections made through local area communication networks, and even connections made through wide area networks such as the Internet. In various embodiments, the UI device set 123 may include components such as display screens, speakers, microphones, wearable devices (such as goggles and smart watches), keyboards, mice, printers, touch pads, game controllers, and haptic devices. The storage 124 is external storage, such as an external hard drive, or insertable storage, such as an SD card. The storage 124 may be persistent and / or volatile. In some embodiments, the storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (e.g., where computer 101 stores and manages a large database locally), this storage may be provided by a peripheral storage device designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple, geographically distributed computers. IoT sensor set 125 is made up of sensors that may be used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0037] The network module 115 is a collection of computer software, hardware, and firmware that enables the computer 101 to communicate with other computers over the WAN 102. The network module 115 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of the network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of the network module 115 are performed on physically separate devices such that the control function manages multiple different network hardware devices. Computer readable program instructions for carrying out the methods of the present invention may be downloaded to the computer 101 from an external computer or external storage device, typically through a network adapter card or network interface included in the network module 115.

[0038] WAN 102 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology now known or later developed for communicating computer data. In some embodiments, the WAN may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include computer hardware such as copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers.

[0039] End-user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise that operates computer 101) and may take any of the forms discussed above in connection with computer 101. EUD 103 typically receives useful and useful data from the operation of computer 101. For example, in the hypothetical case where computer 101 is designed to provide recommendations to the end user, the recommendations would typically be communicated from network module 115 of computer 101 over WAN 102 to EUD 103. In this manner, EUD 103 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 103 may be a client device such as a thin client, a heavy client, a mainframe computer, a desktop computer, and the like.

[0040] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 may be controlled and used by the same entity that operates computer 101. Remote server 104 represents a machine that collects and stores useful and useful data for use by other computers, such as computer 101. For example, in the hypothetical case where computer 101 is designed and programmed to provide recommendations based on past data, this past data may be provided to computer 101 from remote database 130 of remote server 104.

[0041] A public cloud 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer functionality, particularly data storage (cloud storage) and computing power, without direct active management by users. Cloud computing typically leverages sharing of resources to achieve coherence and economies of scale. Direct active management of the computing resources of the public cloud 105 is performed by computer hardware and / or software of a cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments that run on various computers that make up the computers of the host physical machine set 142, which is the universe of physical computers available in and / or at the public cloud 105. The virtual computing environments (VCEs) typically take the form of virtual machines from the virtual machine set 143 and / or containers from the container set 144. It is understood that these VCEs can be stored as images and can be transferred among and between various hosts of physical machines, either as images or after instantiation of the VCEs. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCE, and manages active instantiations of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that enables public cloud 105 to communicate over WAN 102.

[0042] Some further description of Virtualized Computing Environments (VCEs) is now provided. A VCE may be stored as an "image." A new active instance of a VCE may be instantiated from an image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system level virtualization. This refers to a feature of an operating system where the kernel allows the existence of multiple isolated user space instances called containers. These isolated user space instances typically behave as real computers from the perspective of the programs running within them. A computer program running on a normal operating system may utilize all of the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container may only use the contents of the container and the devices assigned to the container, a feature known as containerization.

[0043] A private cloud 106 is similar to a public cloud 105, except that the computing resources are available only for use by a single enterprise. Although the private cloud 106 is shown as being in communication with the WAN 102, in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., types of private, community, or public clouds), often each implemented by a different vendor. While each of the multiple clouds remains a separate, discrete entity, the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the multiple constituent clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0044] According to the present embodiment, the prompt optimization program 150 may be a program that can select a predetermined number of examples from the training data set. The prompt optimization program 150 may then identify defects in the printing operation based on the tracked print data. The prompt optimization program 150 may then concatenate each of the selected examples with the current prompt of the target model to obtain a set of candidate prompts. The prompt optimization program 150 may then calculate a loss value for the validation data set for each individual candidate prompt in the set of candidate prompts. The prompt optimization program 150 may then replace the current prompt with the individual candidate prompt having the lowest calculated loss value that is lower than or equal to the original loss value for the validation set for the current prompt to obtain an updated prompt. As a result, the prompt optimization program 150 provides improved generation and employment of progressively optimized combination prompts for tuning the target model. The described embodiment provides an incremental prompt design method that gradually improves prompts while ensuring reliable performance. The described embodiments iteratively add prompt components by leveraging examples from the training dataset that yield the minimum loss, progressively increasing the specificity and relevance of the prompts to elicit more accurate responses from the subject model. The described embodiments may iteratively generate and employ progressively optimized combination prompts for tuning the subject model by processing subsequent batches of examples from the training dataset until there is no discernible benefit from updating the most recently updated prompts for the computed minimum loss value or given validation set.

[0045] Referring now to FIG. 2, an operational flowchart is provided for an illustrative process 200 for generating and employing progressively optimized combination prompts for tuning a subject model, according to at least one embodiment.

[0046] At 202, the prompt optimizer 150 may select a predetermined number of examples from the training dataset. For example, at this stage, the prompt optimizer 150 may be configured to select a predetermined batch of 10 examples from an exemplary training dataset "D". In some embodiments where the dataset is large, the prompt optimizer 150 may be configured to search for the k best prompt examples within a corresponding high search space. For example, in some embodiments with a large training dataset, the prompt optimizer 150 may be configured to perform a centroid-based selection that leverages geometric techniques to find the k best prompt examples within an exponentially high search space by finding c centroids, then finding the n nearest neighbors, and sampling only among them to find the predetermined number of examples from the training dataset. These methods for selecting a predetermined number of examples from the training dataset may be significantly more efficient when compared to methods such as grid search in training datasets that are large and have high-dimensional spaces. In other embodiments, the prompt optimizer 150 may be configured to perform a random search, so that different examples from the training dataset that can be combined with a given prompt may be considered “hyperparameters” (random search for hyperparameters).

[0047] At 204, the prompt optimization program 150 may then concatenate each of the selected examples with the current prompt of the target model to obtain a set of candidate prompts. For example, for a current example prompt "P" of an example target model "T," the prompt optimization program 150 may concatenate each of the ten selected example examples "E1"-"E10" with the current prompt "P" to obtain a set of candidate prompts that includes ten distinct combination prompts. The obtained candidate prompts may then be compared to the original current prompt to evaluate whether the candidate prompts would be suitable replacements for the original current prompt.

[0048] Next, at 206, the prompt optimization program 150 may calculate a loss value for the validation data set for each individual candidate prompt in the set of candidate prompts. Returning to the example above, at this stage, the prompt optimization program 150 may calculate a loss value for any suitable validation set for each of the candidate prompts obtained in stage 204. For example, the prompt optimization program 150 may calculate a loss value for an example validation data set "V" for a first candidate prompt "CP1" obtained by concatenating an illustrative example "E1" with the current prompt "P". The calculated loss value for each of the candidate prompts may then be compared to the loss value associated with the original prompt "P" to identify which candidate prompt would be the most efficient and suitable replacement for the original current prompt "P".

[0049] At 208, the prompt optimization program 150 may replace the current prompt with the respective candidate prompt having the lowest calculated loss value that is lower than or equal to the original loss value for the validation set for the current prompt to obtain an updated prompt. Returning to the above example, if a set of candidate prompts "CP1"-"CP10" is obtained, the prompt optimization program 150 would calculate 206 a loss value for an example validation data set "V" for each of the set of candidate prompts. At step 208, the prompt optimization program 150 replaces the original current prompt "P" with the candidate prompt having the lowest calculated loss value, e.g., "CP1", that is lower than or equal to the original loss value for the validation set for the current prompt "P" to obtain an updated prompt "UP1". Note that the prompt optimization program 150 replaces the original current prompt with the candidate prompt having the lowest calculated loss value for the validation set only if the loss value for the candidate prompt is lower than or equal to the original loss value for the validation set for the original current prompt "P". In other words, a candidate prompt is used as a replacement only if that replacement brings benefits in terms of the loss value calculated on the validation set, thereby ensuring an improvement of the prompts employed for the target model.

[0050] In an embodiment, as shown at 210, the prompt optimization program 150 may be configured to iteratively generate and employ progressively optimized combination prompts for tuning the subject model by processing subsequent batches of examples from the training dataset in an amount corresponding to a predetermined number of examples until the minimum loss value calculated for the latest candidate prompt is higher than or equal to the most recently updated prompt. In other words, the prompt optimization program 150 continuously performs steps 202-208 described above to progressively optimize the combination prompts employed by the subject model. In an embodiment, the prompt optimization program 150 may be configured to repeat steps 202-208 until the minimum loss value calculated for the latest candidate prompt is higher than or equal to the loss value calculated for the most recently updated prompt. In other words, the prompt optimization program 150 may continue until further updates to the most recently updated version of the prompt (the most recently updated prompt) do not produce any discernible benefit with respect to the minimum loss value calculated for the validation set for the most recently updated prompt (the comparison loss value). Accordingly, prompt optimizer 150 continuously and iteratively calculates a loss value for the validation set for the most recently updated prompt each time a given prompt is updated and replaced by a given candidate prompt. The calculated loss value corresponding to the most recently updated prompt then serves as a baseline for determining whether a future update or replacement will be beneficial in terms of the calculated loss value.

[0051] In other embodiments, prompt optimization program 150 may be configured to repeat steps 202-208 until a predetermined number of examples, in other words a threshold number of examples, have been incorporated into the current prompt "P." For example, prompt optimization program 150 may be configured to repeat steps 202-208 until a threshold number of three examples have been incorporated into the original current prompt "P."

[0052] FIG. 3 shows a diagram of an example process 300 for generating and employing progressively optimized combination prompts for tuning a target model in accordance with the described embodiment and the above-described stages of process 200 in FIG.

[0053] 3, a current prompt "P" at 310 is optimized for tuning a target model (not shown). A training dataset at 320 provides a source for prompt optimizer 150 to select a batch or sample containing "k" examples to be concatenated with current prompt "P" at 330 to obtain "k" candidate prompts. Prompt optimizer 150 may then utilize a validation set at 340 to calculate a loss value for each candidate prompt. At 350, "l" is selected from the training dataset and a training set is selected from the training dataset and a training set is selected from the training dataset. new The candidate with the lowest calculated loss value on the validation set in 330, denoted by "l pre ” to the loss value associated with the original current prompt at 360. new The calculated loss value corresponding to l pre If the loss value is higher than the loss value corresponding to l, the original prompt "P" is returned at 370 without being replaced or altered. new The calculated loss value corresponding to l pre If the loss value corresponding to the candidate prompt "P" is lower than the loss value corresponding to the candidate prompt "P" then the original prompt "P" is replaced at 380 by the candidate prompt having the lowest calculated loss value.

[0054] 4 shows an illustrative algorithm that may be employed to perform an exemplary process for generating and employing progressively optimized combination prompts for tuning a subject model, according to at least one embodiment. The algorithm 400 shown in FIG. 4 may be used to perform the process shown in FIG. 3 described above for generating progressively optimized combination prompts.

[0055] FIG. 5 shows a series of tables illustrating comparative results (F-1 scores) for entity matching and lexicon matching tasks for an exemplary question answering model tuned according to the described embodiments and a comparison model tuned using either random sampling or manual selection. The model performed these tasks on nine diverse and publicly available datasets, each of which is represented by the column titles "DBLP.ACM, DBLP.GoogleScholar," etc., shown in table 510. For example, the table shown in 510 shows entity matching task results for two separate language models, "Flan-T5" and "UL2," respectively, tuned using the described embodiments for generating progressively optimized combinatorial prompts, represented by the row labeled "CP." Table 510 also includes comparative entity matching task results for the same two language models tuned using random sampling and manual selection, represented by the rows labeled "random" and "manual," respectively. The results in table 510 include averages over five seeds with standard deviations. It may be noted that the presently described embodiment provides the highest average F-1 score for both models with the more favorable standard deviation.

[0056] FIG. 5 further shows a second table 520 including lexicon matching task results testing the same two separate language models "Flan-T5" and "UL2", respectively, tuned using the described embodiment, represented by the row labeled "Combined". Table 520 also includes comparative lexicon matching task results for the same two language models tuned using random sampling and manual selection, represented by the rows labeled "Random" and "Manual", respectively. The results in table 520 also include averages over five seeds with standard deviations. It may be noted that the currently described embodiment provides similar F-1 scores for both models for the lexicon matching task.

[0057] It can thus be seen that the prompt optimization program 150 provides improved generation and employment of progressively optimized combination prompts for tuning the subject model. The described embodiments provide a progressive prompt design methodology that gradually improves the prompts while ensuring reliable performance. The described embodiments iteratively add prompt components by leveraging examples from the training dataset that result in the minimum loss, progressively increasing the specificity and relevance of the prompts to elicit more accurate responses from the subject model. The described embodiments may iteratively generate and employ progressively optimized combination prompts for tuning the subject model by processing subsequent batches of examples from the training dataset until there is no discernible benefit from updating the most recently updated prompts for the calculated minimum loss value or for a given validation set. Furthermore, the described embodiments have been shown to enable better or similar performance when compared to randomly or manually tuned language models while providing shorter prompts. This is beneficial since shorter prompts are preferred because they allow for fewer calculations from the LLM. The shorter prompts generated by the described embodiments thus allow for a more efficient and systematic search of the text space compared to manual searching.

[0058] It can be appreciated that Figures 2-5 are only intended to illustrate example implementations and are not intended to suggest any limitations on how different embodiments may be implemented. Many modifications to the depicted environments can be made based on design and implementation requirements.

[0059] The description of various embodiments of the present invention is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will become apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been selected to best explain the principles of the embodiments, practical applications, or technical improvements over the art found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-based method for generating and employing progressively optimized combination prompts for tuning a subject model, comprising: selecting a predetermined number of examples from the training data set; concatenating each of the selected examples with a current prompt of a target model to obtain a set of candidate prompts; calculating a loss value for each candidate prompt in the set of candidate prompts on a validation data set; and replacing the current prompt with the respective candidate prompt having a lowest calculated loss value that is lower than or equal to an original loss value for the validation data set for the current prompt to obtain an updated prompt. A computer-based method comprising:

2. processing subsequent batches of examples from the training data set in an amount corresponding to the predetermined number of examples until employing the progressively optimized combination of prompts results in a minimum loss value that is greater than or equal to a comparison loss value for the validation data set associated with a most recently updated prompt. iteratively generating and employing the progressively optimized combination prompts for tuning the target model by The computer-based method of claim 1 further comprising:

3. The computer-based method of claim 1 or 2, wherein the target model comprises a question answering system.

4. processing subsequent batches of examples from the training data set in an amount corresponding to the predetermined number of examples until a predetermined threshold number of examples have been incorporated into the prompt. iteratively generating and employing the progressively optimized combination prompts for tuning the target model by The computer-based method of claim 1 further comprising:

5. The step of selecting the predetermined number of examples from the training data set comprises: performing a centroid-based selection of said predetermined number of examples based on geometric techniques.

3. The computer-based method of claim 1 or 2, further comprising:

6. The step of selecting the predetermined number of examples from the training data set comprises: performing a random search of the training data set for hyperparameters that include the predetermined number of examples.

3. The computer-based method of claim 1 or 2, further comprising:

7. continually calculating a minimum loss value for the most recent prompt to provide a baseline loss value; The computer-based method of claim 1 or 2, further comprising:

8. 1. A computer system comprising: one or more processors, one or more computer readable memories, one or more computer readable tangible storage media, and program instructions stored on at least one of the one or more computer readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer readable memories. the computer system comprising: selecting a predetermined number of examples from the training data set; concatenating each of the selected examples with a current prompt of the target model to obtain a set of candidate prompts; calculating a loss value for each of the candidate prompts in the set of candidate prompts on a validation data set; and replacing the current prompt with the respective candidate prompt having a lowest calculated loss value that is lower than or equal to the original loss value for the current prompt on the validation data set to obtain an updated prompt; A computer system capable of performing a method comprising:

9. processing subsequent batches of examples from the training data set in an amount corresponding to the predetermined number of examples until a progressively optimized combination of prompts is employed to produce a minimum loss value that is higher than or equal to a comparison loss value for the validation data set associated with a most recently updated prompt. iteratively generating and employing the incrementally optimized combination prompts for tuning the target model. The computer system of claim 8 further comprising:

10. The computer system of claim 8 or 9, wherein the target model comprises a question answering system.

11. processing subsequent batches of examples from the training data set in an amount corresponding to the predetermined number of examples until a predetermined threshold number of examples have been incorporated into the prompt. Iteratively generating and employing incrementally optimized combination prompts for tuning the target model. The computer system of claim 8 further comprising:

12. The step of selecting the predetermined number of examples from the training data set comprises: performing a centroid-based selection of said predetermined number of examples based on geometric techniques.

10. The computer system of claim 8 or 9, further comprising:

13. The step of selecting the predetermined number of examples from the training data set comprises: performing a random search of said training data set for hyperparameters that include said predetermined number of examples; 10. The computer system of claim 8 or 9, further comprising:

14. A procedure that continually calculates the minimum loss value for the most recent prompt to provide a baseline loss value 10. The computer system of claim 8 or 9, further comprising:

15. To the processor: selecting a predetermined number of examples from the training data set; concatenating each of the selected examples with a current prompt of the target model to obtain a set of candidate prompts; calculating a loss value for each of the candidate prompts in the set of candidate prompts on a validation data set; and replacing the current prompt with the respective candidate prompt having a lowest calculated loss value that is lower than or equal to the original loss value for the current prompt on the validation data set to obtain an updated prompt; A computer program for executing the following:

16. the processor comprising: processing subsequent batches of examples from the training data set in an amount corresponding to the predetermined number of examples until a progressively optimized combination of prompts is employed to produce a minimum loss value that is higher than or equal to a comparison loss value for the validation data set associated with a most recently updated prompt. iteratively generating and employing the incrementally optimized combination prompts for tuning the target model.

16. The computer program product of claim 15 further comprising:

17. 17. The computer program product of claim 15 or 16, wherein the target model comprises a question answering system.

18. the processor comprising: processing subsequent batches of examples from the training data set in an amount corresponding to the predetermined number of examples until a predetermined threshold number of examples have been incorporated into the prompt. Iteratively generating and employing incrementally optimized combination prompts for tuning the target model.

16. The computer program product of claim 15 further comprising:

19. The step of selecting the predetermined number of examples from the training data set comprises: performing a centroid-based selection of said predetermined number of examples based on geometric techniques.

17. A computer program according to claim 15 or 16, further comprising:

20. The step of selecting the predetermined number of examples from the training data set comprises: performing a random search of said training data set for hyperparameters that include said predetermined number of examples; 20. The computer program of claim 18, further comprising:

Images