Neural network tuning using a text encoder

Abstract

Methods, systems, and non-transient computer-readable media for tuning a generative text-to-image neural network. Text prompts are processed using a pre-trained text encoder to acquire embedded text prompts, and the embedded text prompts are used by a pre-trained diffusion model to generate images. Reward scores are iteratively determined for images while the pre-trained diffusion model is fixed, and the weights of the pre-trained text encoder are updated to fine-tune the neural network to improve the quality of the generated images. Additionally, the reward scores for the images can then be determined while the updated weights of the text encoder are fixed to update the weights of the pre-trained diffusion model to further fine-tune the neural network.

Description

Technology Field Cross-reference regarding related applications

[0001] This application claims priority to U.S. application serial number 18 / 407,232 filed on January 8, 2024, the contents of which are incorporated herein by reference in their entirety.

[0002] The examples described herein generally relate to tuning a generative neural network, and in particular to tuning a text-to-image generative neural network by tuning the text encoder of the neural network based on the aesthetic quality of the generated images. Background Technology

[0003] Diffusion-based text-to-image generative models, such as Stable Diffusion (SD) used in image editing, super-resolution, and video synthesis, are bringing innovation to the field of content creation and enabling significant advancements in areas like image editing and video synthesis. Despite their capabilities, these models have limitations. For example, synthesizing images that align well with input text remains difficult, and multiple runs with carefully crafted prompts are often required to achieve satisfactory results. New techniques are needed to efficiently improve the results achievable with diffusion-based text-to-image generative models. Brief explanation of the drawing

[0004] In drawings that are not necessarily drawn to scale, similar numbers may represent similar components in different drawings. Some non-limiting examples are illustrated in the drawings of the attached drawings,

[0005] Figure 1 illustrates a diagram of a general process for tuning a text-to-image generative neural network.

[0006] Figure 2 is a flowchart illustrating a series of operations for tuning a text-to-image generative neural network.

[0007] FIG. 3 is a block diagram of an exemplary computing device for implementing one or more examples of the present disclosure. Specific details for implementing the invention

[0008] Various implementations and details are described with reference to examples for tuning a generative text-to-image neural network comprising a pre-trained text encoder and a pre-trained diffusion model. Text prompts are processed using the pre-trained text encoder to acquire embedded text prompts, and the embedded text prompts are used by the pre-trained diffusion model to generate images. Reward scores are iteratively determined for images while the pre-trained diffusion model is fixed, and the weights of the pre-trained text encoder are updated in response to the reward scores to fine-tune the neural network to improve the quality of the generated images. Additionally, the reward scores for the images can then be determined while the updated weights of the text encoder are fixed to update the weights of the pre-trained diffusion model in response to the reward scores to further fine-tune the neural network.

[0009] In one example, instead of replacing the text encoder used in a stable diffusion (SD) network architecture (e.g., Contrastive Language-Image Pre-Training (CLIP)) with another large language model, the present disclosure fine-tunes the pre-trained text encoder through the use of image quality compensation techniques, which leads to improvements in quantitative benchmarks and human evaluations. The techniques described herein also enable controllable image generation through the interpolation of different text encoders fine-tuned with various compensations. To fine-tune the text encoder, the techniques described herein may be combined with techniques for fine-tuning the diffusion model in a neural network architecture (e.g., a U-shaped encoder-decoder network architecture; UNet) to further improve generative image quality.

[0010] The techniques described herein utilize machine learning and various neural networks in various examples. As used herein, the term “machine learning” refers to the process of constructing and implementing algorithms capable of learning from data and making predictions about the data. Generally, machine learning can operate by building models from exemplary inputs, such as a training set of text-image pairs, to make data-driven predictions or decisions. Machine learning may include neural networks and / or machine learning models.

[0011] As used herein, the term “neural network” refers to a machine learning model that can be tuned (e.g., trained) based on inputs to approximate unknown functions. In particular, the term neural network may include a model of interconnected neurons that communicate and learn to approximate complex functions and generate outputs based on multiple inputs provided to the model. For example, the term neural network includes an algorithm (or set of algorithms) that implements deep learning techniques utilizing a set of algorithms to model high-level abstractions of data using supervised data to tune the parameters of the neural network.

[0012] Additionally, in one or more examples, the term neural network may include deep convolutional neural networks (i.e., "CNNs") or other types of deep neural networks. The description and figures below generally refer to a CNN comprising lower layers (e.g., convolutional, deconvolutional, and pooling layers) and upper layers (e.g., fully connected layers and classifiers).

[0013] As used herein, the terms “loss function” or “loss model” refer to a function that represents the error loss between feature vectors and / or probability vectors in a multidimensional vector space. Machine learning algorithms (e.g., neural networks) may be iteratively trained to minimize and / or maximize the error loss based on ground truths. For example, the loss function provides feedback to one or more layers of the neural network to tune or fine-tune those layers—this feedback is back-propagated. Examples of loss functions include the sigmoid unit function, a SoftMax classifier with cross-entropy loss, a residual loss function, a perceptual loss function, a total variance loss function, a texture loss function, a hinge loss function, and a least squares loss function.

[0014] FIG. 1 illustrates a general process (100) for training a text-to-image generative neural network comprising a text encoder (102), a diffusion model (104), and a reward model (106). In one example, the text encoder (102) is a conventional pre-trained text encoder (e.g., CLIP), and the diffusion model (104) is a conventional pre-trained diffusion model (e.g., SD such as UNet). The reward model (106) includes one or more models for analyzing the quality of images generated by the diffusion model (104) based on text prompts (118) input to the text encoder (102).

[0015] Despite being trained on large datasets, SD still faces two challenges. First, SD often generates images that are not well aligned with the provided prompts. Second, generating visually satisfactory images frequently requires multiple runs using different random seeds and manual prompt engineering. To address the first challenge, other researchers have replaced the CLIP text encoder used in conventional SD implementations with other large language models such as T5 and SDXL. Language models like T5 have an order of magnitude more parameters than CLIP, which results in additional storage and computational overhead. To address the second challenge, other researchers have fine-tuned the pre-trained UNet from SD using paired image-caption datasets. Nevertheless, models trained on limited datasets may still struggle to generate high-quality images for unseen prompts.

[0016] When reviewing the pipeline of text-to-image generation, the inventors recognized that both the text encoder and the diffusion model (e.g., UNet) affect the quality of the synthesized images. Unlike conventional techniques, the examples described herein fine-tune the pre-trained text encoder used in the generative model to improve performance, which results in better image quality and improved text-to-image alignment.

[0017] The examples provided herein utilize end-to-end fine-tuning techniques to enhance pre-trained text encoders. Instead of relying on paired text-image datasets, reward function models (e.g., models trained to automatically evaluate image quality, such as aesthetic models, and models that understand the performance of text-to-image models) can be used to improve the text encoder in a differentiable manner. By using text prompts during training, the techniques described herein enable on-the-fly synthesis of training images and reduce the burden of storing and loading large image datasets.

[0018] The examples described herein provide improvements in image quality and text-to-image alignment for well-trained text-to-image diffusion models. Compared to using larger text encoders, such as T5 and SDXL, this avoids additional computational and storage overhead. Compared to prompt engineering, this also reduces the risk of generating irrelevant content.

[0019] Accordingly, the examples provide an effective and stable text encoder fine-tuning pipeline supervised by (publicly available) compensation functions. Alignment constraints can be implemented to preserve the capability and generality of large-scale CLIP-pre-trained text encoders, which makes the approach described herein the first universal compensation fine-tuning paradigm among concurrent techniques. Furthermore, the text encoder fine-tuning described herein is orthogonal to UNet fine-tuning, so additional quality improvements can be obtained by subsequently fine-tuning UNet after fine-tuning the text encoder.

[0020] It is difficult to evaluate the performance of text-to-image models. Early methods used Prechet Inception Distance (FID:) to measure image quality. Automated metrics such as inception distance were used, and CLIP scores were used to evaluate text-to-image alignment. However, subsequent studies have pointed out that these scores exhibit limited correlations with human perception. To address these discrepancies, recent research has explored training models specifically designed to evaluate image quality for text-to-image models. Examples include image reward and human preference scores, which utilize human-annotated images to train quality estimation models. The examples described herein utilize these models in conjunction with image aesthetics models as reward functions to enhance visual quality and text-to-image alignment for text-to-image diffusion models.

[0021] In response to the inherent limitations of pre-trained diffusion models, various strategies have been proposed to enhance generation quality by focusing on aspects such as image color, composition, and background. One approach utilizes reinforcement learning to fine-tune diffusion models. Another field fine-tunes diffusion models using differentiable reward functions. Following this trend, subsequent research has extended the pipeline with trainable LoRA weights alongside text-to-image models. Unlike these strategies and trends, the examples described herein explore a novel approach to fine-tuning text encoders using differentiable reward functions, which is a previously unexplored dimension.

[0022] Another area of ​​research focuses on enhancing user-provided text to generate images of improved quality. Researchers use large language models, such as LLAMA, to refine or optimize text prompts. By improving the quality of the prompts, text-to-image models can synthesize higher-quality images. However, the use of additional language models introduces increased computational and storage requirements. By fine-tuning the text encoder according to the examples provided herein, the model can gain a finer understanding of the given text prompts and eliminate the need for additional language models and associated overhead.

[0023] Referring to FIG. 1, the end-to-end text encoder fine-tuning process (100) is based on prompt data and reward functions. Text prompts (118) from a user are forwarded to a previously trained text encoder (102). The text encoder (102) embeds the text prompts (118), and the embedded text prompts are forwarded to a diffusion model (104) (previously trained with text-image pairs). The diffusion model (104) may be a conventional SD diffusion model, such as a denoising diffusion probabilistic model (DDPM) or a denoising diffusion implicit model (DDIM), which generates intermediate diffusions in latent spaces (110 and 112), leading to a text-image pair diffusion (108) and an output image (114), respectively. The reward model (106) calculates reward loss scores (116), which are backpropagated to update the weights of the previously trained text encoder (102) (and optionally the previously trained diffusion model (104)) during the fine-tuning second training process to maximize the reward score from the reward model (106).

[0024] The diffusion model (104) transforms the actual data distribution, i.e., text-image pairs, into a noise distribution (108) (Z through a reversible noise removal process T ), for example, converts to a Gaussian distribution. To reduce computational cost, due to the number of noise removal steps, latent diffusion models (LDM) use a stable diffusion architecture such as UNet, where actual data is encoded through a variational autoencoder (VAE), to create latent image diffusion spaces (110)(Z t ) and (112)(Z t-1 A noise removal process is performed in ). The latent image data space (112) is decoded into an image (114) during the inference time.

[0025] Details regarding an example of process (100) are now described. Formally, the distribution (x, p) It is called a pair of actual image and prompt data extracted from (for notation simplification, x also represents data encoded by VAE), and Let be called a diffusion model with parameters θ, and cast Assuming a text encoder parameterized by , training a text-to-image LDM under the objective function of noise prediction can be formulated as follows in Equation 1: Here is actual noise; t is the time step; is a noise-added sample, and α t represents the signal, and σ t represents noise, both are determined by the scheduler; c is It is a text embedding that makes it so.

[0026] During the training of conventional SD models, text encoderT The weights are fixed. For example, a text encoder from a CLIP model is optimized through a contrastive objective between text and images. Therefore, it does not necessarily learn the semantic meaning of the prompt, which results in images generated using this text encoder not necessarily aligning well with a given prompt.

[0027] After the text-to-image diffusion model is trained, the system samples Gaussian noise for the same text prompt using numerous samplers, such as DDIM, which iteratively samples from t to the previous step t' using the denoising process expressed in Equation 2 below until t becomes 0:

[0028] One approach to improving generation quality during the sampling stage is through classifier-free guidance (CFG). By adjusting the guidance scale within the CFG, the system can further balance the trade-off between the fidelity of the synthesized image and text-image alignment. Specifically, in the case of the text-conditional image generation process, If is configured to represent null text input, classifier-free guidance can be defined as follows in Equation 3:

[0029] Two techniques for fine-tuning a text encoder using reward guidance include direct fine-tuning with rewards and prompt-based reward fine-tuning. These techniques are described sequentially below.

[0030] In the case of direct fine-tuning using rewards, in an exemplary general training process of diffusion models, the system forward diffusion, To perform this, sampling is done from actual data and random noise, and noise removal UNet is performed on this. A performs (noise) prediction. As in Equation 2, z t' Instead of calculating, the system alternatively predicts the original data as shown in the following Equation 4: Here is an estimated real sample, which is an image for the text-to-image diffusion model.

[0031] This formulation works for both pixel space and latent space diffusion models, and in latent diffusion It is post-processed by a VAE decoder before being input to the compensation models. Since the decoding process is also differentiable, this process is omitted from the formulation for simplification, and This is the predicted image. Once this is obtained, the system uses public reward models denoted by R to evaluate the quality of the generated image. Therefore, to improve the text encoder used in the diffusion model, the system uses a learning objective function that maximizes the quality scores predicted by the reward models to determine the weights of the text encoder, i.e., at T. Optimize.

[0032] In one example, the system is an image-based reward model , for example, aesthetic score predictors, and text-image alignment-based reward models For example, both Human Preference Score Version 2 (HPSV2) and PickScore are used. Consequently, the loss function for maximizing reward scores can be defined as follows in Equation 5: When optimizing Equation 5, the weights for all reward models and UNet models are fixed, whereas in one example, only the weights of the CLIP text encoder are modified.

[0033] For prompt-based compensation fine-tuning, specific text prompt p and initial noise Given, the noise removal process of Equation 2 is repeatedly solved It obtains, which can then be substituted into Equation 5 to calculate the reward scores. Consequently, the system It can accurately predict , and furthermore, the need for paired text-image data can be eliminated by performing compensation fine-tuning using only predefined denoising schedules and prompts, such as 25-step DDIM. Since each time step of the training process is differentiable, at T The gradient for updating can be calculated, for example, using the chain rule of Equation 6 as follows:

[0034] Solving Equation 6 involves the accumulation of the computation graph in the reverse chain, so the initial (noisy) time steps, i.e., t={ T , T It may be impossible to execute in terms of memory for -1, ..., ..., .... Gradient checkpointing can be utilized to trade memory for computational efficiency. Intuitively, intermediate results are recalculated on the fly. Therefore, training can be viewed as solving one step at a time. However, using gradient checkpointing, the system can technically train the text encoder for each time step. The proposed prompt-based reward fine-tuning is further exemplified in the following algorithm: algorithm

[0035] Compensated losses silver It is weighted by and can be linearly combined as shown in Equation 7 below:

[0036] Different reward functions with varying weights can be combined. However, some reward functions are inherently limited in terms of capability and training scale. Consequently, fine-tuning using only a single reward can lead to fatal forgetting and mode collapse.

[0037] To solve this problem, in one example, CLIP space similarity is set as an always-online constraint in Equation 8 as follows: And in Equation 7 It guarantees. Specifically, in this example, the system maximizes the cosine similarity between text embeddings and image embeddings. Text embeddings are obtained from forward propagation, while image embeddings are predicted images. CLIP's image encoder It is calculated by transmitting to. Original text encoder image encoder It is pre-trained in large-scale contrast learning paired with. Consequently, CLIP constraints preserve consistency between fine-tuned text embeddings and the original image domain, ensuring capability and generalization.

[0038] Fine-tuning approaches for text encoders are orthogonal to UNet reward fine-tuning, meaning that text encoders and UNets can be optimized under similar learning objective functions to further improve performance. The examples of fine-tuned text encoders described herein can be seamlessly fitted to pre-trained UNets of stable diffusion and can be used for downstream tasks other than text-to-image generation. To preserve this characteristic and avoid domain shift, UNets are fine-tuned text encoders It can be fine-tuned by fixing . The learning objective function for UNet is similar to Equation 6, where instead The parameters θ are optimized.

[0039] Reward weights in Equation 7 Instead of adjusting, alternatively, the system can train dedicated text encoders optimized for each reward and mix-match them during the inference phase for flexible and controllable generation.

[0040] The examples described herein provide a stable and robust framework for fine-tuning a pre-trained text encoder to improve text-to-image generation. By utilizing only prompt datasets and predefined reward functions, exemplary systems can improve generation quality compared to pre-trained text-to-image models, reinforcement learning-based approaches, and prompt engineering. To stabilize the reward fine-tuning process and avoid mode decay, a new similarity constraint paradigm may be implemented.

[0041] FIG. 2 depicts a flowchart (200) containing blocks citing a series of operations. In various examples, a text-to-image generative neural network described with reference to FIG. 1 implements the operations. The text-to-image generative neural network and the series of operations may be implemented by one or more computing devices, such as one or more server devices, one or more client devices, or a combination of server devices and client devices. Additionally, the series of operations of FIG. 2 may be implemented on one or more computing devices, such as a computing device (300) ( FIG. 3).

[0042] FIG. 2 illustrates a series of operations according to specific examples, but alternative examples may perform at least one of omitting, adding to, rearranging, or modifying any of the illustrated operations. The series of operations of FIG. 2 may be performed as part of a method. Alternatively, a non-transient computer-readable medium may include instructions that cause a computing device (e.g., at least one of a client device or a server device) to perform the series of operations of FIG. 2 when executed by one or more processors. In other examples, a system performs the series of operations of FIG. 2.

[0043] In block (202), a text encoder processes text prompts. In one example, the text encoder is a pre-trained text encoder, such as CLIP, used within a UNet neural network implemented on a computing device. The text encoder processes the text prompts to generate embedded text prompts for processing by the neural network.

[0044] In block (204), the diffusion model generates images based on text prompts. In one example, the diffusion model is a pre-trained SD model within a UNet neural network implemented on a computing device. The pre-trained SD model is pre-trained using many text-image data pairs. In response to receiving a text prompt (block (202)) from a text encoder, the pre-trained SD model generates an image.

[0045] In block (206), the reward model(s) implemented in the computing device iteratively determine reward scores for the generated images until convergence, with the diffusion model fixed. In one example, the reward model includes one or more aesthetic models configured to evaluate the quality of the images (block (204)) generated by the diffusion model with the diffusion model fixed.

[0046] In block (208), the computing device updates the weights of the text encoder based on reward scores (block (206)) to fine-tune the pre-trained text encoder while the diffusion model is fixed. In one example, the weights of the diffusion model are fixed during the fine-tuning of the pre-trained text encoder.

[0047] In block (210), the reward model(s) implemented in the computing device iteratively determine reward scores for the generated images with the text encoder fixed until convergence. In one example, the reward model includes one or more aesthetic models configured to evaluate the quality of the images (block (204)) generated by the diffusion model with the text encoder fixed. The reward model may be the same reward model used to determine the reward scores for the generated images with the diffusion model fixed (block (206)).

[0048] In block (212), the computing device updates the weights of the diffusion model based on the reward scores (block (210)) while the text encoder is fixed in order to fine-tune the pre-trained diffusion model while the text encoder is fixed. In one example, the weights of the text encoder are fixed during the fine-tuning of the pre-trained diffusion model.

[0049] As used herein, the term “digital environment” generally refers to an environment implemented, for example, as a standalone application (e.g., a personal computer or mobile application running on a computing device), as an element of an application, as a plugin for an application, as a library function or functions, as a computing device, and / or as a cloud computing system. A digital media environment enables a computing system to train and use a number of neural networks and / or machine learning models as described herein.

[0050] Examples of the present disclosure may include or utilize special-purpose or general-purpose computers, such as one or more processors and system memory, as discussed in more detail below. Examples within the scope of the present disclosure also include physical and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. In particular, one or more of the processes described herein may be implemented at least partially as instructions that are implemented on a non-transient computer-readable medium and executable by one or more computing devices (e.g., any media content access devices described herein). Generally, a processor (e.g., a microprocessor) receives instructions from a non-transient computer-readable medium (e.g., memory) and performs one or more processes including one or more of the processes described herein by executing said instructions.

[0051] Computer-readable media may be any available media that can be accessed by a general-purpose or special-purpose computer system. Computer-readable media that store computer-executable instructions are non-transient computer-readable storage media (devices). Computer-readable media that return computer-executable instructions are transmission media. Accordingly, as an example rather than a limitation, the examples of the present disclosure may include at least two distinctly different types of computer-readable media, namely non-transient computer-readable storage media (devices) and transmission media.

[0052] Non-transient computer-readable storage media (devices) include RAM, ROM, EEPROM, CD-ROM, solid-state drives (“SSDs”) (e.g., based on RAM), flash memory, phase-change memory (“PCM”), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store a desired means of program code in the form of computer-executable instructions or data structures and can be accessed by a general-purpose or special-purpose computer.

[0053] "Network" is defined as one or more data links that enable the transmission of electronic data between computer systems and / or modules and / or other electronic devices. When information is transmitted or provided to a computer via a network or other communication connection (wired, wireless, or a combination of wired and wireless), the computer appropriately regards that connection as a transmission medium. Transmission media may include networks and / or data links that can be used to transmit desired means of program code in the form of computer-executable instructions or data structures and can be accessed by a general-purpose or special-purpose computer. The above combinations must also be included within the scope of computer-readable media.

[0054] Furthermore, upon reaching various computer system components, means of program code in the form of computer-executable instructions or data structures can be automatically transferred from the transmitting media to non-transient computer-readable storage media (devices) (or vice versa). For example, computer-executable instructions or data structures received via a network or data link can be buffered in RAM within a network interface module (e.g., "NIC") and then eventually transferred to computer system RAM and / or less volatile computer storage media (devices) of the computer system. Therefore, it should be understood that non-transient computer-readable storage media (devices) may also (or even primarily) be included in computer system components utilizing the transmitting media.

[0055] Computer executable instructions include instructions and data that, for example, when executed by a processor, cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or group of functions. In some examples, computer executable instructions are executed by a general-purpose computer to convert the general-purpose computer into a special-purpose computer that implements the elements of the present disclosure. Computer executable instructions may be, for example, binaries, intermediate-form instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and / or methodological operations, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the features or operations described above. Rather, the described features and operations are disclosed as exemplary forms that implement the claims.

[0056] Those skilled in the art will understand that the present disclosure may be implemented in network computing environments having many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multi-processor systems, microprocessor-based or programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablets, pagers, routers, switches, etc. The present disclosure may also be implemented in distributed system environments where local and remote computer systems connected via a network (by wired data links, wireless data links, or a combination of wired and wireless data links) both perform tasks. In a distributed system environment, program modules may be located on both local and remote memory storage devices.

[0057] Examples of the present disclosure may also be implemented in cloud computing environments. As used herein, the term “cloud computing” refers to a model for enabling on-demand network access to a shared pool of configurable computing resources. For example, cloud computing may be used in marketplaces to provide ubiquitous and convenient on-demand access to a shared pool of configurable computing resources. A shared pool of configurable computing resources can be rapidly provisioned through virtualization, released with low management effort or service provider interaction, and then scaled accordingly.

[0058] A cloud computing model may be composed of various characteristics, such as, for example, on-demand self-service, extensive network access, resource pooling, rapid elasticity, and measured service. A cloud computing model may also expose various service models, such as, for example, Software as a Service (“SaaS”), Platform as a Service (“PaaS”), and Infrastructure as a Service (“IaaS”). A cloud computing model may also be deployed using different deployment models, such as private cloud, community cloud, public cloud, and hybrid cloud. Furthermore, as used herein, the term “cloud computing environment” refers to an environment in which cloud computing is used.

[0059] FIG. 3 illustrates a block diagram of an exemplary computing device (300) that may be configured to perform one or more of the processes described above. It will be understood that one or more computing devices, such as the computing device (300), may represent the computing devices described above. In one or more examples, the computing device (300) may be a mobile device (e.g., mobile phone, smartphone, PDA, tablet, laptop, camera, tracker, watch, wearable device, etc.). In some examples, the computing device (300) may be a non-mobile device (e.g., desktop computer or another type of client device). Furthermore, the computing device (300) may be a server device including cloud-based processing and storage capabilities.

[0060] As illustrated in FIG. 3, the computing device (300) may include one or more processor(s) (302), memory (304), storage device (306), input / output ("I / O") interfaces (308), and a communication interface (310) that can be coupled to communicate via a communication infrastructure (e.g., a bus (312)). Although the computing device (300) is illustrated in FIG. 3, the components exemplified in FIG. 3 are not intended to be limiting. Additional or alternative components may be used in other examples. Furthermore, in certain examples, the computing device (300) includes fewer components than those illustrated in FIG. 3. The components of the computing device (300) illustrated in FIG. 3 will now be described in additional detail.

[0061] In certain examples, the processor(s) (302) includes hardware for executing instructions such as those constituting a computer program. As an example, but not as a limitation, to execute instructions, the processor(s) (302) may retrieve (or fetch) instructions from an internal register, internal cache, memory (304), or storage device (306), decode them, and execute them.

[0062] A computing device (300) includes memory (304) coupled to a processor(s) (302). The memory (304) may be used to store data, metadata, and programs for execution by the processor(s). The memory (304) may include one or more volatile and non-volatile memories, such as random access memory (“RAM”), read-only memory (“ROM”), solid-state disk (“SSD”), flash, phase-change memory (“PCM”), or other types of data storage. The memory (304) may be internal or distributed memory.

[0063] The computing device (300) includes a storage device (306) for storing data or commands. As an example, but not as a limitation, the storage device (306) may include the non-transient storage medium described above. The storage device (306) may include a hard disk drive (HDD), a solid-state drive (SSD), flash memory, a universal serial bus (USB) drive, or a combination of these or other storage devices.

[0064] As illustrated, the computing device (300) includes one or more I / O interfaces (308) provided to allow a user to provide input to the computing device (300) (e.g., user strokes), receive output from the computing device (300), and otherwise transmit data to and from the computing device (300). These I / O interfaces (308) may include a mouse, a keypad or keyboard, a touch screen, a camera, an optical scanner, a network interface, a modem, other known I / O devices, or a combination of I / O interfaces (308). The touch screen may be activated by a stylus or a finger.

[0065] The I / O interfaces (308) may include one or more devices for providing output to a user, which include (but are not limited to) a graphics engine, a display (e.g., a display screen), one or more output drivers (e.g., display drivers), one or more audio speakers, and one or more audio drivers. In certain examples, the I / O interfaces (308) are configured to provide graphic data to a display for presentation to a user. The graphic data may represent one or more graphic user interfaces and / or any other graphic content capable of serving a particular implementation.

[0066] The computing device (300) may further include a communication interface (310). The communication interface (310) may include hardware, software, or both. The communication interface (310) provides one or more interfaces for communication (e.g., packet-based communication) between the computing device and one or more other computing devices or one or more networks. As an example, but not as a limitation, the communication interface (310) may include a network interface controller (NIC) or network adapter for communicating with Ethernet or other wired-based networks, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network such as WI-FI. The computing device (300) may further include a bus (312). The bus (312) may include hardware, software, or both for connecting the components of the computing device (300) to one another.

[0067] In the foregoing specification, the present invention has been described with reference to specific exemplary examples. Various examples and aspects of the invention(s) are described with reference to the details discussed herein, and the accompanying drawings illustrate various examples. The foregoing description and drawings are illustrative of the invention and should not be construed as limiting the invention. Numerous specific details are described to provide a thorough understanding of the various examples of the invention.

[0068] The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. The described examples should be regarded as illustrative and not limiting in all respects. For example, the methods described herein may be performed with fewer or more steps / actions, or the steps / actions may be performed in different orders. Additionally, the steps / actions described herein may be repeated or performed in parallel with one another, or in parallel with different instances of the same or similar steps / actions. Accordingly, the scope of the invention is defined by the appended claims rather than by the foregoing description. All modifications falling within the equivalent scope and meaning of the claims shall be included within that scope.

Claims

Claim 1 A method for tuning a generative text-to-image neural network comprising a pre-trained text encoder and a pre-trained diffusion model, comprising: processing text prompts using the pre-trained text encoder to obtain embedded text prompts; generating images in response to the embedded text prompts using the pre-trained diffusion model; iteratively determining reward scores for the images until convergence while the pre-trained diffusion model is fixed; and updating the weights of the pre-trained text encoder in response to the iteratively determined reward scores. Claim 2 A method according to claim 1, further comprising: a step of repeatedly determining reward scores for the images until convergence while the pre-trained text encoder having updated weights is in a fixed state; and a step of updating the weights of the pre-trained diffusion model in response to the reward scores repeatedly determined for the images until convergence while the pre-trained text encoder having updated weights is in a fixed state. Claim 3 In paragraph 1, the step of repeatedly determining the above reward scores is A method comprising the step of evaluating the quality of the image using an aesthetics model. Claim 4 A method according to claim 1, further comprising the step of processing the images with a variational autoencoder decoder before iteratively determining the reward scores for the images. Claim 5 In claim 1, the step of iteratively determining reward scores for the images until convergence while the pre-trained diffusion model is fixed is A method comprising a step of maximizing quality scores predicted by one or more reward models. Claim 6 In paragraph 5, the method wherein one or more of the above-mentioned compensation models include an image-based compensation model. Claim 7 In claim 6, the method wherein the one or more of the above-mentioned compensation models further include a text-image alignment-based compensation model. Claim 8 In paragraph 5, the step of iteratively determining reward scores for the images until convergence while the pre-trained diffusion model is fixed is A method further comprising the step of fixing the weights of one or more of the above-mentioned reward models. Claim 9 In claim 1, the pre-trained text encoder includes a contrastive language-image pre-training (CLIP) model, and the method is A method comprising the step of setting the similarity of the CLIP model as an always-on constraint while maintaining the weights of the CLIP model greater than 0. Claim 10 In claim 9, the step of setting the similarity of the CLIP model as an always-on constraint while maintaining the weights of the CLIP model greater than 0 is: A method comprising the step of maximizing the cosine similarity between text embeddings by the pre-trained text encoder and image embeddings by the pre-trained diffusion model. Claim 11 A system for tuning a generative text-to-image neural network, comprising: a pre-trained text encoder that processes text prompts to acquire embedded text prompts; a pre-trained diffusion model that generates images in response to said embedded text prompts; and a reward model that iteratively determines reward scores for said images until convergence while said pre-trained diffusion model is fixed, wherein the weights of said pre-trained text encoder are updated in response to said iteratively determined reward scores. Claim 12 A system according to claim 11, wherein the reward model additionally determines reward scores for the images until convergence while the pre-trained text encoder having updated weights is fixed; and the weights of the pre-trained diffusion model are updated in response to the reward scores repeatedly determined for the images until convergence while the pre-trained text encoder having updated weights is fixed. Claim 13 In claim 11, the system comprises an aesthetic model that iteratively determines reward scores by evaluating the quality of the images. Claim 14 A system according to claim 11, further comprising a variational autoencoder decoder that processes the images before iteratively determining the reward scores for the images. Claim 15 In claim 11, the system wherein the reward model iteratively determines reward scores for the images in a fixed state until convergence by maximizing quality scores predicted by one or more reward models. Claim 16 In paragraph 15, the system wherein one or more of the above-mentioned compensation models include an image-based compensation model. Claim 17 In paragraph 16, the system wherein one or more of the above-mentioned reward models further include a text-image alignment-based reward model. Claim 18 In claim 11, the pre-trained text encoder comprises a contrastive language-image pre-learning (CLIP) model having similarity constraints, wherein the similarity constraints are set to always be on and the weights of the CLIP model are set to be greater than 0, a system. Claim 19 A non-transient computer-readable medium comprising instructions for tuning a generative text-to-image neural network comprising a pre-trained text encoder and a pre-trained diffusion model, wherein, when the instructions are executed by a processor, To process text prompts using the pre-trained text encoder to obtain embedded text prompts; Using the above-mentioned pre-trained diffusion model, to generate images in response to the above-mentioned embedded text prompts; To allow the above-mentioned pre-trained diffusion model to iteratively determine reward scores for the above images until convergence while in a fixed state; and A non-transient computer-readable medium comprising a generative text-to-image neural network to update the weights of the pre-trained text encoder in response to the repeatedly determined reward scores. Claim 20 In paragraph 19, when the above instructions are executed by the processor, The pre-trained text encoder having updated weights is fixed and iteratively determines reward scores for the images until convergence; and A non-transient computer-readable medium further comprising a generative text-to-image neural network to update the weights of the pre-trained text encoder having the updated weights in response to reward scores repeatedly determined for the images until convergence in a fixed state.

Images