Prompt element generation as input to generative model

Abstract

An example embodiment of the present disclosure provides an example method for generating a prompt element for use as an input to generate a model. The example method includes obtaining user input data including initial cue data. The example method includes selecting one or more suggested cue elements based at least in part on initial cue data. The example method includes transmitting one or more suggested cue elements to be presented for display via a user interface as selectable user interface elements. The example method includes obtaining second user input data, the second user input data including data indicating a selection of one or more suggested cue elements. The example method includes, in response to obtaining second user input data, providing the second user input to an image generation model to generate an output image including a visual representation associated with one or more suggested cue elements.

Description

Technical Field

[0001] The present disclosure generally relates to the generation of prompt elements for use as input to a generative model. Background Art

[0002] Generative models include machine learning models that are able to generate image outputs based on the input data obtained. Hint engineering involves providing better input hints to receive better-than-average outputs from generative models. Summary of the invention

[0003] Various aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0004] In one example aspect, the present disclosure provides an example computer-implemented method. The example computer-implemented method includes obtaining, by a prompt element suggestion model, user input data including initial prompt data from a client device. The example computer-implemented method includes selecting, by the prompt element suggestion model, one or more suggested prompt elements based at least in part on the initial prompt data. The example computer-implemented method includes transmitting, by the prompt element suggestion model, one or more suggested prompt elements to be presented to a client device to be displayed as selectable user interface elements via a user interface. The example computer-implemented method includes obtaining, by the prompt element suggestion model, second user input data from a client device, the second user input data including data indicating a selection of one or more suggested prompt elements. The example computer-implemented method includes, in response to obtaining the second user input data, providing the second user input to an image generation model to generate an output image including a visual representation associated with the one or more suggested prompt elements.

[0005] In some embodiments, the example computer-implemented method includes updating a prompt element suggestion model based on second user input data.

[0006] In some embodiments of the example computer-implemented method, the prompt element suggestion model includes a machine learning language model.

[0007] In some embodiments of the example computer-implemented method, the image generation model is configured to perform an operation. The computer-implemented method may include: generating an output image based on the obtained second user input data; performing a verification operation based on the output image; and updating the image generation model based on the performed verification operation.

[0008] In some embodiments of the example computer-implemented method, the system includes a prompt element generation component, the prompt element generation component including a machine learning language model configured to generate one or more prompt elements. In some embodiments, generating a suggested prompt element includes: obtaining initial prompt data and context data; obtaining one or more content elements from a content provider inventory database; and generating the one or more prompt elements based on the initial prompt data, the context data, and the one or more content elements from the content provider inventory database. In some embodiments, generating the suggested prompt element includes: determining a similarity between a first characteristic associated with a first content element from the content provider inventory database that is consistent with the obtained context data; and generating a first prompt element based on the determined similarity and the first content element.

[0009] In some embodiments of the example computer-implemented method, the machine learning model for generating the output image comprises an image generation model. In some embodiments of the example computer-implemented method, the method comprises tuning the image generation model by: providing a training prompt as an input to the image generation model; obtaining an output from the image generation model comprising a generated image; comparing the output comprising the generated image to an approved image associated with the training prompt; and tuning the image generation model based on comparing the output comprising the generated image to the approved image associated with the training prompt.

[0010] In some embodiments of the example computer-implemented method, one or more suggested prompt elements are generated in real-time.

[0011] In some embodiments of the example computer-implemented method, one or more suggested hint elements and associated data are stored in a cache.

[0012] In some embodiments of the example computer-implemented method, one or more suggested hint elements are selected from a cache by: determining a distance between initial hint data and data associated with one or more hint elements and associated data stored in the cache; and selecting a first hint element and associated data stored in the cache based on the distance between the initial hint data and data associated with the first hint element of the one or more hint elements.

[0013] In some embodiments of an example computer-implemented method, an image generation model generates an interactive image including embedded information that, when accessed, causes a user interface to automatically update with a browser associated with a prompt element for selection serving as a prompt for the image generation model.

[0014] In some embodiments of the example computer-implemented method, selecting, by the prompt element suggestion model, one or more suggested prompt elements is based on at least one of a quality associated with the suggested prompt elements, a relevance of the suggested prompt elements, or a ranking of the suggested prompt elements.

[0015] In some embodiments of the example computer-implemented method, selecting one or more suggested prompt elements by the prompt element suggestion model includes: determining one or more keywords based on obtained initial prompt data; transmitting a request for data including one or more prompt elements associated with the one or more keywords; and obtaining one or more prompt elements associated with the one or more keywords.

[0016] In some embodiments of the example computer-implemented method, the prompt element suggestion model is a component of the image generation model.

[0017] In some embodiments of the example computer-implemented method, the image generation model is a generative machine learning model.

[0018] In some embodiments of the example computer-implemented method, the prompt element data includes a token.

[0019] In some embodiments of the example computer-implemented method, selecting, by the prompt element suggestion model, one or more suggested prompt elements is performed based at least in part on a bidding process.

[0020] In an example aspect, the present disclosure provides an example system for generating a prompt element for use as an input to a generation model, the system comprising one or more processors and one or more memory devices, the one or more memory devices storing instructions executable to cause the one or more processors to perform operations. In some implementations, the one or more memory devices may include one or more transient or non-transitory computer-readable media, the one or more transient or non-transitory computer-readable media storing instructions executable to cause the one or more processors to perform operations. In the example system, the operation may include obtaining, by the prompt element suggestion model, user input data including initial prompt data from a client device. In the example system, the operation may include selecting, by the prompt element suggestion model, one or more suggested prompt elements based at least in part on the initial prompt data. In the example system, the operation may include transmitting, by the prompt element suggestion model, one or more suggested prompt elements to be presented to the client device, to be displayed as selectable user interface elements via a user interface. In the example system, the operation may include obtaining, by the prompt element suggestion model, second user input data from the client device, the second user input data including data indicating a selection of one or more suggested prompt elements. In an example system, the operations may include, in response to obtaining second user input data, providing the second user input to an image generation model to generate an output image including a visual representation associated with one or more suggested prompt elements.

[0021] In an example aspect, the present disclosure provides an example transient or non-transitory computer-readable medium, which is embodied in a computer-readable storage device and stores instructions that, when executed by a processor, cause the processor to perform operations. In the example transient or non-transitory computer-readable medium, the operation includes obtaining, by a prompt element suggestion model, user input data including initial prompt data from a client device. In the example transient or non-transitory computer-readable medium, the operation includes selecting, by the prompt element suggestion model, one or more suggested prompt elements based at least in part on the initial prompt data. In the example transient or non-transitory computer-readable medium, the operation includes transmitting, by the prompt element suggestion model, to the client device one or more suggested prompt elements to be presented, to be displayed as selectable user interface elements via a user interface. In the example transient or non-transitory computer-readable medium, the operation includes obtaining, by the prompt element suggestion model, second user input data from the client device, the second user input data including data indicating a selection of one or more suggested prompt elements. In an example transitory or non-transitory computer-readable medium, the operations include, in response to obtaining second user input data, providing the second user input to an image generation model to generate an output image including a visual representation associated with one or more suggested prompt elements. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] A detailed discussion of the embodiments for those skilled in the art is set forth in this specification with reference to the accompanying drawings, in which:

[0023] Figure 1 depicts a block diagram of an example system for performing a prompt element for suggesting a prompt element for use as input to a generative model according to an example embodiment of the present disclosure;

[0024] Figure 2 A swim lane diagram depicting an example method for performing a prompt element for suggesting a prompt element for use as input to a generative model according to an example embodiment of the present disclosure;

[0025] Figure 3 A flow chart depicting an example data flow for retrieving and matching prompt elements used as input to generate a model according to an example embodiment of the present disclosure;

[0026] Figure 4 depicts an example database for storing prompt elements and associated keyword data;

[0027] Figure 5 A flow chart depicting an example data flow for suggesting prompt elements for use as input to a generative model according to an example embodiment of the present disclosure;

[0028] Figure 6depicts a flow chart for training a machine model to be fine-tuned to perform a prompt element that suggests use as input to a generative model, according to an example embodiment of the present disclosure;

[0029] Figure 7 An example method for suggesting prompt elements for use as input to a generative model according to an example embodiment of the present disclosure is depicted; and

[0030] Figure 8 Depicted is a block diagram of an example computing system that performs suggestion elements for use as input to a generative model, according to an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The present disclosure provides improved suggested prompt element data for input into a generative model. For example, the generative model can be an image generation model, a language model, or a group of models capable of generating images, audio-visual or other forms of content as output. The present disclosure may include obtaining an initial input provided by a user via a client device, and in response, generating one or more suggested prompt elements to present to the user to complete the initially obtained prompt. The suggested prompt element generation model can be trained using a feedback loop by obtaining data associated with the generated output and adjusting the model.

[0032] The suggested prompt elements may be generated by a first party computing system associated with the generation model or by a third party content provider computing system. The suggested prompt elements may be generated based on content elements or content items stored in a content provider database. The suggested prompt elements may be generated in real time or may be retrieved from a cached storage device based on similarities between initial inputs. Some implementations may include a selection process that may include a bidding structure for selecting one or more suggested prompt elements to display to a user. The prompt elements may include tokens, words, optional elements, or other data used as input to the generation model to create an output content item.

[0033] As described herein, a client device may obtain user input data including an initial prompt. The initial prompt may be obtained by an image generation model or a prompt element suggestion component associated with the image generation model. The prompt element suggestion component associated with the image generation model may transmit data including the initial prompt and context data to a prompt element generation system (e.g., a content provider prompt element generation system or a third-party prompt element generation system). The prompt element generation system may include a prompt element generation model that may generate or otherwise collect one or more suggested prompt elements. The suggested prompt elements may be obtained by the prompt element suggestion component. The prompt element suggestion component may perform a selection process to select one or more prompt elements to transmit to the client device. The selection process may include scoring or ranking the prompt elements.

[0034] At the client device, one or more selected prompt elements may be provided via an image generation application. The image generation application may obtain input including a selection of the one or more selected prompt elements, a rejection of one of the one or more selected prompt elements, or a more free-form input obtained from a user. The prompt element suggestion component may use the refined prompts or other feedback data to train the prompt element suggestion component to make better suggestions in the future.

[0035] The technology of the present disclosure can provide a variety of technical effects and benefits. For example, various aspects of the technology can allow the number of calls to the generation model to be reduced by generating higher quality prompts, and the higher quality prompts enable the generation model to produce higher quality outputs. By recommending prompt elements to users, the system can reduce the processing and errors caused by incomplete or incorrect prompts. Prompt engineering can be a difficult task, and determining the appropriate prompts to be input into the generation model to obtain a satisfactory image or other output may result in iterative calls to the generation model, which may waste processing resources and bandwidth due to redundant calls to the model. The present disclosure can predict the user's intention based on the initial prompt and context data, and can provide suggested prompt elements based on the initial prompt, context data, or additional selection criteria. In addition, by using feedback loops to train and update models (e.g., language models, machine learning models, large language models), the model can be continuously fine-tuned and trained to generate better suggested prompt elements. This can additionally reduce the number of updated prompts provided by users and reduce the number of calls to the image generation model.

[0036] Improvements associated with the systems and methods discussed herein may be further understood with reference to the accompanying drawings.

[0037] Figure 1 A block diagram of an example system 100 for recommending suggested prompt elements for use as input to a generation model according to an example embodiment of the present disclosure is depicted. The system 100 may include a prompt element suggestion component 105, an image generation component 120, a prompt element generation component 125, a client device 135, a prompt element database 150, or a content provider database communicatively connected via a network 160.

[0038] The prompt element suggestion component 105 may include a prompt element retrieval component 110 or a prompt element scoring component 115. The prompt element suggestion component 105 may include one or more models, including, for example, a prompt element suggestion model. The prompt element retrieval component 110 may be configured to retrieve one or more suggested prompt elements in response to receiving a request for suggested prompt elements. Additionally or alternatively, the prompt element retrieval component 110 may be configured to match one or more suggested prompt elements with an initial input data request. The request for suggested prompt elements may be transmitted by a client device 135 or an image generation component 120. For example, the client device 135 may include an image generation application 145. The image generation application 145 may be associated with a graphical user interface that may provide an interactive display via a display of the client device 135. The image generation application 145 may obtain user input data via the user interface. The user input data may be transmitted to the image generation component 120 or the prompt element suggestion component 105 to request one or more relevant suggested prompt elements. For example, the image generation application 145 may include a user input component capable of obtaining an initial prompt provided by a user as an input to the image generation model 140 .

[0039] As described herein, the image generation model 140 may include a language model capable of obtaining user input including an initial prompt, determining an intent associated with the initial prompt, and generating an image based on the obtained initial prompt. In some implementations, the image generation model 140 may include one or more models that are communicatively coupled to perform an image generation task based on the obtained user input (e.g., the initial prompt).

[0040] In some instances, the initial prompt data may include data indicating that an image generation application is being launched (e.g., a user opens an application, a user begins an application session). For example, before the user provides any text or natural language input, the system may perform the methods and operations described herein to suggest or generate prompt element data to be provided for display to the user via an interface associated with the image generation application. In some instances, the image generation component 120 may be called by other applications to generate images (e.g., communicating with a web-based application, communicating with a social media application, communicating with a search application, or any other application).

[0041] After receiving the initial user prompt data (e.g., by an interface associated with the image generation application 145), the client device 135 may transmit a request for one or more prompt elements to be recommended (e.g., to the user) via the image generation application 145. The suggested prompt elements may be obtained by the prompt element suggestion component 105 and transmitted to the client device 135 to be provided for display as selectable user interface elements.

[0042] In some implementations, the prompt element retrieval component 110 may obtain the prompt element from the prompt element generation component 125, the prompt element database 150, or the content provider database 155. The prompt element generation component 125 may include one or more prompt element generation components. The prompt element generation component 125 may be associated with a content provider or a content management system. The prompt element generation component 125 may include one or more prompt element generation models 130. The prompt element generation model 130 may include one or more models capable of obtaining user input (including initial prompts), determining user intent, and generating one or more recommended prompt elements. Additionally or alternatively, the prompt element generation model 130 may generate additional data (e.g., keywords, associated data, characteristic data) associated with one or more prompt elements. For example, the prompt element generation model 130 may include a generation model capable of obtaining natural language input or other structured input and generating a structured output including suggested prompt elements.

[0043] The prompt elements may be transmitted to the prompt element suggestion component 105 for use in near real time or may be transmitted for storage in a data store such as the prompt element database 150 or the content provider database 155. In some implementations, the prompt elements may be cached in a local storage device on the client device 135. Thus, by performing on-device selection of suggested prompt elements based on comparison with cached prompt elements, latency may be reduced compared to initial prompt input data.

[0044] Prompt element database 150 may include one or more data stores that may store data including prompt elements and associated prompt element data. For example, prompt element data may include suggested prompt elements generated by prompt element generation component 125. Additionally or alternatively, prompt element data may include data obtained via user input (e.g., by a user associated with a content management computing system or a content provider computing system). Prompt element data may additionally or alternatively include data including keywords associated with prompt elements or other characteristic data associated with prompt elements.

[0045] The content provider database 155 may include one or more data stores associated with one or more content providers. The content provider database 155 may store content provider data. The content provider data may include content element components, prompt element data (e.g., as discussed with respect to prompt element database 150), content element performance data, or any other relevant content provider data.

[0046] The prompt element scoring component 115 can obtain one or more prompt elements. The prompt element scoring component 115 can determine the relevance of the prompt element to the initial prompt. The relevance can be used to generate a score for each of the prompt elements. In some instances, the relevance can be determined based on an analysis of one or more models of the prompt element. In some instances, one or more prompt elements can have a pre-assigned relevance with some common initial prompt inputs. In some instances, the relevance can be determined based on a semantic relationship graph between terms. The system can compare candidate prompt elements and initial prompt input data with a semantic graph to determine the degree of separation between corresponding terms. In some instances, the semantic graph can be updated based on data indicating which suggested prompt elements are ultimately selected by the user and used as input to the image generation model.

[0047] Additionally or alternatively, the prompt elements may be ranked. The prompt element suggestion component 105 may transmit data including the prompt element data to the image generation component 120 or the client device 135. The prompt element data may be obtained by the image generation application 145 and may be provided for display via a user interface of the client device 135. The user interface may obtain data including the user's selection of the prompt and transmit data including the updated prompt to the image generation component 120.

[0048] Figures 2 to 6 The discussion of data flow throughout the system described herein will continue.

[0049] Figure 2 An example swim lane diagram of a data flow 200 is depicted according to an example embodiment of the present disclosure. For example, the data flow 200 may include a client device 205, an image generation component 210, a prompt element suggestion component 215, and a prompt element generation component 220.

[0050] Initial prompt data 225 can be obtained at the client device 205 and transmitted to the image generation component 210. The image generation component 210 can generate a request for a prompt element (e.g., automatically) in response to obtaining the initial prompt data. The image generation component 210 can transmit the request for the prompt element 230 to the prompt element suggestion component 215. The prompt element suggestion component 215 can obtain the request for the prompt element 230 and transmit data including the initial prompt data and context data 235. The context data can be obtained from the client device 205, the image generation component 210, or the prompt element suggestion component 215.

[0051] Initial prompt data may include natural language input, selection of previously provided prompts, or any other input. In some instances, the input may be provided by a user via an input component. In some instances, the input may be provided by another computing system, or the input may be automatically provided in response to a client device or completion of a certain preliminary action that serves as a trigger for initiating the process. In some instances, the initial prompt data may include a natural language indication of the content that the user wishes the image generation component 210 to generate. For example, a user may provide an initial input "a cat sits on a chair" or "a man wins a marathon" or any other input prompt so that the image generation component 210 generates an output image based thereon.

[0052] Additionally or alternatively, the user may be engaged in a search session or a conversation with another application. The application or a model associated with the application may determine user intent based on the conversation and automatically initiate a prompt element suggestion pipeline. This may not only produce a recommendation for a suggested prompt element, but may additionally or alternatively provide a recommendation to launch an image generation application or an image generation component of another application.

[0053] The context data may include data associated with the context of the client device image generation application session. For example, the context data may include data about previous prompts. In some instances, the image generation component 210 may be one of many language models utilized by the client device. For example, a user may participate in a session with a single input component that is connected to a language model, a search model, an image generation model, or other generation models that provide output in response to receiving an input prompt from the user. The user may be directed to a session with the image generation component 210.

[0054] For example, the system can initiate a user session based on obtaining user input indicating a selection of an application to launch. The system can obtain user input including questions about "planning a birthday party," "designing invitations," and "designing party graphics." The model can provide recommendations for a party, and initiate a session with an image generation model to generate party invitations or graphics.

[0055] The image generation component 210 can obtain context data associated with the session for selecting or generating prompt elements to be suggested to the user to improve the user's input prompt.

[0056] The prompt element generation component 220 can generate or retrieve one or more prompt elements and associated data 240 in response to obtaining the initial prompt data and the context data. Figure 5 Generating one or more prompt elements is described in more detail.

[0057] For example, continuing with the birthday party example, the prompt element generation component 220 can generate or retrieve one or more prompt elements. The prompt element can be associated with a specific type of cake that can be purchased at a local bakery or party location (such as a local amusement park, restaurant, or event venue). The prompt element can have additional associated data (e.g., metadata) that can be invisible to the user but can be read by the image generation component 210 to generate one or more images consistent with the selected prompt element and the associated data.

[0058] Prompt element data 245 may be transmitted from the prompt element generation component 220 to the prompt element suggestion component 215 .

[0059] One or more operations may be performed at the prompt element suggestion component 215. For example, the prompt element suggestion component 215 may retrieve one or more prompt elements 250 or score one or more prompt elements 255. For example, the prompt element suggestion component 215 may obtain one or more prompt element data 245 and determine which prompt elements are most relevant given the initial prompt data and the context data 235. In some instances, the prompt element suggestion component 215 may score and rank one or more prompt elements. For example, the prompt element suggestion component 215 may determine the score of each of the corresponding prompt elements to determine whether to provide the prompt element as a suggested prompt element (e.g., transmitting data that can be read by the client device to provide the suggested prompt element for display via the interface of the client device 205). In some instances, multiple prompt elements may be ranked. The ranking of each corresponding prompt element may be used to determine the order in which the ranked prompt elements are presented via the user interface on the client device 205.

[0060] Selected prompt element data 260 associated with the one or more suggested prompt elements may be transmitted from the prompt element suggestion component 215 to the client device 205. The selected prompt element data 260 may include data associated with the one or more selected prompt elements, including content to be displayed via the user interface, additional metadata that may be provided for input into the generation model, scores for the corresponding prompt elements, or other relevant data.

[0061] At the client device 205, one or more selected prompt elements may be provided to the user for display. The user may select one or more suggested prompt elements for use as input to the image generation model of the image generation component 210. Updated prompt data 265 indicating the prompt elements selected by the user may be transmitted from the client device 205 to the image generation component 210. For example, the user may provide input to select (or reject) one or more of the selected prompt elements. The image generation model may obtain data including the updated prompt data 265, and in response to obtaining the updated prompt data 265, the image generation model may automatically generate an output. The output may include output data 270 that may be transmitted from the image generation component 210 to the client device 205. The output data 270 may include an image 275 or embedded information 280. The image 275 may include a graphical description of the interpretation of the updated prompt data 265 by the image generation model.

[0062] Returning to the birthday party planning example, an initial prompt may be "Invite Jim to a birthday party on July 21, 2023 at 7pm". One or more selected prompt elements may be associated with one or more locations or activities in Jim's geographic area. For example, the selected prompt elements may include "Italian restaurant", "nightclub", "game arcade", and "skating rink". The user may select and utilize the game arcade, resulting in the generation of updated prompt data. For example, the updated prompt may include "Create an invitation to Jim to attend a birthday party at the game arcade at 7pm on July 21, 2023. The invitation is themed with old arcade games". The output data may include a game arcade-themed invitation containing the date, time, location, and other information. In some instances, the image generation application may include a conversational interface. The conversational interface may allow follow-up questions to further customize the recommendation and provide improved prompt element suggestions.

[0063] The image generation component 210 and the prompt element suggestion component 215 can obtain feedback data 285. The feedback data 285 can be used by the image generation component 210 or the prompt element suggestion component 215 to update the corresponding model associated with the corresponding component. For example, the feedback data 285 can be used to verify the image generated by the image generation component 210. The feedback data 285 can be used to improve the retrieval, matching, scoring or ranking of the prompt elements by the prompt element suggestion component 215.

[0064] As described herein, feedback data 285 may include survey response data 290 or refined prompt data 295. For example, a survey may be provided to be displayed via a user interface on a client device 205 or a different client device 205, and the survey may be ranked or scored by a human observer. In some instances, an automatic preliminary check or score may be performed to rate the satisfaction of the image generated by the image generation model. In some instances, an automatic preliminary check or score may be performed to rate the satisfaction of the prompt element suggestion provided as a prompt. In some instances, the computing system may compare the output image with the output image expected based on the prompt element (e.g., labeled training data) suggested by previous use. In some instances, the suggested prompt element may include sponsorship or brand content. In some implementations, a feedback system may be provided to determine whether the image generated by the image generation component 210 is satisfactory or consistent with the brand guidelines of the content provider.

[0065] In some instances, satisfaction with the recommended prompt elements can be determined based on the refined prompt data 295. For example, if the client device 205 obtains input from the user rejecting a suggested prompt, the system can infer that the suggested prompt is not as relevant as predicted from the initial prompt. This feedback data 285 can be used as training data for a personalized model located on the client device or a general model located on a server computing system or in some way separate from the client device. Figure 6 and Figure 8 Model training is further described.

[0066] Additionally or alternatively, the refined prompt data 295 may be obtained after the output image has been displayed by the user interface of the client device 205. For example, the client device may obtain user input that includes a more detailed prompt, a prompt that is different from the initial prompt, or a prompt that is different from the updated prompt. In some implementations, the system may perform the prompt element suggestion process until an indication of user satisfaction is obtained. For example, the system may continue to provide recommended prompt elements to the user as the user refines the input prompt until the user stops requesting updated images from the image generation component 210.

[0067] In some implementations, the system may perform the prompt element suggestion process a limited number of times before allowing the user to refine the input prompt. The image generation component 210 may determine a satisfaction score associated with the output image based at least in part on the number of times the prompt is refined after receiving the initial output image. This feedback may be used to train the image generation component 210 to improve the output images generated in future image generation sessions.

[0068] In some implementations, the prompt element suggestion component 215 is part of the image generation component 210. The prompt element suggestion component 215 can perform a number of operations. The operations may include Figure 3 For example, Figure 3 An example data flow 300 for retrieving and matching cue elements according to an example embodiment of the present disclosure is depicted. For example, the data flow 300 may include transmitting data from a cue element suggestion component 305, a keyword server 310, and a content management server 315.

[0069] The keyword fingerprint retrieval component 320 may transmit the initial user prompt 325 or the context data 327 to the keyword server 310. The keyword server 310 may include a user prompt keyword expansion component 330 and a keyword database 335.

[0070] The user prompt keyword expansion component 330 can obtain the initial user prompt 325 and the context data 327, and determine a plurality of relevant keyword fingerprints to be attached to the initial user prompt. In some implementations, the keyword server can be associated with a content provider. In some implementations, the keyword server can be associated with a content management service provider. The initial user prompt 325 and the context data 327 can be transmitted as a data structure in any form. The data structure can include, for example, a standard HTTP request frame. For example, the initial user prompt 325 and the context data 327 can be encoded in a data binary long object using a standard format {prompt element, keyword value} pair. In some implementations, standard prompt element templates can be created and stored based on various themes and category requirements. In some implementations, themes can be associated with different product types (e.g., vertical products). The initial user prompt 325 and the context data 327 can be partially randomized or otherwise encoded to remove any personal data (e.g., personally identifiable information).

[0071] In some instances, keywords and associated prompts may be stored in a keyword database 335. Thus, the keyword server 310 may generate keywords or otherwise retrieve keywords associated with the obtained initial user prompt 325. The keyword server 310 may transmit keyword fingerprint data 340 to the keyword fingerprint retrieval component 320. In some implementations, multiple content provider server computing systems may provide prompt elements that match the initial user prompt. Pairs of keyword fingerprints and provided prompt elements may be stored in a content management server database (e.g., prompt element database 355). Each prompt element may be stored in a data structure that includes a prompt element and one or more prompt element fingerprints (e.g., keyword fingerprints).

[0072] In some examples, the prompt elements may be stored in the form of key values ​​(keyval), where the key is the keyword fingerprint and the value is a list of prompt element fingerprints associated with the keyword fingerprint. Thus, the keyword fingerprint retrieval component 320 may obtain a plurality of keyword fingerprints associated with the initial user prompt 325 for parsing the prompt element database 355 at the content management server 315 to obtain relevant suggested prompt elements.

[0073] In some implementations, the keyword server 310 can be associated with a content management server 315. In some implementations, the keyword server 310 can be one or more keyword servers associated with one or more different content providers. In some implementations, the content providers can include creative asset providers, advertisers, publishers, artists, or other content providers.

[0074] In response to the keyword fingerprint retrieval component 320 obtaining the keyword fingerprint data 340, the prompt element retrieval component 345 can initiate retrieval of prompt elements from the content management server 315. The prompt element retrieval component 345 can transmit the keyword fingerprint data 350 to the content management server 315. The content management server 315 can obtain the keyword fingerprint data 350 and retrieve one or more related prompt elements and associated prompt element data 360 from the prompt element database 355. An example prompt element database is depicted in diagram 400.

[0075] Go to Figure 4, which depicts an example prompt element database 400 (e.g., prompt element database 355). Prompt element database 400 may include prompt element data for multiple content providers. For example, prompt element data may include keyword fingerprint data (e.g., keyword fingerprints 410, 425, and 440) and prompt element data (e.g., prompt elements 415, 430, 445). The keyword fingerprint data and prompt element data may be associated with content provider A 405, content provider B 420, or content provider C 435. Due to the structure of prompt element database 400, content management server 315 can easily locate prompt elements 415, 430, 445 associated with obtained keyword fingerprint data 350.

[0076] For example, a keyword fingerprint may include a keyword such as "pets". Hint elements associated with the keyword fingerprint pets may include "dog food from content provider A" or "cat carrier from content provider A". Additional examples may include the keyword "chair" and a hint element for a specific model of a chair, sofa, or other item associated with a content provider and associated with the keyword "chair". These examples are provided for illustrative purposes only and are not intended to be limiting.

[0077] In some examples, prompt elements may be provided to the content management server from one or more content provider servers. Prompt elements stored in prompt element database 400 may be matched with initial user prompts based on determined keyword fingerprints. In some examples, initial user prompts may be stored in the prompt element database in association with related keyword fingerprints. Thus, the prompt element database may be continuously updated.

[0078] Back to Figure 3 , prompt element data 360 may be transmitted from the content management server 315 to the prompt element retrieval component 345. In response to obtaining the prompt element data 360, the prompt element retrieval component 345 may transmit the prompt element data 360 to the scoring component 365. For example, given the obtained initial user prompt 325 and the context data 327, one or more keyword fingerprints may be generated. The prompt element retrieval component 345 may use the keyword fingerprints to retrieve one or more related prompt elements. Thus, the input received by the content management server 315 may include one or more keyword fingerprints and associated keyword fingerprint data 350, and the output generated by the content management server 315 may include one or more prompt elements and associated prompt element data 360.

[0079] Scoring component 365 can obtain one or more retrieved prompt elements and associated prompt element data 360, and in response, determine a score for each respective prompt element and rank the respective prompt elements. The score for each respective prompt element can be determined in a variety of ways.

[0080] In some implementations, the score may be determined as a function of multiple quality signals. For example, the function may be:

[0081] prompt_element_score = f(CPB,PCTR,PR)

[0082] Where CPB is the content provider bid, PCTR is the predicted click-through rate, and PR is the predicted relevance.

[0083] One or more scored prompt elements may be ranked based on the corresponding scores. Scoring component 365 may select the top N prompt elements to provide for transmission to the client device for display as selectable prompt element suggestions via a user interface.

[0084] The performance of one or more prompt elements can be determined by the computing system and used to determine the predicted click-through rate, predicted relevance or other metrics. For example, in some implementations, the cost of each click can be determined (for example, using a second-highest price auction). The computing system can obtain data indicating which suggested prompt elements of the N suggested prompt elements presented at the client device are selected. The data associated with the selection (for example, click) can be reported back to one or more relevant content providers via the content management server 315. The metrics associated with the suggested prompt element performance can be used to change prompt element retrieval, matching, scoring or ranking in the future. In some implementations, data associated with multiple prompt element suggestions can be aggregated and analyzed over time. One or more reports or information graphics can be provided to be displayed via the content provider device. In some instances, the content management server can use reports or underlying performance data to automatically update the bidding strategy associated with the content activities of the corresponding content provider. For example, the bid associated with a specific prompt element (for example, and associated content items or advertisements) can be updated.

[0085] In some instances, prompt elements may be generated in real time or near real time. Figure 5 Depicted is an example data flow 500 for generating one or more prompt elements.For example, the prompt elements may be generated by a content provider computing system or a content management computing system.

[0086] The prompt element suggestion component 505 can transmit the initial prompt data and the context data 515 to the prompt element generation component 520. The prompt element generation component 520 can be associated with a content provider or a content management service. In some examples, the prompt element generation component 520 can include one or more generation models 525. The one or more generation models 525 can obtain the initial prompt data and the context data 515 as input and generate one or more prompt elements and associated generated prompt element data 535 as output.

[0087] For example, the prompt element suggestion component 505 can transmit data including the initial prompt data and the context data 515 to the prompt element generation component 520. The prompt element generation component 520 can include one or more generation models 525. The one or more generation models 525 can process the initial prompt data and the context data 515 to determine the intent associated with the initial prompt data and the context data. The one or more generation models can process the input data to generate one or more prompt elements and the generated prompt element data 535 as output.

[0088] In some implementations, the prompt element generation component 520 can be communicatively coupled to a content provider database 530. The content provider database 530 can store data associated with a content provider. For example, the data can include data associated with one or more existing prompt elements (e.g., words, phrases, metadata known to produce a specific output image when processed by the image generation model, embedded data, links, or other associated data), one or more existing content elements, one or more existing content item activities, or other data associated with a content provider. In some implementations, the prompt element generation component 520 can use data from the content provider database 530 as input to the generation model 525. The generation model 525 can generate one or more prompt elements and associated prompt element data based on the initial prompt data and the context data 515 and the data from the content provider database 530.

[0089] The generated prompt element data 535 can be transmitted from the prompt element generation component 520 and obtained by the prompt element suggestion component 505 .

[0090] Figure 6 A block diagram of an example feedback loop 600 for training an image generation model 670 or a model 650 of a prompt element suggestion component is depicted. For purposes of illustration, Figure 6The term "training" is used to describe the feedback loop 600. Note that the term "training" is intended to include techniques for customizing the corresponding model, including but not limited to fine-tuning or prompt tuning. In some implementations, the models described herein may include a large language model (LLM) that is initially trained on a large amount of data. The present disclosure provides a method for fine-tuning or prompt tuning a corresponding model based on obtained user input and other relevant data to allow the model to better perform its corresponding task (e.g., suggesting prompt elements and generating images).

[0091] Feedback loop 600 may include client device 602, image generation component 605, prompt element suggestion component 610, and model trainer 615. As described herein, client device 602 may include image generation application 620, user interface 625, and user input component 603. Client device 602 may obtain input including initial prompt data 635 via user input component 630. In some implementations, initial prompt data 635 may be obtained by image generation component 605. In some implementations, initial prompt data 635 may be directly transmitted to prompt element suggestion component 610 along with context data. Image generation component may obtain initial prompt data 635 via application programming interface (API). For example, API may obtain initial prompt data 635 and generate a data structure that may include initial prompt data and context data. Prompt element suggestion component 610 may obtain a data structure including initial prompt data and context data 645 and generate output. The output may include prompt element data 655.

[0092] For example, the prompt element suggestion component 610 may include one or more models 650. The one or more models 650 may be machine learning models. The one or more models 650 may be configured to obtain or generate one or more suggested prompt elements (e.g., by the methods described herein). Using the models 650, the prompt element suggestion component 610 may generate and transmit prompt element data 655. The image generation component 605 may obtain the prompt element data 655 and transmit the selected prompt element data 660 to the client device 602. In some instances, the prompt element suggestion component 610 may transmit the prompt element data 655 directly to the client device 602.

[0093] The selected prompt element data 660 may include instructions that, when executed by one or more processors of the client device 602, cause the client device 602 to automatically update the user interface 625 to display the selected prompt element via the user interface 625. For example, the image generation application 620 may include an interface comprising an input bar. When obtaining user input via the user input component 630, the system selects the prompt elements to be provided to the user interface 625 to display the suggestions. For example, the suggestions may be in the form of a drop-down menu, an optional element (e.g., a button), a link, or other display formats. The computing system may obtain data indicating that the user selects or rejects one or more suggested prompt elements. The image generation application may facilitate the transmission of data by transmitting the updated prompt data 665 to the image generation model 670 of the image generation component 605, including the selection or rejection of the suggested prompt elements.

[0094] The image generation component 605 can obtain the updated prompt data 665 and generate output data 675 using the image generation model 670. The image generation model 670 can include one or more language models configured to generate an output image based on the ingestion of the prompt as an input to the model. The output data 675 can include image data 677 and embedded information data 679. For example, the output image can include a generated image that includes an item associated with a content provider (such as a certain brand of shoes on a runner's feet, a specific style of chair). In some implementations, the embedded information 679 can include data for generating an optional component within the image generation application 620, which is displayed via the user interface 625 so that the item associated with the content provider (e.g., a certain brand of shoes, a specific style of chair) can be presented as an interactive user interface element. In response to obtaining data including a user's selection of an interactive user interface element, the user interface 625 of the client device 602 can be automatically updated to provide a website associated with the link or perform some other action based on the embedded information associated with the visual element.

[0095] For example, continuing with the shoes example, the user can select the shoes in the image of the runner who completed the marathon, and in response to clicking on the shoes, a web page where the shoes can be purchased can be displayed. In addition or alternatively, a coupon or some other offer can be provided to be displayed via the user interface 625.

[0096] In some implementations, the user may provide updated prompt data indicating a refined prompt. In some instances, the image generation application 620 may obtain user input via a survey presented to the user. The computing system (e.g., a content management server system) may store data related to the performance of previously suggested prompt elements or output image data. Performance data (e.g., performance data 686) or other data related to prompt element and image performance may be stored in the database 690.

[0097] Model trainer 615 can use training data 680 to train image generation model 670 or model 650. In some examples, model trainer 615 is part of feedback loop 600. Feedback loop 600 can include obtaining data from client device 602 for use as training data 680. Additionally or alternatively, training data 680 can include data obtained from one or more databases 690.

[0098] The training data may include, but is not limited to, survey response data 682, refined prompt data 684, performance data 686, or any other relevant data. Figure 8 Model training is further described in .

[0099] Model 650 may include a prompt element generation model configured to generate one or more prompt elements. Prompt element suggestion component 610 may access content element performance metrics to update model 650 to generate prompt elements with better performance. For example, content element performance metrics may include click counts, interaction rates, impression counts, or other performance signals. By continuously updating model 650 based on content element performance metrics, model 650 may continuously improve functions such as prompt element generation, ranking, scoring, suggestions, or any other prompt element related functions.

[0100] In some examples, the model trainer 615 can be in a feedback loop 600 with the image generation model 670. For example, the system can determine that the user has selected a suggested prompt element to use as an input to the image generation model 670. The image generation model 670 can generate an output image 677. The output image 677 may be satisfactory to the user or may be unsatisfactory to the user. User satisfaction can be determined in a variety of ways, including, for example, user surveys or quantitative methods.

[0101] Survey response data 682 can be obtained via providing a user survey. For example, in some instances, the user survey can be random or otherwise triggered. For example, the user survey can include one or more questions to determine the performance of the generated image. For example, the question can include whether the generated image is relevant to the user's needs, whether the image meets the user's standards, the quality of the generated image, a request to rank the image on a numerical scale, or other questions related to the qualitative nature of the generated image. In some instances, images that have been generated based on selected repetitive prompt elements (e.g., shoes, cat food, furniture, etc. of a specific brand) can be analyzed to determine the quality and metrics indicating that the image meets the brand guidelines. For example, there can be an automated process that can determine the quality of a portion of an image associated with a brand element and compare the portion of the image with a known satisfactory image associated with the brand. The model can be trained for specific brand-related images to improve the output of the image generation model. Additionally or alternatively, the system can determine the number of images to be passed to manual review based on determining the difference between a known satisfactory image and a generated image. Therefore, the image generation model can be constantly fine-tuned to produce improved images.

[0102] The refined prompt data 684 can be associated with a quantitative method for determining performance. For example, the quantitative method can determine the performance of the image generation model based on the frequency with which refined prompts are obtained via user input after viewing an image generated using the selected prompt element. For example, the model trainer 615 can infer that the user provided an updated prompt after viewing an image generated using the suggested prompt element, indicating that the generated image was not satisfactory to the user. Therefore, an increase in the number of subsequent refined prompts (e.g., and a larger amount of refined prompt data 684) can be associated with a need to improve the images generated using the suggested prompt element or to improve the suggested prompt element.

[0103] Figure 7 A flow chart of an example method 700 for generating a prompt element for use as input to a generative model is depicted. Figure 7 The steps are depicted in a specific order for the purpose of illustration and discussion, but the method of the present disclosure is not limited to the specific order or arrangement shown. The various steps of method 700 may be omitted, rearranged in various ways, combined or adjusted without departing from the scope of the present disclosure.

[0104] At (702), method 700 may include obtaining, by a prompt element suggestion model, user input data including initial prompt data from a client device. For example, a computing system (e.g., computing system 800) may obtain, by a prompt element suggestion model, user input data including initial prompt data from a client device. As described herein, the initial prompt data may include natural language input received via a user input component of the client device. The initial prompt data may be received via a user interface associated with an image generation application located on a user device.

[0105] In some examples, the prompt element suggestion model may be part of a prompt element suggestion component.For example, the method may additionally or alternatively include obtaining, by the prompt element suggestion component, from a client device a user input including initial prompt data.

[0106] For example, an image generation application may be an application in which a user provides input to an input field of a desired image to be created. For example, a user may provide input including "a cat sitting on a chair". This initial input is quite broad and may produce a variety of image outputs that may not be consistent with the user's ultimate intent. For example, one interpretation of the prompt is a life-size cat sitting on a life-size chair. Another interpretation of the prompt may be a life-size cat sitting on a human chair, or a cat-sized cat sitting on a cat-sized chair (or even a cat bed). The present disclosure provides a system for training a model to better predict the user's intent and recommend one or more additional prompt elements (e.g., words, phrases, etc.) to improve the input provided to the image generation model.

[0107] In some embodiments, method 700 may include transmitting initial prompt data and context data to a prompt element generation component. For example, a computing system may transmit initial prompt data and context data to a prompt element generation component. The prompt element generation component may include a prompt element generation model. The prompt element generation model may be a machine learning language model (e.g., LLM). As described herein, the context data may be data associated with a current user session. For example, a client device may be associated with a search session, or may have other context data associated with a session. For example, the context data may include a date or time, a location, or other information.

[0108] Continuing with the above example, a user session may indicate a search for a chair or other bed for your pet cat. The context may include a previous search session looking for various offers for cat-sized chairs, cat beds, or other options for cat lounging. The initial prompt may be "Cat sitting on a chair in my living room." Additionally, for example, the context data may include an uploaded picture of the user's living room.

[0109] In some embodiments, method 700 may include obtaining, by a prompt element suggestion component, one or more suggested prompt elements from a prompt element generation component. For example, the computing system may obtain, by the prompt element suggestion component, one or more suggested prompt elements from a prompt element generation component. As described herein, the prompt element generation component may include a language model to generate the suggested prompt elements. In some instances, the method may include a prompt element generation component that includes a machine learning language model configured to generate the one or more prompt elements.

[0110] In some implementations, generating the suggested prompt elements may include obtaining initial prompt data and context data. Generating the suggested prompt elements may include obtaining one or more content elements from a content provider database. Generating the suggested prompt elements may include generating the one or more prompt elements based on the initial prompt data, the context data, and the one or more content elements from a content provider inventory database.

[0111] For example, continuing with our cat sitting on a chair example, the initial user input may be "the cat sits on the furniture in my living room," and the context data may include an uploaded photo of a living room. The suggested prompt element component may obtain or generate multiple related prompt elements. For example, the system may perform image processing on the input to determine available open space. The size of the space may be used to help select the prompt element so that the generated image has a piece of furniture of the correct size.

[0112] In some examples, generating one or more suggested prompt elements may include determining a similarity between a first characteristic associated with a first content element from a content provider inventory database that is consistent with the obtained context data. Generating one or more suggested prompt elements may include generating a first prompt element based on the similarity. For example, generating the first prompt element may be based on the determined similarity and the first content element.

[0113] For example, the characteristics may be an association of "cat" with "pet" and an association of "furniture" with "cat bed," "scratching post tower," "cat-sized chair," or other similar characteristics. Suggested prompt elements may be selected or generated based on keywords associated with the initial input prompt.

[0114] One or more suggested prompt elements can be generated in real time or near real time. In addition or alternatively, one or more suggested prompt elements and associated data can be stored in a cache. One or more suggested prompt elements can be selected from the cache. For example, the method may include determining the distance between initial prompt data and data associated with one or more prompt elements and associated data stored in the cache. The method may include selecting the first prompt element and associated data stored in the cache based on the distance between the initial prompt data and data associated with a first prompt element in the one or more prompt elements. For example, the associated data may include features, associated keywords, tags, instructions that are not visible to the user but readable by the image generation model, or other data.

[0115] The prompt element suggestion component may include a language model. The prompt element generation component may include an artificial intelligence large language model. A large language model or LLM may include a deep learning model with a large number (e.g., thousands, millions, or billions) of parameters, and the deep learning model is trained for a large amount of text. LLM can be used to perform various natural language processing (NLP) tasks. NLP tasks may include, for example, generating text, summarizing text, converting text, generating images, text classification, answering questions, powering a chatbot or virtual assistant, etc.

[0116] The prompt element data may include tokens. For example, a token may include a smaller unit of text that can be processed by a model (e.g., a large language model, an artificial intelligence large language model). For example, a token may include a character, a word, a portion of a word, one or more symbols, or some other unit of character that can be processed as input or generated as output by a model.

[0117] At (704), method 700 may include selecting, by the prompt element suggestion model, one or more suggested prompt elements based at least in part on the initial prompt data. For example, the computing system may select, by the prompt element suggestion model, one or more suggested prompt elements based at least in part on the initial prompt data.

[0118] The selection of one or more suggested prompt elements by the prompt element suggestion model may be based on at least one of a quality associated with the suggested prompt elements, a relevance of the suggested prompt elements, or a ranking of the suggested prompt elements.

[0119] Selecting, by the prompt element suggestion model, one or more suggested prompt elements may include determining one or more keywords based on the obtained initial prompt data. Selecting, by the prompt element suggestion model, one or more suggested prompt elements may include transmitting a request for data including one or more prompt elements associated with the one or more keywords. Selecting, by the prompt element suggestion model, one or more suggested prompt elements may include obtaining one or more prompt elements associated with the one or more keywords.

[0120] In some implementations, the prompt element suggestion model can be a component of the image generation model. As described herein, the prompt element suggestion model can be part of the prompt element suggestion component. In some implementations, operations described as being performed by the prompt element suggestion model can be performed by the prompt element suggestion component. The image generation model can be a generative machine learning model.

[0121] The selection of one or more suggested prompt elements by the prompt element suggestion model may be performed at least in part based on a bidding process. For example, the bidding process may include a dynamic content selection process for determining a content item or, in this example, a prompt element that is suitable for the current context. For example, each generated prompt element may have associated data, which may include an expected associated output image, associated keywords, or other data indicating a subject or other data that may be used to determine the relevance of the prompt element to the query (e.g., initial prompt data) obtained by the prompt element suggestion component. For example, the quality scores of multiple candidate prompt elements may be determined based on a comparison of the initial prompt data with the candidate prompt element data to determine one or more prompt elements with the highest score. In some instances, the prompt element may include data including a word, multiple words, or tokens (e.g., objects, descriptions, etc.) to be displayed via a user interface. In some instances, the prompt element data may include data that enables the image generation model to generate a predetermined image based on the input when obtained by a client device (e.g., and an image generation application). For example, a prompt element that displays the following phrase "white waterproof running shoes with good support" via a user interface may be transmitted together with additional data that enables the shoes of Company A to be included in the generated image. In some implementations, the hint element can relate to historical information or other information that can be used for presentation as part of a generated output image.

[0122] For example, returning to the cat example, there may be multiple related prompt elements available for recommendation. The available prompt elements may include a first prompt element that is a "cat scratching post tower" associated with a first content provider and a "cat bed" associated with a second content provider. The prompt element suggestion component may score and rank the available prompt elements and the grouping data associated therewith for transmission to the client device and display to the user.

[0123] At (706), method 700 may include transmitting, by the prompt element suggestion model, one or more suggested prompt elements to be presented to the client device for display as selectable user interface elements via the user interface. For example, the computing system may transmit, by the prompt element suggestion model, one or more suggested prompt elements to be presented to the client device for display as selectable user interface elements via the user interface. As described herein, the suggested prompt elements may be presented in a variety of formats. For example, the formats may include selectable items in a drop-down menu, selectable buttons, pre-populated input fields, or any other formats.

[0124] At (708), method 700 may include obtaining, by the prompt element suggestion model, second user input data from the client device, the second user input data including data indicating a selection of one or more suggested prompt elements. For example, the computing system may obtain, by the prompt element suggestion model, second user input data from the client device, the second user input data including data indicating a selection of one or more suggested prompt elements.

[0125] Additionally or alternatively, method 700 may include obtaining, by the prompt element suggestion model, second user input data from the client device. The second user input data may include at least one of: (i) data indicating a selection of one or more suggested prompt elements or (ii) updated prompt data. The updated prompt data may include, for example, new input data including an updated or rephrased input prompt, may include an updated input prompt related to an entirely new topic, or may include some other update to the input prompt.

[0126] In some implementations, method 700 may include updating the prompt element suggestion model based on the second user input data.For example, the method may include obtaining user input data including a selection of one of the recommended prompt elements.

[0127] At (710), method 700 may include, in response to obtaining second user input data, providing the second user input to an image generation model to generate an output image including a visual representation associated with one or more suggested prompt elements. For example, the computing system may, in response to obtaining the second user input data, provide the second user input to an image generation model to generate an output image including a visual representation associated with one or more suggested prompt elements. As described herein, the output generated by the image generation model may include image data, audio data, audiovisual data, or any other type of output that can be displayed via a client device.

[0128] In some implementations, method 700 includes an image generation model configured to perform an operation. The operation may include generating an output image based on the obtained second user input data. The operation may include performing a validation operation based on the output image. The operation may include updating the image generation model based on the validation operation performed.

[0129] The image generation model may generate an interactive image including embedded information.The embedded information may be information that, when accessed, causes the user interface to automatically update with a browser associated with a prompt element for selection serving as a prompt for the image generation model.

[0130] For example, the client device may obtain data indicating that the user selected a "cat bed" recommendation prompt element. The "cat bed" recommendation prompt element may have additional associated embedded data such that when an image including "a cat on a cat bed in my living room" is generated, an image of a cat sitting on the associated cat bed in the provided living room is generated. The generated image may include embedded content such that when the user selects the image or selects a portion of the image (e.g., the cat bed), the user interface is updated and redirected to a website associated with the cat bed.

[0131] Method 700 may include training or tuning an image generation model. For example, the image generation model may be a machine learning model. Method 700 may include tuning the image generation model. Tuning the image generation model may include providing a training prompt as an input to the image generation model. Tuning the image generation model may include obtaining an output including a generated image from the image generation model. Tuning the image generation model may include comparing the output including the generated image with an approved image associated with the training prompt. Tuning the image generation model may include tuning the image generation model based on comparing the output including the generated image with an approved image associated with the training prompt. For example, various weights and parameters associated with the model may be adjusted to improve the output of the model. In some instances, the image generation model may be trained or tuned to predict a known portion of the output image based on data associated with the second user prompt data. For example, the second user prompt data may include a selected prompt element. Selected prompt elements can have associated data that provides additional input information to the model (e.g., without being displayed to the user via a user interface), but is ingested by the image generation model to produce a known portion of the output image (e.g., a specific shoe, a specific physical object presented in the image, etc.).

[0132] In some instances, the prompt element suggestion component or prompt element suggestion model can be a component of an image generation model. In some instances, the prompt element suggestion component can be communicatively coupled to the image generation model. The image generation model can be a generative machine learning model. In some instances, the image generation model can be a set of models with different functions that work together to process the obtained input prompt and generate an image to be provided for display.

[0133] Although Figure 7 An implementation utilizing an image generation model is described, but the described method can be applied to any generation model that is capable of obtaining input prompt data and generating output based on the obtained input prompt data.

[0134] Figure 8 A block diagram of an example computing system 800 that performs prompt generation and recommendation for input into a generative model to improve the output of the generative model is shown according to an example embodiment of the present disclosure. The computing system 800 includes a client computing system 802, a server computing system 804, a training computing system 806, a content provider computing system 808, and an image generation computing system 810 that are communicatively coupled via a network 805.

[0135] The client computing system 802 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop computer), a mobile computing device (e.g., a smartphone or tablet computer), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0136] The client computing system 802 includes one or more processors 812 and a memory 814. The one or more processors 812 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operatively connected. The memory 814 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc. and combinations thereof. The memory 814 can store data 816 and instructions 818 that are executed by the processor 812 to cause the client computing system 802 to perform operations.

[0137] In some implementations, the client computing system 802 may include an image generation application 820. The image generation application 820 may include an application downloaded to a user device. Additionally or alternatively, the image generation application 820 may include a web-based application. The image generation application 820 may communicate with an application programming interface (e.g., an image generation API 890) to interface with the image generation computing system 810. For example, the API may facilitate interaction between the client computing system 802, the server computing system 804, and the image generation computing system 810.

[0138] In some implementations, the client computing system may include a user interface 822. The user interface 822 may include a graphical user interface, an audio user interface, a touch user interface, or any other user interface. The client computing system may include a user input component 824. The user input component 824 may be associated with the user interface 822 and may be able to obtain user input. For example, the user input may include touch, audio, or other user input. In some instances, the user input component 824 may be able to obtain user input and convert the user input into a computer-readable form.

[0139] The server computing system 804 includes one or more processors 826 and a memory 828. The one or more processors 826 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operatively connected. The memory 828 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc. and combinations thereof. The memory 828 can store data 830 and instructions 832 that are executed by the processor 826 to cause the server computing system 804 to perform operations.

[0140] In some implementations, the server computing system 804 includes or is otherwise implemented by one or more server computing devices. In the case where the server computing system 804 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0141] The server computing system 804 can be configured to obtain data from the client computing system 802 (e.g., via the image generation application 820). For example, the server computing system 804 can utilize the obtained user input data to update or train one or more models 834 (e.g., keyword generation model 836, prompt element suggestion model 838, prompt element generation model 840).

[0142] As described above, the server computing system 804 may store or otherwise include one or more models 834 (e.g., a keyword generation model 836, a prompt element suggestion model 838, a prompt element generation model 840). For example, the model 834 (e.g., the keyword generation model 836, the prompt element suggestion model 838, the prompt element generation model 840) may be or may otherwise include various statistical or machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a Transformer model). Model 834 may include a keyword generation model 836, a prompt element suggestion model 838, and a prompt element generation model 840. The keyword generation model 836 may generate keywords for the initial prompt obtained by the user to determine one or more related keywords to be used to retrieve or generate suggested prompt elements. The prompt element suggestion model 838 may retrieve, score, or rank one or more obtained prompt elements. The prompt element generation model 840 may be configured to generate one or more prompt elements. The example prompt element suggestion model 838 may include a prompt element generation model. Figure 5 and Figure 6 The prompt element generation model of the prompt element suggestion model 838 and the prompt element generation model 840 are discussed.

[0143] The server computing system 804 may include a database 842. The database 842 may store historical signal data. The historical signal data may include previously acquired signal data associated with previously rendered cue elements or generated images obtained from an image generation computing system, content provider computing system 808, or server computing system 804.

[0144] The content provider computing system 808 includes one or more processors 854 and a memory 856. The one or more processors 854 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operatively connected. The memory 856 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc. and combinations thereof. The memory 856 can store data 858 and instructions 860 that are executed by the processor 854 to cause the content provider computing system 808 to perform operations.

[0145] In some implementations, the content provider computing system 808 includes or is otherwise implemented by one or more server computing devices. In the case where the content provider computing system 808 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0146] The content provider computing system 808 may include a database 862. The database 862 may store prompt element data 864 and content element data 866. The prompt element data 865 may include prompt elements, keyword values, or other characteristic data associated with prompt elements. The content element data 866 may include content elements, asset groups, or other content-related data.

[0147] Content provider computing system 808 can be communicatively connected to server computing system 804 via network 805. In some instances, content provider computing system 808 can be a first party computing system associated with server computing system 804. In some instances, content provider computing system 808 can be associated with a third party content provider (e.g., an advertiser). There can be more than one content provider computing system 808.

[0148] As described above, the content provider computing system 808 may store or otherwise include one or more models 868 (e.g., prompt element generation model 870). For example, model 868 (e.g., prompt element generation model 870) may be or may otherwise include various statistical or machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a Transformer model). Model 868 may include a prompt element generation model 870. The prompt element generation model 870 may be configured to generate one or more prompt elements. Reference Figure 5 and Figure 6 An example prompt element generation model 870 is discussed.

[0149] The image generation computing system 810 includes one or more processors 872 and a memory 874. The one or more processors 872 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be a processor or multiple processors operatively connected. The memory 874 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc. and combinations thereof. The memory 874 can store data 876 and instructions 878 executed by the processor 872 to cause the image generation computing system 810 to perform operations.

[0150] In some implementations, the image generation computing system 810 includes or is otherwise implemented by one or more server computing devices. In the case where the image generation computing system 810 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0151] As described above, the image generation computing system 810 may store or otherwise include one or more models 880 (e.g., image generation model 888). For example, model 880 (e.g., image generation model 888) may be or may otherwise include various statistical or machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Some example machine learning models may utilize attention mechanisms, such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a Transformer model). Model 880 may include an image generation model 888. The image generation model 888 may be configured to generate one or more images or other generated outputs. Reference Figure 5 and Figure 6 Discuss example image generation model 888.

[0152] Client computing system 802 or server computing system 804 may train model 834, 868, or 880 via interaction with a training computing system 806 communicatively coupled via network 805. Training computing system 806 may be separate from server computing system 804 or may be part of server computing system 804.

[0153] The training computing system 806 includes one or more processors 844 and a memory 846. The one or more processors 844 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operatively connected. The memory 846 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc. and combinations thereof. The memory 846 can store data 847 and instructions 848 that are executed by the processor 844 to cause the training computing system 806 to perform operations. In some implementations, the training computing system 806 includes one or more server computing devices or is otherwise implemented by one or more server computing devices.

[0154] The training computing system 806 may include a model trainer 850 that trains a machine learning model 840, 868, or 880 stored at the client computing system 802, the server computing system 804, the content provider computing system 808, or the image generation computing system 810 using various training or learning techniques such as, for example, error back propagation. For example, a loss function may be back propagated through the model to update one or more parameters of the model (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters in multiple training iterations.

[0155] In some implementations, performing error backpropagation may include performing truncated backpropagation through time. The model trainer 850 may perform a variety of generalization techniques (eg, weight decay, backoff, etc.) to improve the generalization capabilities of the model being trained.

[0156] Specifically, the model trainer 850 can train the models 840, 868, 880 based on a set of training data 852. The training data 852 can include, for example, historical signal data, publisher-rendered native content item data, user input data, conversion data, user device location data, click data, or any other relevant data (e.g., data stored in database 842, data stored in database 862, etc.).

[0157] In some implementations, if the user has consented, the training examples may be provided by the client computing system 802. Thus, in such implementations, the models 834, 861, 880 provided to the client computing system 802 may be trained by the training computing system 806 based on user-specific data received from the client computing system 802. In some instances, this process may be referred to as personalizing the model.

[0158] The model trainer 850 includes computer logic for providing the desired functionality. The model trainer 850 can be implemented in hardware, firmware, or software that controls a general purpose processor. For example, in some implementations, the model trainer 850 includes a program file stored on a storage device, loaded into a memory, and executed by one or more processors. In other implementations, the model trainer 850 includes one or more sets of computer executable instructions stored in a tangible computer readable storage medium such as RAM, a hard disk, or an optical or magnetic medium.

[0159] The network 805 can be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. In general, communications over the network 805 can be carried via any type of wired or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), or protection schemes (e.g., VPN, secure HTTP, SSL).

[0160] The machine learning models described in this specification can be used for a variety of tasks, applications, or use cases.

[0161] In some implementations, the input of the machine learning model of the present disclosure may be image data. The machine learning model may process the image data to generate an output. As an example, the machine learning model may process the image data to generate an image recognition output (e.g., recognition of image data, potential embedding of image data, encoded representation of image data, hashing of image data, etc.). As another example, the machine learning model may process the image data to generate an image segmentation output. As another example, the machine learning model may process the image data to generate an image classification output. As another example, the machine learning model may process the image data to generate an image data modification output (e.g., a change in image data, etc.). As another example, the machine learning model may process the image data to generate an encoded image data output (e.g., an encoded and / or compressed representation of image data, etc.). As another example, the machine learning model may process the image data to generate an amplified image data output. As another example, the machine learning model may process the image data to generate a predicted output.

[0162] In some implementations, the input of the machine learning model of the present disclosure may be text or natural language data. The machine learning model may process the text or natural language data to generate an output. As an example, the machine learning model may process the natural language data to generate a language encoding output. As another example, the machine learning model may process the text or natural language data to generate a potential text embedding output. As another example, the machine learning model may process the text or natural language data to generate a conversion output. As another example, the machine learning model may process the text or natural language data to generate a classification output. As another example, the machine learning model may process the text or natural language data to generate a text segmentation output. As another example, the machine learning model may process the text or natural language data to generate a semantic intent output. As another example, the machine learning model may process the text or natural language data to generate an amplified text or natural language output (e.g., text or natural language data with a higher quality than the input text or natural language, etc.). As another example, the machine learning model may process the text or natural language data to generate a prediction output.

[0163] In some implementations, the input of the machine learning model of the present disclosure may be speech data. The machine learning model may process speech data to generate an output. As an example, the machine learning model may process speech data to generate a speech recognition output. As another example, the machine learning model may process speech data to generate a speech conversion output. As another example, the machine learning model may process speech data to generate a potential embedding output. As another example, the machine learning model may process speech data to generate an encoded speech output (e.g., an encoded and / or compressed representation of speech data, etc.). As another example, the machine learning model may process speech data to generate an amplified speech output (e.g., speech data of higher quality than the input speech data, etc.). As another example, the machine learning model may process speech data to generate a text representation output (e.g., a text representation of the input speech data, etc.). As another example, the machine learning model may process speech data to generate a predicted output.

[0164] In some implementations, the input of the machine learning model of the present disclosure may be latent coded data (e.g., a latent space representation of an input, etc.). The machine learning model may process the latent coded data to generate an output. As an example, the machine learning model may process the latent coded data to generate a recognition output. As another example, the machine learning model may process the latent coded data to generate a reconstruction output. As another example, the machine learning model may process the latent coded data to generate a search output. As another example, the machine learning model may process the latent coded data to generate a re-clustering output. As another example, the machine learning model may process the latent coded data to generate a prediction output.

[0165] In some implementations, the input to the machine learning model of the present disclosure may be statistical data. The statistical data may be, represent, or otherwise include data calculated from some other data source. The machine learning model may process the statistical data to generate an output. As an example, the machine learning model may process the statistical data to generate a recognition output. As another example, the machine learning model may process the statistical data to generate a prediction output. As another example, the machine learning model may process the statistical data to generate a classification output. As another example, the machine learning model may process the statistical data to generate a segmentation output. As another example, the machine learning model may process the statistical data to generate a visualization output. As another example, the machine learning model may process the statistical data to generate a diagnostic output.

[0166] In some cases, the machine learning model can be configured to perform a task including encoding input data to achieve reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task can be an audio compression task. The input can include audio data, and the output can include compressed audio data. In another example, the input includes visual data (e.g., one or more images or videos), the output includes compressed visual data, and the task is a visual data compression task. In another example, the task can include generating an embedding for the input data (e.g., input audio or visual data).

[0167] In some cases, the input includes visual data, and the task is a computer vision task. In some cases, the input includes pixel data of one or more images, and the task is an image processing task. For example, the image processing task may be image classification, where the output is a set of scores, each score corresponding to a different object class and representing the possibility that one or more images depict an object belonging to the object class. The image processing task may be object detection, where the image processing output identifies one or more regions in one or more images, and for each region, identifies the possibility that the region depicts an object of interest. As another example, the image processing task may be image segmentation, where the image processing output defines the corresponding possibility of each category in a set of predetermined categories for each pixel in one or more images. For example, the set of categories may be foreground and background. As another example, the set of categories may be object classes. As another example, the image processing task may be depth estimation, where the image processing output defines a corresponding depth value for each pixel in one or more images. As another example, the image processing task may be motion estimation, where the network input includes multiple images, and the image processing output defines the motion of the scene depicted at the pixel between images in the network input for each pixel of one of the input images.

[0168] In some cases, the input includes audio data representing a spoken utterance, and the task is a speech recognition task. The output may include a text output mapped to the spoken utterance. In some cases, the task includes encrypting or decrypting the input data. In some cases, the task includes a microprocessor performance task, such as branch prediction or memory address translation.

[0169] Figure 8 An example computing system that can be used to implement the present disclosure is shown. Other computing systems can also be used. For example, in some implementations, the client computing system 802 can include a model trainer 850 and training data 852. In such implementations, the models 834, 868, 880 can be trained and used locally at the client computing system 802. In some such implementations, the client computing system 802 can implement the model trainer 850 to personalize the models 834, 868, 880 based on user-specific data.

[0170] The techniques discussed herein refer to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. The inherent flexibility of computer-based systems allows for a variety of possible configurations, combinations, and divisions of tasks and functions between components. For example, the processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0171] Although the subject matter has been described in detail with respect to various specific example embodiments of the subject matter, each example is provided by way of explanation, rather than limitation of the present disclosure. After obtaining an understanding of the foregoing, those skilled in the art can easily produce changes, modifications, and equivalents to such embodiments. Therefore, the present disclosure does not exclude such modifications, changes, and / or additions to the subject matter that will be readily understood by those of ordinary skill in the art. For example, a feature shown or described as part of one embodiment can be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover such changes, changes, and equivalents.

[0172] The steps depicted or described are merely illustrative and may be omitted, combined, or performed in an order different from that depicted or described; the numbering of the depicted steps is merely for ease of reference and does not imply that any particular order is necessary or preferred.

[0173] The functions or steps described herein may be embodied in computer-usable data or computer-executable instructions executed by one or more computers or other devices to perform one or more functions described herein. Typically, such data or instructions include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific data types when executed by one or more processors in a computer or other data processing device. Computer-executable instructions may be stored in computer-readable media such as hard disks, optical disks, removable storage media, solid-state memory, read-only memory (ROM), random access memory (RAM), etc. It should be understood that the functions of such instructions may be combined or distributed as needed. In addition, the functions may be embodied in whole or in part in firmware or hardware equivalents such as integrated circuits, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. Specific data structures may be used to more effectively implement one or more aspects of the present disclosure, and such data structures are contemplated to be within the scope of computer-executable instructions and computer-usable data described herein.

[0174] Although not required, it will be appreciated by those of ordinary skill in the art that the various aspects described herein may be embodied as methods, systems, devices, or one or more computer-readable media storing computer-executable instructions. Thus, the various aspects may take the form of all-hardware embodiments, all-software embodiments, all-firmware embodiments, or embodiments combining software, hardware, or firmware aspects in any combination.

[0175] As described herein, various methods and actions may operate across one or more computing devices or networks. The functions described may be distributed in any manner, or may be located in a single computing device (eg, server, client computer, user device, etc.).

[0176] Various aspects of the present disclosure have been described with respect to illustrative embodiments thereof. Numerous other embodiments, modifications, and variations within the scope and spirit of the appended claims may occur to one of ordinary skill in the art upon careful reading of this disclosure. For example, one of ordinary skill in the art may appreciate that the steps depicted or described may be performed in an order different from that described, or that one or more of the steps shown may be optional or combined. Any and all features in the following claims may be combined or rearranged in any possible manner.

[0177] Various aspects of the present disclosure have been described with respect to its illustrative embodiments. Those of ordinary skill in the art may think of numerous other embodiments, modifications and changes within the scope and spirit of the appended claims after carefully reading the present disclosure. Any and all features in the following claims may be combined or rearranged in any possible manner. Therefore, the scope of the present disclosure is by way of illustration rather than limitation, and the present disclosure does not exclude such modifications, changes and / or additions to the subject matter that will be easily understood by those of ordinary skill in the art. In addition, a list of example elements connected by conjunctions such as "and", "or", "but" is used herein to describe terms. It should be understood that such conjunctions are provided only for the purpose of explanation. For example, a list connected by a specific conjunction such as "or" may refer to "at least one" or "any combination" of the example elements listed therein, wherein "or" should be understood as "and / or" unless otherwise indicated. In addition, terms such as "based on" should be understood as "based at least in part on".

[0178] Although the subject matter has been described in detail with respect to various specific example embodiments of the subject matter, each example is provided by way of explanation, rather than limitation of the present disclosure. After obtaining an understanding of the foregoing, those skilled in the art can easily produce changes, modifications, or equivalents to such embodiments. Therefore, the present disclosure does not exclude such modifications, changes, and / or additions to the subject matter that would be readily understood by those of ordinary skill in the art. For example, a feature shown or described as part of one embodiment can be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover such changes, modifications, or equivalents.

Claims

1. A computer-implemented method comprising: obtaining, by the prompt element suggestion model, user input data including initial prompt data from a client device; selecting, by the prompt element suggestion model, one or more suggested prompt elements based at least in part on the initial prompt data; transmitting, by the prompt element suggestion model, to the client device the one or more suggested prompt elements to be presented for display as selectable user interface elements via a user interface; obtaining, by the prompt element suggestion model from the client device, second user input data including data indicative of a selection of the one or more suggested prompt elements; as well as In response to obtaining the second user input data, the second user input is provided to an image generation model to generate an output image including a visual representation associated with the one or more suggested prompt elements.

2. The computer-implemented method of claim 1 , comprising: The prompt element suggestion model is updated based on the second user input data.

3. A computer-implemented method as described in any preceding claim, wherein the prompt element suggestion model comprises a machine learning language model.

4. The computer-implemented method of any preceding claim, wherein the image generation model is configured to perform operations comprising: generating an output image based on the obtained second user input data; performing a verification operation based on the output image; and The image generation model is updated based on the performed validation operation.

5. The computer-implemented method of any preceding claim, comprising a prompt element generation component comprising a machine learning language model configured to generate one or more prompt elements.

6. The computer-implemented method of claim 5, wherein generating the suggested prompt element comprises: Obtaining the initial prompt data and context data; Obtain one or more content elements from a content provider inventory database; as well as One or more prompt elements are generated based on the initial prompt data, the context data, and the one or more content elements from the content provider inventory database.

7. The computer-implemented method of claim 5 or 6, wherein generating the one or more prompt elements comprises: determining a similarity between first characteristics associated with first content elements from the content provider inventory database consistent with the obtained context data; as well as A first prompt element is generated based on the determined similarity and the first content element.

8. A computer-implemented method as claimed in any preceding claim, wherein the machine learning model used to generate the output image comprises an image generation model, the method comprising: The image generation model is tuned in the following way: providing a training cue as input to the image generation model; obtaining an output including a generated image from the image generation model; comparing the output including the generated image to an approved image associated with the training cue; as well as The image generation model is tuned based on comparing the output including the generated image to the approved image associated with the training cue.

9. The computer-implemented method of any preceding claim, wherein the one or more suggested prompt elements are generated in real time.

10. The computer-implemented method of any preceding claim, wherein the one or more suggested hint elements and associated data are stored in a cache.

11. The computer-implemented method of claim 10, wherein the one or more suggested hint elements are selected from the cache by: determining a distance between the initial hint data and data associated with one or more hint elements and associated data stored in the cache; and The first hint element and associated data stored in the cache are selected based on a distance between the initial hint data and data associated with a first hint element of the one or more hint elements.

12. The computer-implemented method of claim 4, wherein the image generation model generates an interactive image including embedded information that, when accessed, causes the user interface to automatically update using a browser associated with a selected prompt element used as a prompt for the image generation model.

13. A computer-implemented method as described in any preceding claim, wherein the selection of the one or more suggested prompt elements by the prompt element suggestion model is based on at least one of a quality associated with the suggested prompt elements, a relevance of the suggested prompt elements, or a ranking of the suggested prompt elements.

14. The computer-implemented method of any preceding claim, wherein selecting, by the prompt element suggestion model, one or more suggested prompt elements comprises: determining one or more keywords based on the obtained initial prompt data; transmitting a request for data including one or more prompt elements associated with the one or more keywords; as well as The one or more prompt elements associated with the one or more keywords are obtained.

15. The computer-implemented method of any preceding claim, wherein the prompt element suggestion model is a component of an image generation model.

16. The computer-implemented method of claim 15, wherein the image generation model is a generative machine learning model.

17. A computer-implemented method as claimed in any preceding claim, wherein the prompt element data comprises a token.

18. The computer-implemented method of any preceding claim, wherein selecting, by the prompt element suggestion model, the one or more suggested prompt elements is performed based at least in part on a bidding process.

19. A computing system comprising: one or more processors; as well as One or more computer-readable media storing instructions executable to cause the one or more processors to perform operations including the method of any one of claims 1 to 18.

20. One or more computer-readable media storing instructions executable by one or more processors to perform operations comprising the method of any one of claims 1 to 18.

Images