Content creation based on text-to-image generation

Abstract

This disclosure describes techniques for generating content. Text may be received. The text is associated with at least one user-created video. At least one image may be generated based at least in part on the text using at least one machine learning model. A video may be generated based at least in part on the at least one image. The video may include content overlaid on the at least one image.

Description

[Technical field]

[0001] This application claims priority to U.S. Patent No. 17 / 715,645, filed April 7, 2022 (titled "Content Creation Based on Text-to-Image Generation"), the disclosure of which is incorporated herein by reference in its entirety. [Background technology]

[0002] Increasingly, communication is conducted using Internet-based tools. Internet-based tools can be any software or platform. Existing social media platforms allow users to communicate with each other by sharing information such as images and videos through static applications or web pages. As communication devices such as mobile phones become more and more sophisticated, people continue to seek new ways of entertainment, social networking and communication. [Brief description of the drawings]

[0003] The following detailed description can be better understood when read in conjunction with the accompanying drawings, in which: For purposes of illustration, there are shown in the accompanying drawings exemplary embodiments of various aspects of the disclosure; however, the invention is not limited to the specific methods and instrumentalities disclosed.

[0004] [Figure 1] FIG. 1 illustrates an exemplary system for distributing content.

[0005] [Diagram 2] FIG. 2 illustrates an exemplary user interface of a content delivery application in accordance with the present disclosure.

[0006] [Diagram 3] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0007] [Figure 4A] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0008] [Figure 4B] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0009] [Diagram 5] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0010] [Figure 6A] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0011] [Figure 6B] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0012] [Figure 7A] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0013] [Figure 7B] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0014] [Figure 8] FIG. 2 illustrates another exemplary user interface of a content delivery application in accordance with the present disclosure.

[0015] [Figure 9] FIG. 2 illustrates an exemplary content creation method executable by a user device in accordance with the present disclosure.

[0016] [Figure 10] FIG. 1 illustrates another exemplary content creation method executable by a user device in accordance with the present disclosure.

[0017] [Figure 11] FIG. 1 illustrates another exemplary content creation method executable by a user device in accordance with the present disclosure.

[0018] [Figure 12] FIG. 1 illustrates an example computing device that can be used to perform any of the methods disclosed herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] A user may want to generate content using one or more images. For example, a user may want to generate a video using one or more images as a background (i.e., backdrop). However, it may be time-consuming and / or difficult for a user to find or generate a suitable image for such a purpose. For example, a user may have to spend several minutes, or even hours, searching for a suitable image. If a user cannot find an existing suitable image, the user may have to spend additional time creating a suitable image from scratch. Some existing text-to-image applications allow a user to input a text prompt and receive a generated image related to the input text prompt. However, such existing applications lack user interaction. As a result, such existing applications do not encourage creativity among their users, and users may quickly become bored with the existing applications. Therefore, there is a need for improved techniques for content creation using text-to-image generation.

[0020] Described herein are improved techniques for content creation using text-to-image generation. Such techniques are utilized to generate images from text. The images are blended with other content (i.e., live camera feeds, on-screen text graphics, etc.) to create videos. The improved techniques for content creation using text-to-image generation described herein utilize text-to-image generation capabilities in combination with more diverse user case scenarios. For example, the improved techniques for content creation using text-to-image generation described herein do not only respond to one text prompt, but also respond to a sequence of sentences to generate multiple images that form a storytelling video. Thus, the improved techniques for content creation using text-to-image generation described herein utilize text-to-image generation capabilities to provide a more interactive user experience.

[0021] The improved techniques for content creation using text-to-image generation described herein may be utilized by a system such as system 100 shown in FIG 1. FIG 1 illustrates an exemplary system 100 for delivering content. System 100 may include a cloud network 102 and a number of client devices 104a-d. Cloud network 102 and the number of client devices 104a-d may communicate with each other via one or more networks 120.

[0022] Cloud network 102 may be located in a data center, such as a single building, or may be distributed across different geographic locations (e.g., several buildings). Cloud network 102 may provide services through one or more networks 120. Network 120 may include various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and / or similar devices. Network 120 may include physical links, such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, etc. Network 120 may include wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0023] The cloud network 102 may include multiple computing nodes 118 hosting various services. In one embodiment, the nodes 118 host a content service 112. The content service 112 may include a content streaming service, such as an Internet Protocol video streaming service. The content service 112 may be configured to deliver the content 116 via various transmission technologies. The content service 112 is configured to provide the content 116, such as video, audio, text data, combinations thereof, etc. The content 116 may include content streams (e.g., video streams, audio streams, information streams), content files (e.g., video files, audio files, text files), and / or other data. The content 116 may be stored in the database 114. For example, the content service 112 may include a video sharing service, a video hosting platform, a content distribution platform, a collaborative gaming platform, etc.

[0024] In one embodiment, the content 116 delivered or provided by the content service 112 includes short videos. A short video may have a duration up to a predetermined time limit, such as one minute, five minutes, or other predetermined minutes. By way of example and not limitation, a short video may include at least one and no more than four 15-second segments linked together. Short video durations can provide viewers with rapid, continuous entertainment, allowing users to watch a large amount of video within a short time frame. Such rapid, continuous entertainment can be popular on social media platforms.

[0025] A short video may include a pre-recorded audio overlay, such as music or sounds from a television show or movie. When a short video includes a pre-recorded audio overlay, the short video may feature one or more people lip-syncing, dancing, or otherwise moving their bodies along with the pre-recorded audio. For example, a short video may feature a "dance challenge" that an individual completes to a hit song. Or, a short video may feature two people participating in a lip-syncing or dancing duet. As another example, a short video may feature an individual completing a challenge that requires them to move their body to correspond to the pre-recorded audio overlay, e.g., to the beat or rhythm of a pre-recorded song that is featured by the pre-recorded audio overlay. Other short videos may not include a pre-recorded audio override. For example, these short videos may feature individuals playing sports, playing pranks, or giving advice such as beauty and fashion advice, cooking tips, home decor tips, etc.

[0026] In one embodiment, the content 116 may be output to different client devices 104 over the network 120. The content 116 may be streamed to the client devices 104. The content stream may be a stream of short videos received from the content service 112. Multiple client devices 104 may be configured to access the content 116 from the content service 112. In one embodiment, the client devices 104 may include a content application 106. The content application 106 outputs (e.g., displays, renders, presents) the content 116 to a user associated with the client device 104. The content may include video, audio, comments, text data, etc.

[0027] The client devices 104 may include any type of computing device, such as a mobile device, a tablet device, a laptop computer, a desktop computer, a smart television or other smart device (e.g., a smart watch, a smart speaker, smart glasses, a smart helmet), a gaming device, a set-top box, a digital streaming device, a robot, etc. The client devices 104 may be associated with one or more users. A single user may access the cloud network 102 using one or more of the client devices 104. The client devices 104 may travel to different locations and access the cloud network 102 using different networks.

[0028] The content service 112 may be configured to receive input from a user. The user may be registered as a user of the content service 112 or may be a user of a content application 106 running on a client device 104. The user input may include a short video created by the user, a user comment associated with the short video, or a "like" associated with the short video. The user input may include a connection request and user input data, such as text data, digital image data, or user content. The connection request may include a request from a client device 104a-d to connect to the content service 112. The user input data may include information, such as a video and / or user comments, that a user connected to the content service 112 desires to share with other connected users of the content service 112.

[0029] The content service 112 may receive different types of input from users using different types of client devices 104. For example, a user using a content application 106 on a first user device, such as a mobile phone or tablet, may be able to use the content application 106 to create and upload short videos. A user using the content application 106 on a different mobile phone or tablet may be able to view, comment on, and "like" the short videos or comments written by other users. In another example, a user using the content application 106 on a smart TV, laptop, desktop, or gaming device may not be able to use the content application 106 to create and upload short videos or comment on the short videos. Instead, a user using the content application 106 on a smart TV, laptop, desktop, or gaming device may only be able to use the content application 106 to view short videos, view comments left by other users, and "like" the short videos.

[0030] In one embodiment, a user can use a content application 106 on a client device 104 to create and upload a short video to the cloud network 102. The client device 104 can access an interface 108 of the content application 106. The interface 108 may include input elements. For example, the input elements may be configured to allow a user to create a short video. To create the short video, the user may give the content application 106 permission to access an image capture device, such as a camera or a microphone, of the client device 104. Using the content application 106, the user may select the duration of the short video or set the speed of the short video, for example, "slow motion" or "speed up."

[0031] A user can edit a short video using the content application 106. A user can add one or more text, filters, sounds, or effects, such as beauty effects, to the short video. To add a pre-recorded audio overlay to the short video, a user can select a song or sound clip from a sound library of the content application 106. The sound library may include different songs, sound effects, or audio clips from movies, albums, and television shows. In addition to or instead of adding a pre-recorded audio overlay to the short video, a user can add narration to the short video using the content application 106. The narration may be sound recorded by the user using a microphone of the client device 104. A user can add a text overlay to the short video and can specify when they want the text overlay to appear in the short video using the content application 106. A user can assign subtitles, location tags, and one or more hashtags to the short video to indicate the subject of the short video. The content application 106 may prompt the user to select a frame of the short video to use as a "cover image" for the short video.

[0032] After a user creates a short video, the user can use the content application 106 to upload the short video to the cloud network 102 and / or store the short video locally on the user device 104. When a user uploads a short video to the cloud network 102, the user may select whether the short video is viewable by all other users of the content application 106 or by only a subset of users of the content application 106. The content service 112 may store the uploaded short video and any metadata associated with the short video in one or more databases 114.

[0033] In one embodiment, a user can provide input on a short video using a content application 106 on a client device 104. The client device 104 may access an interface 108 of the content application 106 that allows the user to provide input associated with the short video. The interface 106 may include an input element. For example, the input element may be configured to receive input from the user, such as a comment or a "like" to be associated with a particular short video. If the input is a comment, the content application 106 may allow the user to set an emoji to be associated with their input. The content application 106 may determine time information about the input, such as when the user wrote the comment. The content application 106 may transmit the input and associated metadata to the cloud network 102. For example, the content application 106 may transmit the comment, an identifier of the user who wrote the comment, and time information about the comment to the cloud network 102. The content service 112 may store the input and associated metadata in a database 114.

[0034] The content service 112 may be configured to output the uploaded short videos and user inputs to other users. A user may be registered as a user of the content service 112 and view short videos created by other users. The user may be a user of a content application 106 running on a client device 104. The content application 106 may output (display, render, present) the short videos and user comments to a user associated with the client device 104. The client device 104 may access an interface 108 of the content application 106. The interface 108 may include an output element. The output element may be configured to display information about different short videos so that a user can select and view the short videos. For example, the output element may be configured to display multiple cover images, subtitles, or hashtags associated with the short videos. The output element may also be configured to arrange the short videos according to a category associated with each short video.

[0035] In one embodiment, user comments associated with a short video may be output to other users viewing the same short video. For example, all users accessing the short video may see the comments associated with the short video. The content service 112 may output the short video and the associated comments simultaneously. The comments may be output by the content service 112 in real-time or near real-time. The content application 106 may display the short video and the comments on the client device 104 in various ways. For example, the comments may be displayed in an overlay on top of the content or next to the content. As another example, a user who wants to see other users' comments associated with the short video may need to select a button to view the comments. The comments may be animated when displayed. For example, the comments may be scrolled across the short video or across an overlay.

[0036] The computing nodes 118 may process tasks associated with the content services 112. The computing nodes 118 may be implemented as one or more computing devices, one or more processors, one or more virtual computing instances, combinations thereof, and the like. The computing nodes 118 may be implemented by one or more computing devices. The computing devices may include virtualized computing instances. The virtualized computing instances may include virtual machines, such as emulations of computer systems, operating systems, servers, and the like. The virtual machines may be loaded by the computing devices based on a virtual image and / or other data that defines the particular software for the emulation (e.g., operating system, specialized application, server). As the need for different types of processing services changes, different virtual machines may be loaded and / or terminated on one or more computing devices. A hypervisor may be implemented to manage the use of different virtual machines on the same computing device.

[0037] As described above, a user may create and upload short videos to the cloud network 102 using a content application 106 on a client device 104. In one embodiment, a short video created by a user via a content application 106 on a client device 104 may be a short video featuring a user moving body parts in accordance with the rhythm or beat of a pre-recorded audio overlay (e.g., a pre-recorded audio overlay as described above). For example, a short video created by a user may be a short video featuring a user moving their body parts (e.g., the user's head and / or eyes) to correspond to the rhythm or beat of a song or song clip. As another example, a short video created by a user via a content application 106 on a client device 104 may be a short video featuring a user moving different user body parts (e.g., different users' heads and / or eyes) to correspond to the rhythm or beat of a song or song clip. In another example, a short video includes an object including at least one part having a movement within multiple images of the short video.

[0038] The client device 104 can access an interface 108 of the content application 106. The interface 108 includes an input The input elements may include inputs to the content application 106. By way of example and not limitation, the input elements may be configured to enable a user to create a short video featuring an object moving one of its parts (e.g., the user or another user moving a body part) to the rhythm or beat of a pre-recorded audio overlay. For example, to create a short video featuring a user moving a body part to the rhythm or beat of a pre-recorded audio overlay, the user may give permission to the content application 106 to access an image capture device such as a camera or microphone of the client device 104.

[0039] The client device 104 may use the feed captured by the image capture device to detect (e.g., locate) one or more parts of an object, such as one or more body parts of a user. For example, the client device 104 may use the feed captured by the image capture device to detect the user's head or face. The one or more body parts detected by the camera may include a body part that the user moves to the rhythm or beat of a pre-recorded audio overlay while creating a short video. For example, if the user moves his / her head / face to the rhythm or beat of a pre-recorded audio overlay while creating a short video, the client device 104 may use the feed captured by the image capture device to detect the user's head or face.

[0040] In one embodiment, at least one of the content service 112 or the client device 104 includes one or more machine learning models 110. The machine learning models 110 may be utilized to generate at least one image based at least in part on the text. The text may be associated with a video created by at least one user of the content service 112. For example, if at least one user of the content service 112 wants to create a video with a floral background, the text may include words such as "flower," "floral," and "garden." The text may be received by the content service 112 and / or the client device 104. The text may be entered by at least one user of the content service 112, who wants to create a video, for example. For example, the text may be entered by at least one user into the interface 108 of the content application 106. The text may be entered, for example, using a keyboard of the client device 104 or via voice commands. Additionally or alternatively, the text may include lyrics of one or more songs. The lyrics for the song may be manually entered by at least one user (e.g., within the interface 108 of the content application 106) and / or the song may be a song that is pre-stored in a sound library of the content application 106. If the song is a song that is already stored in the sound library of the content application 106, the lyrics (i.e., text) associated with the song may already be known.

[0041] In some embodiments, the machine learning model 110 may be utilized to generate at least one image based on both the text and the style selection. The style selection may indicate at least one of a color, texture, or art style to be associated with a video created by at least one user of the content services 112. For example, if at least one user of the content services 112 wants to create a video with a black and white background, the style selection may indicate a black and white color scheme. The style selection may be received by the content services 112 and / or the client device 104. The style selection may be input by at least one user of the content services 112, for example, who wants to create a video. For example, the style selection may be input by at least one user in the interface 108 of the content application 106. The style selection may be input by selecting an icon corresponding to a desired style on the interface 108 of the content application 106, for example, using a keyboard of the client device 104 and / or via voice command.

[0042] FIG. 2 illustrates an example of a user interface (UI) 200 of a content application 106. The UI 200 may be utilized by at least one user of the content service 112 who wishes to create a video. The at least one user may enter text into a text box 202. As described above, the text may be associated with a video created by the at least one user of the content service 112. The text may include any amount of characters and / or words. For example, the text may include a single word, a phrase, a sentence, lyrics of one or more songs, and / or any other text. It should be understood that in some embodiments, the text may not be entered manually by the at least one user. For example, if the text includes lyrics of a song, the user may select a song that is pre-stored in a sound library of the content application 106 and the lyrics (i.e., text) associated with the song may already be known.

[0043] In an embodiment, the at least one user may also select a style to be associated with a video created by the at least one user. To select a style, the at least one user may select one or more icons located in style box 204 that correspond to a desired style of the video. For example, five icons located in style box 204 may correspond to a particular style, e.g., a particular color, texture, or art style. The at least one user may select the one or more desired icons by, for example, clicking with his or her finger on the one or more icons that correspond to the desired style of the video.

[0044] For example, as shown in UI 300 of FIG. 3, the at least one user has previously entered text "Cityscape" into text box 202 and selected the fifth icon in style box 204. The text entered by the at least one user and the style selected by the at least one user indicate that the at least one user would like to create a video with a cityscape background in a particular style. After the at least one user has entered text and / or style selection, the user may select button 302 to begin generating the one or more images.

[0045] In some embodiments, the machine learning models 110 include a first machine learning model. The first machine learning model may be configured to receive a text and / or style selection. For example, if a user selects the button 302 to initiate generation of one or more images, the text and / or style selection may be forwarded or sent to the first machine learning model. If at least one user selects a style to be associated with the video, the style may be converted to additional text before being sent to the first machine learning model. For example, the selected style may be converted to additional text. The additional text may be appended to the text entered in the text box 202. The complete text set (i.e., the text entered in the text box 202 and the additional text, if any) may be forwarded or sent to the first machine learning model.

[0046] The first machine learning model may be configured to generate at least one initial image based at least in part on the complete text set. Each word in the complete text set may be mapped to a number or word token. Each number or word token may then be converted into a word vector learned by the first machine learning model. Each word vector may have a dimension of 512. For example, if the complete text set is "an apple on the car", then five word tokens are generated (because there are five words), and then each word token is converted into a 512-dimensional word vector, generating a 5x512 matrix (composed of the five 512-dimensional word vectors). This 5x512 matrix may be used as an input for the first machine learning model. The first machine learning model may generate at least one initial image based on the input. For example, the first machine learning model may output at least one 256x256 image. The content of the at least one initial image may match the meaning of the complete text set.

[0047] In some embodiments, the machine learning models 110 include a second machine learning model. The second machine learning model may be configured to generate training data for training the first machine learning model. The second machine learning model may be configured to receive a plurality of images and generate a plurality of text subtitles corresponding to the plurality of images. For example, the second machine learning model may be configured to receive an image of a white dog and generate a text subtitle corresponding to an image depicting a white dog. The second machine learning model may generate such text subtitles for hundreds, thousands, or millions of images.

[0048] The first machine learning model may be trained on a dataset including the plurality of images and the corresponding plurality of text subtitles. In this way, it can be ensured that the first machine learning model is trained on an accurate image-text pair. In comparison, if the first machine learning model is instead trained on an available open source dataset, the first machine learning model may not perform as well due to common errors in these available open source datasets.

[0049] In some embodiments, the machine learning models 110 include a third machine learning model. The third machine learning model may be configured to enhance the resolution of the initial image output by the first machine learning model. The third machine learning model may be a super-resolution model. For example, the third machine learning model may receive both the complete text and the at least one initial image. The third machine learning model may be configured to generate at least one final image based on the complete text and the at least one initial image. The at least one final image has a higher resolution than the at least one initial image. For example, the at least one final image may be a 1024×1024 image (as opposed to the at least one 256×256 initial image). The at least one final image may be output to at least one user, for example, via an interface 108 of the content application 106.

[0050] In some embodiments, the process of generating the at least one final image using the machine learning model 110 may be time consuming. For example, the machine learning model 110 may take 30 seconds or more to generate a final image. This may result in a poor user experience if the user does not see anything on his / her interface 108 for 30 seconds or more. To remedy this, the machine learning model 110 may generate intermediate effects (e.g., images) gradually while generating the final image. For example, the machine learning model 110 may first generate a portion (e.g., 1 / 8) of the final image on the interface 108, and then another 1 / 8 of the final image (from top to bottom, etc.) until the entire final image is displayed on the screen. or vice versa) gradually, for example every few seconds (i.e., every 2, 3, 4, 5 seconds, for example), another part of the final image may be displayed on the screen, thereby reducing the user's waiting time and improving the user experience.

[0051] 4A and 4B show example UIs 400 and 401 of a content application 106. The UIs 400 and 401 show intermediate effects generated by the machine learning model 110 to reduce user wait time and improve the user experience. As shown in the UI 400, only a portion 402a of the final image is displayed. Over time, additional portions of the final image are gradually displayed. For example, as shown by the UI 401, additional portions of the final image are gradually displayed until the entire final image 402b is displayed on the screen.

[0052] In some embodiments, if the final image is generated entirely on the interface 108, the content application 106 may be configured to generate at least a portion of a real-time camera feed on top of (i.e., overlaid, placed on) the final image. For example, as shown in the UI 500 depicted in FIG. 5, if a user uses the front camera of his / her client device 104, the user's real-time selfie 502 may appear on top of the final generated image 402b such that the final image serves as the background for the user's video. The user may record themselves with the final image in the background.

[0053] In some embodiments, the user may not want to create a video with the final image. For example, the user may want to use a different image as the background of his / her video. If so, the user may select button 504 to restart the image generation process described above. For example, when the user selects button 504, UI 200 reappears and the user may re-enter text (same text or different text) and / or re-select a style (same or different style). FIG. 6A shows an example of UI 600 depicting new text entered by the user in text box 202. After the user has re-entered text and / or re-selected a style, the user may again select button 302 to begin generating the one or more new images. As shown in UI 601 of FIG. 6B, the content application 106 may be configured to generate at least a portion of a real-time camera feed on top of (i.e., overlaid, placed on) the new image. For example, if the user uses the front camera of his / her client device 104, the user's real-time selfie 502 may appear on top of the new image such that the new image serves as the background of the user's video. The user can then record himself / herself with the new image as the background.

[0054] In some embodiments, the content application 106 may place different or additional content on top of the generated image. For example, the content application 106 may place a text graphic or an image graphic (e.g., an emoji) on top of the generated image instead of or in addition to a live camera feed. This content may be overlaid at various locations on the image. The various locations may be selected randomly and / or the locations may be selected according to an algorithm. The content may be of any size, shape, or color. For example, the content may be a text graphic of any size, color, or font.

[0055] In some embodiments, one or more users may wish to create a video narrating a story using the content application 106. A plurality of images corresponding to the story may be generated using text corresponding to the story. For example, the machine learning model 110 may generate an image for each sentence or paragraph of the story. The plurality of images may be, for example, royalty-free. The plurality of images may be used to create a video narrating the story. Instead of a live camera feed of the user appearing on top of the generated images, a text graphic of the text corresponding to the story may be overlaid on the corresponding images. For example, a video may include a plurality of frames, each frame of which displays a particular image and corresponding text of the story. FIG. 7A illustrates an example UI 700 showing a text graphic 702 depicting a sentence of the story overlaid on an image 704 generated by the machine learning model 110. As discussed above, the image 704 may be only one image of a plurality of images generated to narrate the story.

[0056] In some embodiments, one or more users may wish to create a music video for a song using the content application 106. A plurality of images may be generated corresponding to a story using text corresponding to the lyrics of the song. For example, the machine learning model 110 may generate an image for each song lyric. The plurality of images may be, for example, royalty-free. The plurality of images may be used to create a music video for the song. Instead of a live camera feed of the user appearing on top of the generated images, a text graphic of the text of the song lyrics may be overlaid on the corresponding images. For example, a video may include a plurality of frames, each frame of which displays a particular image and corresponding song lyrics. FIG. 7B illustrates an example UI 701 showing a text graphic 706 depicting the lyrics of the song overlaid on an image 708 generated by the machine learning model 110. As discussed above, the image 708 may be only one image of a plurality of images generated for the music video.

[0057] In some embodiments, the machine learning model 110 is configured to generate multiple different images from the same input text prompt. For example, two or more users of the content service 112 may input the same text prompt, and the machine learning model 110 may generate a different image associated with the text prompt for each user. Each of the multiple different images may be randomly generated from the same input text. The machine learning model 110 may be configured to ensure that each of the multiple different images has a correct correlation with the input text.

[0058] Multiple users can use the same text (i.e., the same word, phrase, or sentence) to create a shared virtual world with the multiple different images. FIG. 8 illustrates an example UI 800 depicting multiple users creating videos and sharing a virtual world with different images generated based on the same text entered by the multiple users. Each of the multiple users can travel, play, or interact in the virtual world. For example, each of the multiple users can take creative photos or shoot creative videos with the multiple generated images as a background.

[0059] 9 illustrates an example process 900 performed by a client device (e.g., client device 104). Client device 104 may perform process 900 to create content, such as content for uploading to a content service (i.e., content service 112). Once uploaded to the content service, the content may be viewed by users of the content service. Although illustrated in FIG. 9 as a series of operations, one skilled in the art will recognize that in various embodiments, the operations described may be added, deleted, reordered, or modified.

[0060] As described above, one or more machine learning models (i.e., machine learning models 110) may be utilized to generate at least one image based at least in part on the text. At 902, text may be received. The text may be associated with a video created by at least one user, e.g., one or more users of a content service. For example, if at least one user wants to create a video with a floral background, the text may include words such as "flower," "floral," "garden," etc. The text may be entered, for example, by at least one user who wants to create a video. For example, the text may be entered by at least one user into an interface of a content application. The text may be entered, for example, with a keyboard of a client device or via voice commands. Additionally or alternatively, the text may include lyrics of one or more songs. The lyrics of the songs may be entered manually (e.g., into an interface of the content application) by at least one user. And / or the song may be a song that is pre-stored in a sound library of the content application. If the song is a song that is already stored in a sound library of the content application, the lyrics (i.e., text) associated with the song may already be known.

[0061] At 904, at least one image may be generated using at least one machine learning model based at least in part on the text. For example, a first machine learning model may be configured to generate at least one initial image based at least in part on the text. The at least one initial image may include, for example, at least one 256×256 image. The content of the at least one initial image may match the meaning of the text. A second machine learning model may be configured to generate training data to train the first machine learning model. For example, the second machine learning model may be configured to receive a plurality of images and generate a plurality of text subtitles corresponding to the plurality of images. The second machine learning model may generate such text subtitles for hundreds, thousands, or millions of images. The first machine learning model may be trained on a dataset including the plurality of images and the corresponding plurality of text subtitles. In this way, it can be ensured that the first machine learning model is trained on an accurate image-text pair.

[0062] The third machine learning model may be configured to enhance the resolution of the initial image output by the first machine learning model. The third machine learning model may be a super-resolution model. For example, the third machine learning model may receive both the complete text and the at least one initial image. The third machine learning model may be configured to generate at least one final image based on the complete text and the at least one initial image. The at least one final image has a higher resolution than the at least one initial image. For example, the at least one final image may be a 1024×1024 image (as opposed to the at least one 256×256 initial image). The at least one final image may be output to at least one user, for example, via an interface of a content application.

[0063] At 906, a video may be generated based at least in part on the at least one image. At least one frame of the video may include content overlaid on the at least one image. For example, the content may include at least a portion of a real-time camera feed. The at least a portion of the real-time camera feed may be overlaid on the generated image. For example, if a user uses a front camera of his or her client device, A real-time selfie of the user may appear over the generated image such that the generated image serves as a background for the user's video. The user may record themselves against the background of the generated image. Additionally or alternatively, the content may include text graphics or image graphics (e.g., emojis) over the generated image. For example, as described above, the content may include song lyrics and / or text graphics of a story. This content may be overlaid at various locations on the image. The various locations may be selected randomly. And / or the locations may be selected according to an algorithm. The content may be of any size, shape, or color. For example, the content may be a text graphic of any size, color, or font.

[0064] 10 illustrates an example process 1000 performed by a client device (e.g., client device 104). Client device 104 may perform process 1000 to create content, such as content for uploading to a content service (i.e., content service 112). Once uploaded to the content service, the content may be viewed by users of the content service. Although illustrated in FIG. 10 as a series of operations, one skilled in the art will recognize that in various embodiments, the operations described may be added, deleted, reordered, or modified.

[0065] As described above, one or more machine learning models (i.e., machine learning models 110) may be utilized to generate at least one image based on the text and style selection. At 1002, text may be received. The text may be associated with a video created by at least one user, e.g., one or more users of a content service. For example, if at least one user wants to create a video with a floral background, the text may include words such as "flower," "floral," "garden," etc. The text may be entered, for example, by at least one user who wants to create a video. For example, the text may be entered by at least one user into an interface of a content application. The text may be entered, for example, with a keyboard of a client device or via voice commands. Additionally or alternatively, the text may include lyrics of one or more songs. The lyrics of the songs may be entered manually (e.g., into an interface of the content application) by at least one user. And / or the song may be a song that is pre-stored in a sound library of the content application. If the song is a song that is already stored in a sound library of the content application, the lyrics (i.e., text) associated with the song may already be known.

[0066] At 1004, a style selection may be received to be associated with a video created by the at least one user. The style selection may indicate at least one of a color, texture, or art style to be associated with a video created by the at least one user of the content service. For example, if the at least one user wants to create a video with a black and white background, the style selection may indicate a black and white color scheme. The style selection may be input, for example, by the at least one user who wants to create the video. For example, the style selection may be input by the at least one user in an interface of a content application. The style selection may be input, for example, by selecting an icon corresponding to a desired style on an interface of the content application using a keyboard of a client device and / or via voice commands.

[0067] At 1006, at least one image may be generated based on the text and the style selection using at least one machine learning model. The style selection may be converted into additional text before the at least one machine learning model generates the at least one image. For example, the selected style may be converted into additional text. The additional text may be appended to the text to create a complete text set.

[0068] The complete text set (i.e., the text and the additional text) may be forwarded or transmitted to a first machine learning model. The first machine learning model may be configured to generate at least one initial image based at least in part on the complete text set. The at least one initial image may include, for example, at least one 256×256 image. The content of the at least one initial image may match the meaning of the complete text set. The second machine learning model may be configured to generate training data for training the first machine learning model. For example, the second machine learning model may be configured to receive a plurality of images and generate a plurality of text subtitles corresponding to the plurality of images. The second machine learning model may generate such text subtitles for hundreds, thousands, or millions of images. The first machine learning model may be trained on a dataset including the plurality of images and the corresponding plurality of text subtitles. In this way, it can be ensured that the first machine learning model is trained on the correct image-text pairs.

[0069] The third machine learning model may be configured to enhance the resolution of the initial image output by the first machine learning model. The third machine learning model may be a super-resolution model. For example, the third machine learning model may receive both the complete text and the at least one initial image. The third machine learning model may be configured to generate at least one final image based on the complete text and the at least one initial image. The at least one final image has a higher resolution than the at least one initial image. For example, the at least one final image may be a 1024×1024 image (as opposed to the at least one 256×256 initial image). The at least one final image may be output to at least one user, for example, via an interface of a content application.

[0070] At 1008, a video may be generated based at least in part on the at least one image. At least one frame of the video may include content overlaid on the at least one image. For example, the content may include at least a portion of a real-time camera feed. The at least a portion of the real-time camera feed may be overlaid on the generated image. For example, if a user uses a front camera of his / her client device, a real-time selfie of the user may appear on the generated image such that the generated image serves as a background for the user's video. The user may record themselves against the background of the generated image. Additionally or alternatively, the content may include text graphics or image graphics (e.g., emojis) on the generated image. For example, as described above, the content may include text graphics of song lyrics and / or stories. This content may be overlaid at various locations on the image. The various locations may be selected randomly and / or the locations may be selected according to an algorithm. The content may be of any size, shape, or color. For example, the content may be text graphics of any size, color, or font.

[0071] 11 illustrates an example process 1100 performed by a client device (e.g., client device 104). Client device 104 may perform process 1100 to create content, such as content for uploading to a content service (i.e., content service 112). Once uploaded to the content service, the content may be viewed by users of the content service. Although illustrated in FIG. 11 as a series of operations, one skilled in the art will recognize that in various embodiments, the operations described may be added, deleted, reordered, or modified.

[0072] As described above, one or more machine learning models (i.e., machine learning models 110) may be utilized to generate at least one image based at least in part on the text. At 1102, a second machine learning model may be used to generate a plurality of text subtitles corresponding to a plurality of images. The second machine learning model may be configured to receive a plurality of images and generate a plurality of text subtitles corresponding to the plurality of images. For example, the second machine learning model may be configured to receive an image of a white dog and generate a text subtitle corresponding to an image depicting a white dog. The second machine learning model may generate such text subtitles for hundreds, thousands, or millions of images.

[0073] A first machine learning model may be trained on a dataset including the plurality of images and the corresponding plurality of text subtitles. At 1104, a first machine learning model may be trained on a dataset including the plurality of corresponding text subtitles and the plurality of images. By training the first machine learning model on the image-text pairs generated by the second machine learning model, it can be ensured that the first machine learning model is trained on accurate image-text pairs. In comparison, if the first machine learning model is instead trained on available open source datasets, the first machine learning model may not perform as well due to common errors in these available open source datasets.

[0074] At 1106, text may be received. The text may be associated with a video created by at least one user, e.g., one or more users of a content service. For example, if at least one user wants to create a video with a floral background, the text may include words such as "flower," "floral," "garden," etc. The text may be entered, for example, by at least one user who wants to create a video. For example, the text may be entered by at least one user into an interface of a content application. The text may be entered, for example, with a keyboard of a client device or via voice commands. Additionally or alternatively, the text may include lyrics of one or more songs. The lyrics of the songs may be entered manually (e.g., into an interface of the content application) by at least one user and / or the songs may be songs that are pre-stored in a sound library of the content application. If the song is a song that is already stored in the sound library of the content application, the lyrics (i.e., text) associated with the song may already be known.

[0075] At 1108, the first machine learning model may be used to generate at least one initial image based at least in part on the text. The at least one initial image may include, for example, at least one 256×256 image. The content of the at least one initial image may match the meaning of the text. A third machine learning model may be configured to enhance the resolution of the initial image output by the first machine learning model. The third machine learning model may be a super-resolution model. For example, the third machine learning model may be configured to generate at least one initial image based on the text and the at least one initial image. The third machine learning model may receive both the text and the at least one initial image. The third machine learning model may be configured to generate at least one image based on the text and the at least one initial image. At 1110, the third machine learning model may be used to generate at least one image based at least in part on the text and the at least one initial image. The at least one image generated by the third machine learning model is closely related to the text and has a higher resolution than the at least one initial image. For example, the at least one image may be a 1024×1024 image (as opposed to the at least one 256×256 initial image). The at least one image may be output to at least one user, for example, via an interface of a content application.

[0076] At 1112, a video may be generated based at least in part on the at least one image. At least one frame of the video may include content overlaid on the at least one image. For example, the content may include at least a portion of a real-time camera feed. The at least a portion of the real-time camera feed may be overlaid on the generated image. For example, if a user uses a front camera of his / her client device, a real-time selfie of the user may appear on top of the generated image such that the generated image serves as a background for the user's video. The user may record themselves against the background of the generated image. Additionally or alternatively, the content may include text graphics or image graphics (e.g., emojis) on top of the generated image. For example, as described above, the content may include text graphics of song lyrics and / or stories. This content may be overlaid at various locations on the image. The various locations may be selected randomly and / or the locations may be selected according to an algorithm. The content may be of any size, shape, or color. For example, the content may be text graphics of any size, color, or font.

[0077] Figure 12 illustrates a computing device that may be used in various aspects, such as the services, networks, modules, and / or devices illustrated in Figure 1. With respect to the example architecture of Figure 1, the message service, interface service, processing service, content service, cloud network, and clients may each be implemented by one or more instances of computing device 1200 of Figure 12. The computer architecture illustrated in Figure 12 illustrates a conventional server computer, workstation, desktop computer, laptop computer, tablet, network appliance, PDA, electronic reader, digital mobile phone, or other computing node that may be used to perform any aspect of the computer described herein, such as implementing the methods described herein.

[0078] Computing device 1200 may include a board or "motherboard," which is a printed circuit board that can be connected to multiple components or devices via a system bus or other electrical communication paths. One or more central processing units (CPUs) 1204 may operate in conjunction with a chipset 1206. CPU 1204 may be a standard programmable processor that performs arithmetic and logical operations necessary for the operation of computing device 1200.

[0079] CPU 1204 may perform the necessary operations to transition from one discrete physical state to the next by operating switching elements that distinguish between these states. Switching elements may typically include electronic circuits, such as flip-flops, that maintain one of two binary states, and electronic circuits, such as logic gates, that provide an output state based on a logical combination of the states of one or more other switching elements. These basic switching elements may be combined to form more complex logic circuits, including registers, adders and subtractors, arithmetic logic units, floating point units, etc.

[0080] CPU 1204 may be augmented or replaced by other processing units, such as GPUs, which may include processing units specialized for highly parallel computations, such as, but not necessarily limited to, graphics and other visualization-related processing.

[0081] The chipset 1206 may provide an interface between the CPU 1204 and the remaining components and devices on the board. The chipset 1206 may provide an interface to a random access memory (RAM) 1208 used as a main memory in the computing device 1200. The chipset 1206 may also provide an interface to a computer-readable storage medium, such as a read-only memory (ROM) 1220 or a non-volatile RAM (NVRAM) (not shown), for storing basic routines that may facilitate starting the computing device 1200 and transmitting information between various components and devices. According to aspects described herein, the ROM 1220 or the NVRAM may store other software components necessary for the operation of the computing device 1200.

[0082] Computing device 1200 may operate in a networked environment using logical connections to remote computing nodes and computer systems via a local area network (LAN). Chipset 1206 may include functionality for providing network connectivity via a network interface controller (NIC) 1222, such as a Gigabit Ethernet adapter. NIC 1222 may be capable of connecting computing device 1200 to other computing nodes via network 1216. It should be understood that multiple NICs 1222 may be present within computing device 1200 to connect the computing device to other types of networks and remote computer systems.

[0083] The computing device 1200 may be connected to a mass storage device 1228 that provides non-volatile storage for the computer. The mass storage device 1228 may store system programs, application programs, other program modules, and data, as described in more detail herein. The mass storage device 1228 may be connected to the computing device 1200 through a storage controller 1224 that is connected to the chipset 1206. The mass storage device 1228 may be comprised of one or more physical storage units. The mass storage device 1228 may include a management component 1210. The storage controller 1224 may interface with the physical storage units through a Serial Attached SCSI (SAS) interface, a Serial Advanced Technology Attachment (SATA) interface, a Fibre Channel (FC) interface, or any other type of interface for physically connecting and transmitting data between the computer and the physical storage units.

[0084] Computing device 1200 may store data on mass storage device 1228 by transforming the physical state of the physical storage units to reflect the stored information. The particular transformation of the physical state may depend on a variety of factors and different implementations of the present specification. Examples of such factors include, but are not limited to, the technology for implementing the physical storage devices and whether mass storage device 1228 is characterized as a primary storage device, a secondary storage device, etc.

[0085] For example, computing device 1200 may store information in mass storage device 1228 by issuing instructions via storage controller 1224 to change the magnetic properties of a particular location in a magnetic disk drive unit, the reflective or refractive properties of a particular location in an optical storage unit, or the electrical properties of a particular capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of physical media are possible without departing from the scope and spirit of this specification, and the foregoing examples are provided merely for ease of explanation thereof. Computing device 1200 may also read information from mass storage device 1228 by detecting a physical state or characteristic of one or more particular locations in a physical storage unit.

[0086] In addition to the mass storage device 1228 discussed above, computing device 1200 may access other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. Those skilled in the art will appreciate that computer-readable storage media can be any available media that provides non-transitory storage of data and that can be accessed by computing device 1200.

[0087] By way of example and not limitation, computer readable storage media may include volatile and non-volatile, temporary and non-transient computer readable storage media, as well as removable and non-removable media implemented in any manner or technology. Computer readable storage media includes, but is not limited to, RAM, ROM, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory or other solid state memory technology, compact disc ROM (CD-ROM), digital versatile disc (DVD), high definition DVD ("HD-DVD"), BLU-RAY or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage, other magnetic storage device, or any other medium that can be used to store desired information in a non-transitory manner.

[0088] A mass storage device, such as mass storage device 1228 shown in FIG. 12, may store an operating system for controlling the operation of computing device 1200. The operating system may include a version of the LINUX operating system. The operating system may include a version of Microsoft's WINDOWS® SERVER operating system. According to another embodiment, the operating system may include a version of the UNIX® operating system. Also, various mobile phone operating systems, such as IOS and ANDROID®, may be utilized. It should be understood that other operating systems may also be utilized. Mass storage device 1228 may store other system or applications and data used by computing device 1200.

[0089] The mass storage device 1228 or other computer-readable storage medium may also be encoded with computer-executable instructions that, when loaded into computing device 1200, transform the computing device from a general-purpose computing system into a special-purpose computer capable of implementing aspects described herein. As described above, these computer-executable instructions transform computing device 1200 by defining how CPU 1204 transitions between states. Computing device 1200 may access a computer-readable storage medium that stores computer-executable instructions that, when executed by computing device 1200, can perform the methods described herein.

[0090] The components shown in Figure 12 A computing device such as computing device 1200 may further include an input / output controller 1232 for receiving and processing input from a number of input devices, such as a keyboard, a mouse, a touchpad, a touch screen, an electronic stylus pen, or other types of input devices. Similarly, the input / output controller 1232 may provide output to a display, such as a computer monitor, a flat panel display, a digital projector, a printer, a plotter, or other type of output device. It should be understood that computing device 1200 may not include all of the components shown in FIG. 12, may include other components not explicitly shown in FIG. 12, or may utilize an entirely different architecture than that shown in FIG. 12.

[0091] As described herein, a computing device may be a physical computing device, such as computing device 1200 of Figure 12. A computing node may also include a virtual machine host process and one or more virtual machine instances. Computer-executable instructions may be executed indirectly by the physical hardware of the computing device by interpreting and / or executing instructions that are stored and executed within the context of a virtual machine.

[0092] It should be understood that the methods and systems are not limited to particular methods, components, or implementations, and it should also be understood that the terminology used herein is used for the purpose of describing particular embodiments only, and is not intended to be limiting.

[0093] As used in the specification and the appended claims, the singular forms "a," "one," and "the" include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from "about" one particular value, and / or to "about" another particular value. When such a range is expressed, another embodiment includes the range from the one particular value and / or to the other particular value. Similarly, when values ​​are expressed as approximations, by use of the antecedent "about," it is to be understood that the particular value forms another embodiment. It is further to be understood that the endpoints of each of the ranges are significant both relative to the other endpoint, and independently of the other endpoint.

[0094] "Optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and the specification includes cases where said event or circumstance occurs and cases where said event or circumstance does not occur.

[0095] Throughout the description and claims of this specification, the word "comprise" and variations of that word, such as "including" and "including," mean "including, but not limited to," and are not intended to exclude, for example, other elements, integers, or steps. "Exemplary" denotes "an example of," and is not intended to convey an indication of a preferred or desirable embodiment. "Such as" is used for purposes of interpretation rather than a limiting sense.

[0096] Components are described that can be used to perform the described methods and systems. In describing combinations, subsets, interactions, groups, etc. of these components, specific references to each of the various individual and collective combinations and permutations of these components may not be explicitly described, and each of which is to be understood as specifically contemplated and described herein for all methods and systems. This applies to all aspects of this application, including but not limited to the operations in the described methods. Thus, where there are various additional operations that can be performed, it should be understood that each of these additional operations can be performed in any particular embodiment or combination of embodiments of the described methods.

[0097] The present method and system may be more readily understood by reference to the following detailed description of the preferred embodiment and examples contained therein, as well as the accompanying drawings and their descriptions.

[0098] As will be appreciated by those skilled in the art, the method and system may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the method and system may take the form of a computer program product on a computer-readable storage medium having computer-readable program instructions (e.g., computer software) embodied in the storage medium. More specifically, the method and system may take the form of web-implemented computer software. Any suitable computer-readable storage medium may be utilized, including hard disks, CD-ROMs, optical storage devices, or magnetic storage devices.

[0099] Embodiments of the methods and systems are described below with reference to block diagrams and flowcharts of methods, systems, devices, and computer program products. It should be understood that each block of the block diagrams and flowcharts, and combinations of blocks in the block diagrams and flowcharts, respectively, may be realized by computer program instructions. These computer program instructions may be loaded into a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute on the computer or other programmable data processing apparatus, generate means for implementing the functions specified in one or more blocks of the flowcharts.

[0100] These computer program instructions may be stored in a computer readable memory capable of directing a computer or other programmable data processing apparatus to operate in a particular manner such that the instructions stored in the computer readable memory produce a product including computer readable instructions for implementing the functions defined in one or more blocks of the flowcharts. The computer program instructions may be loaded into a computer or other programmable data processing apparatus to cause the computer or other programmable data processing apparatus to perform a series of operational steps to produce a computer implemented process such that the instructions, which execute on the computer or other programmable data processing apparatus, provide steps for implementing the functions defined in one or more blocks of the flowcharts.

[0101] The various features and processes described above may be used independently of each other or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure. Furthermore, in some implementations, some method or process blocks may be omitted. The methods and processes described herein are not limited to any particular order, and the blocks or states associated therewith may be performed in other orders as appropriate. For example, the described blocks or states may be performed in an order other than the order specifically described, or multiple blocks or states may be combined within a single block or state. The example blocks or states may be performed sequentially, in parallel, or in some other manner. Blocks or states may be added to or deleted from the described example embodiments. The example systems and components described herein may be configured differently than as described. For example, elements may be added, deleted, or rearranged as compared to the described example embodiments.

[0102] It should also be understood that while various items are shown as being stored in memory or on a storage device during use, these items, or portions thereof, may be transferred between memory and other storage devices for purposes of memory management and data integrity. Alternatively, in other embodiments, some or all of the software modules and / or systems may run in memory on another device and communicate with the illustrated computing system via computer-to-computer communications. Furthermore, in some embodiments, some or all of the systems and / or modules may be implemented or provided in other ways, such as at least partially in firmware and / or hardware. Hardware includes, but is not limited to, one or more application specific integrated circuits (ASICs), standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and / or embedded controllers), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and the like. Some or all of the modules, systems, and data structures may be stored (e.g., as software instructions or structured data) on a computer-readable medium, such as a hard disk, memory, network, or portable media product, for reading by an appropriate device or via an appropriate connection. The systems, modules, and data structures may be transmitted as a generated data signal (e.g., as part of a carrier wave or other analog or digital propagated signal) over a variety of computer-readable transmission media, including wireless-based media and wire / cable-based media, or may be contained on a computer-readable transmission medium and take a variety of forms (e.g., as part of a single or multiplexed analog signal, or as a number of discrete digital packets or frames). In other embodiments, such computer program products may take other forms. Thus, the invention may be practiced with other computer system configurations.

[0103] While the methods and systems have been described in connection with preferred embodiments and specific examples, it is not intended that the scope be limited to the specific embodiments, as the embodiments herein are intended in all respects to be illustrative and not restrictive.

[0104] Unless expressly stated, the methods described herein do not require that their operations be performed in a particular order. Thus, where a method claim does not actually recite the order in which its operations are to be followed, or where the claims or specification are not specifically recited to limit the operations to a particular order, no order is intended to be inferred in any manner. This applies to any possible non-expressive basis for interpretation, including logical issues regarding the arrangement of steps or operational flow, the simple meaning derived from grammatical constructions and punctuation, or the number or type of embodiments described in the specification.

[0105] As will be apparent to those skilled in the art, various modifications and variations can be made without departing from the scope or spirit of the present disclosure. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice described herein. It is intended that the specification and illustrative drawings be considered as exemplary only, with its true scope and spirit being indicated by the following claims.

Claims

1. 1. A computer-implemented method for creating content, comprising: receiving text associated with at least one user-created video; generating at least one image based at least in part on the text using at least one machine learning model; and generating the video including content overlaid on the at least one image based at least in part on the at least one image.

2. The method of claim 1 , further comprising receiving a style selection to be associated with the at least one user-created video, wherein the at least one image is generated based on the style selection and the text.

3. The method of claim 2 , wherein the style selection indicates at least one of a color, a texture, or an art style.

4. 2. The method of claim 1 , wherein the at least one machine learning model comprises: a first machine learning model configured to receive the text and generate at least one initial image based at least in part on the text; and a second machine learning model configured to receive a plurality of images and generate a plurality of text subtitles corresponding to the plurality of images, wherein the first machine learning model is trained on a dataset including the plurality of images and the corresponding plurality of text subtitles.

5. 5. The method of claim 4 , wherein the at least one machine learning model further comprises a third machine learning model configured to receive the text and the at least one initial image, the third machine learning model further configured to generate the at least one image based on the text and the at least one initial image, the at least one image having a higher resolution than the at least one initial image.

6. The method of claim 1 , wherein the text includes input from the at least one user, and the content includes at least a portion of a live camera feed.

7. The method of claim 1 , wherein the text includes lyrics for a plurality of songs, and the video includes a plurality of images corresponding to lyrics for the plurality of songs.

8. The method of claim 1 , wherein the at least one image comprises different images generated based at least in part on the same text entered by different users.

9. The method of claim 8 , wherein the video includes content overlaid on at least a subset of the plurality of different images.

10. 1. A system comprising at least one processor in communication with at least one memory, the at least one memory comprising computer readable instructions that, when executed by the at least one processor, are configured to perform operations including: receiving text associated with at least one user-created video; generating, using at least one machine learning model, at least one image based at least in part on the text; and generating the video, the video including content overlaid on the at least one image, based at least in part on the at least one image.

11. 11. The system of claim 10, wherein the operations further include receiving a style selection to be associated with the video created by the at least one user, and wherein the at least one image is generated based on the style selection and the text, the style selection indicating at least one of a color, a texture, or an art style.

12. 11. The system of claim 10, wherein the at least one machine learning model comprises: a first machine learning model configured to receive the text and generate at least one initial image based at least in part on the text; and a second machine learning model configured to receive a plurality of images and generate a plurality of text subtitles corresponding to the plurality of images, wherein the first machine learning model is trained on a dataset including the plurality of images and the corresponding plurality of text subtitles.

13. 13. The system of claim 12, wherein the at least one machine learning model further comprises a third machine learning model configured to receive the text and the at least one initial image, the third machine learning model further configured to generate the at least one image based on the text and the at least one initial image, the at least one image having a higher resolution than the at least one initial image.

14. The system of claim 10 , wherein the text includes input from the at least one user, and the content includes at least a portion of a live camera feed.

15. 11. The system of claim 10, wherein the at least one image includes different images generated based at least in part on the same text entered by different users, and the video includes content overlaid on at least a subset of the different images.

16. 1. A non-transitory computer-readable storage medium having stored thereon computer-readable instructions that, when executed by a processor, cause the processor to perform operations including: receiving text associated with at least one user created video; generating at least one image based at least in part on the text using at least one machine learning model; and generating the video including content overlaid on the at least one image based at least in part on the at least one image.

17. 17. The non-transitory computer-readable storage medium of claim 16, wherein the operations further include receiving a style selection to be associated with the video created by the at least one user, and wherein the at least one image is generated based on the style selection and the text, the style selection indicating at least one of a color, a texture, or an art style.

18. 17. The non-transitory computer-readable storage medium of claim 16, wherein the at least one machine learning model comprises: a first machine learning model configured to receive the text and generate at least one initial image based at least in part on the text; and a second machine learning model configured to receive a plurality of images and generate a plurality of text subtitles corresponding to the plurality of images, wherein the first machine learning model is trained on a dataset including the plurality of images and the corresponding plurality of text subtitles.

19. 20. The non-transitory computer-readable storage medium of claim 18, wherein the at least one machine learning model comprises a third machine learning model configured to receive the text and the at least one initial image, the third machine learning model further configured to generate the at least one image based on the text and the at least one initial image, the at least one image having a higher resolution than the at least one initial image.

20. The non-transitory computer-readable storage medium of claim 16 , wherein the text includes input from the at least one user and the content includes at least a portion of a live camera feed.

Images