Method, apparatus, device and storage medium for generating video content

Abstract

Embodiments of the present disclosure relate to a method, apparatus, device and storage medium for generating video content. The method presented herein includes: obtaining input content of a user via an interaction interface with a virtual object; presenting first response content from the virtual object in the interaction interface, the first response content including at least one candidate image generated based on the input content; determining video generation information based on an interaction operation associated with the first response content, the video generation information including at least a reference image for generating a video; and providing second response content from the virtual object, the second response content including video content generated based on the video generation information. In this way, embodiments of the present disclosure are able to present at least one candidate image generated based on the input content to the user in the process of video generation, so that the quality of video generation can be improved based on the interaction operation of the user with the at least one candidate image.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein relate generally to the field of computers, and in particular to methods, apparatus, devices, and computer-readable storage media for generating video content. Background Technology

[0002] With the development of computer technology, more and more platforms and applications are providing video generation services to users. For example, some platforms or applications can generate corresponding video content based on user-input text or images. How to improve the quality of generated media content is a key concern. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for generating video content is provided. The method includes: acquiring user input via an interactive interface with a virtual object; presenting first response content from the virtual object in the interactive interface, the first response content including at least one candidate image generated based on the input content; determining video generation information based on an interactive operation associated with the first response content, the video generation information including at least a reference image for generating the video; and providing second response content from the virtual object, the second response content including video content generated based on the video generation information.

[0004] In a second aspect of this disclosure, an apparatus for generating video content is provided. The apparatus includes: an acquisition module configured to acquire user input via an interactive interface with a virtual object; a first presentation module configured to present first response content from the virtual object in the interactive interface, the first response content including at least one candidate image generated based on the input content; a first determination module configured to determine video generation information based on an interactive operation associated with the first response content, the video generation information including at least a reference image for generating the video; and a providing module configured to provide second response content from the virtual object, the second response content including video content generated based on the video generation information.

[0005] In a third aspect of this disclosure, an electronic device is provided. The device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the device to perform the method of the first aspect.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to implement the method of the first aspect.

[0007] In a fifth aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method of the first aspect.

[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram is shown of an example environment in which embodiments of the present disclosure may be implemented;

[0011] Figures 2A to 2J Example interfaces according to some embodiments of this disclosure are shown;

[0012] Figure 3 A flowchart illustrating an example process for generating video content according to some embodiments of this disclosure is shown;

[0013] Figure 4 A schematic structural block diagram of an example apparatus for generating video content according to some embodiments of the present disclosure is shown; and

[0014] Figure 5 A block diagram of an electronic device capable of implementing several embodiments of the present disclosure is shown. Detailed Implementation

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0017] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0018] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.

[0019] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information other than that necessary for basic functions will not affect the user's use of basic functions.

[0020] As mentioned above, with the development of computer technology, more and more platforms and applications are providing video generation services to users. For example, some platforms or applications can support the generation of corresponding video content based on user-input text or images. How to improve the quality of generated media content is a key concern.

[0021] Embodiments of this disclosure propose a scheme for generating video content. According to this scheme, user input can be obtained via an interactive interface with a virtual object. Further, a first response from the virtual object can be presented in the interactive interface, the first response including at least one candidate image generated based on the input. Additionally, video generation information can be determined based on interactive operations associated with the first response, the video generation information including at least a reference image for generating the video. Furthermore, a second response from the virtual object can be provided, the second response including video content generated based on the video generation information.

[0022] Based on this approach, embodiments of this disclosure can provide at least one candidate image generated based on user input before providing video content from virtual objects. This allows for improved quality of video generation information used for video generation through user interaction with the at least one candidate image. Consequently, embodiments of this disclosure can generate higher-quality video content based on higher-quality video generation information.

[0023] Therefore, embodiments of this disclosure can present at least one candidate image generated based on input content to the user during the video generation process, thereby improving the quality of video generation based on the user's interaction with at least one candidate image.

[0024] The following section provides a detailed description of various example implementations of this scheme, with reference to the accompanying drawings.

[0025] Example Environment

[0026] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, example environment 100 may include electronic device 110.

[0027] In this example environment 100, electronic device 110 may run an application 120 that supports the generation of video content. Application 120 may be any suitable type of application for generating video content, and examples may include, but are not limited to, media applications, social applications, or other suitable applications. User 140 may interact with application 120 via electronic device 110 and / or its attached devices.

[0028] exist Figure 1 In environment 100, if application 120 is active, electronic device 110 can use application 120 to present interface 150 for supporting the generation of video content.

[0029] In some embodiments, electronic device 110 communicates with server 130 to provide services to application 120. Electronic device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, handheld computers, portable gaming terminals, VR / AR devices, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio receivers, e-book devices, gaming devices, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. In some embodiments, electronic device 110 can also support any type of user-facing interface (such as "wearable" circuitry).

[0030] Server 130 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms. Server 130 may include, for example, computing systems / servers such as mainframes, edge computing nodes, computing devices in a cloud environment, etc. Server 130 can provide backend services for applications 120 that support content presentation in electronic devices 110.

[0031] A communication connection can be established between server 130 and electronic device 110. This communication connection can be established via wired or wireless means. The communication connection may include, but is not limited to, Bluetooth, mobile network, Universal Serial Bus (USB), and Wireless Fidelity (WiFi) connections; the embodiments of this disclosure are not limited in this respect. In the embodiments of this disclosure, server 130 and electronic device 110 can achieve signaling interaction through the communication connection between them.

[0032] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure.

[0033] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0034] Example Interaction

[0035] The following will be referenced Figures 2A to 2J To describe some example interaction processes according to embodiments of the present disclosure. Figures 2A to 2J Example interfaces 200A to 200J according to some embodiments of the present disclosure are shown. Interfaces 200A to 200J can be, for example, provided by... Figure 1 The electronic device 110 shown is provided.

[0036] In some embodiments, the electronic device 110 may present as follows: Figure 2A The interface 200A shown may include an interface for generating video content. For example... Figure 2A As shown, electronic device 110 can present an interface 200A for user 140 to interact with a virtual object (e.g., virtual object 210), and generate video content through the interaction between user 140 and the virtual object. In some scenarios, interface 200A may include an interactive interface for user 140 to interact with the virtual object. It should be understood that, on the one hand, the virtual object may have a virtual avatar for interaction. On the other hand, the virtual object may also utilize an appropriate model to drive its interactive behavior, such a model may include any appropriate machine learning model, for example, a generative model. In this disclosure, the various interactive processes and / or generation operations performed by the virtual object can be actually performed using an appropriate model corresponding to the virtual object.

[0037] In some scenarios, such an interactive interface may include a one-on-one chat session between user 140 and the virtual object, or at least a group chat session with user 140 and the virtual object as participants. In other scenarios, such a chat interface may also include a conversational search interface associated with the virtual object, which may search for content based on the conversation between user 140 and the virtual object, and then present the search results through messages replied by the virtual object. In some other scenarios, such an interactive interface may also have other suitable forms. For example, the interactive interface may correspond to the interactive space of the virtual object, which may support multi-modal conversational interactions between user 140 and the virtual object.

[0038] In some scenarios, user 140 can trigger input to instruct a virtual object to generate corresponding video content based on the user 140's input. For example, user 140 can trigger the sending of message 221 (e.g., including displaying an image of a small ball) and message 222 (e.g., including the text "Turn the image into multiple small balls and make it move"). In some examples, such messages 221 and 222 can be referred to as the user's input. In some examples, the user's input may include at least one of text, images, etc. Thus, electronic device 110 can provide video content generated by the virtual object.

[0039] To improve the quality of the generated video content, in some embodiments of this disclosure, the electronic device 110 may, upon receiving input from the user 140, present first response content from a virtual object in an interactive interface. Such first response content includes at least one candidate image generated based on the input from the user 140.

[0040] Furthermore, the electronic device 110 can determine video generation information based on the user 140's interactive actions on the content 231. Such video generation information may include at least a reference image for generating the video. Therefore, the electronic device 110 can provide second response content from the virtual object, which may include video content generated based on such video generation information.

[0041] Based on this approach, embodiments of this disclosure can improve the quality of video generation information through interactive operations associated with at least one candidate image, thereby improving the quality of generated video content based on such video generation information. For ease of explanation, the following description first refers to the accompanying drawings. Figure 2B To be continued Figure 2F Let's take at least one candidate image as an example to describe it in detail.

[0042] In some embodiments, according to Figure 2B As shown in the interface 200B, the electronic device 110 can provide virtual object response content 231 (also known as, first response content) to messages 221 and 222, such content 231 may include at least one candidate image (e.g., image 232) generated based on messages 221 and 222.

[0043] As an example, such content 231 could include the reply text to message 222, "I will add multiple balls to this image." Furthermore, electronic device 110 could display an image 232 generated based on messages 221 and 222, which could include multiple balls. Based on such image 232, user 140 could quickly preview the visual effect of the video content to be generated. Furthermore, user 140 could interact with content 231 to trigger an interaction based on content 231 to determine video generation information, which could at least include reference images used to generate the video.

[0044] As examples, such image 232 may be generated using a model. For instance, electronic device 110 may input messages 221 and 222 (or prompts generated based on message 222) into a model to obtain image 232 generated by the model. As an example, such a model may include any suitable image generation model, such as a generative model, which may output an image satisfying the prompt based on the input image and the prompt. As an example, such a model may be deployed in electronic device 110 and / or server 130. As an example, such a model may also be deployed in other terminals or servers. Furthermore, electronic device 110 may invoke such a model through a communication connection with other terminals or servers. Thus, embodiments of this disclosure can improve the quality of image 232 and image generation efficiency.

[0045] As an example, user 140 can confirm image 232 to use it as a reference image. In some examples, user 140 may also trigger a first regeneration request associated with image 232 to be sent to the virtual object. For example, user 140 may trigger a first regeneration request to be sent to the virtual object by sending messages such as "Adjust image XXX" or "Adjust image XX by method XXX".

[0046] Furthermore, the electronic device 110 can present at least one updated image generated based on such a first regeneration request. Thus, the user 140 can confirm a second image among such at least one updated image to identify the confirmed second image as a reference image. Alternatively or additionally, the user 140 can also trigger a regeneration request associated with such at least one updated image to further trigger the regeneration of images, and so on; embodiments of this disclosure do not exhaustively describe this.

[0047] In some embodiments, the virtual object can provide user 140 with more information for generating the video, further assisting the user in efficiently adjusting the video generation information. For example, such first response content may also include at least one candidate cue for generating the video, thereby the determined video generation information may also include a target cue determined based on such at least one candidate cue.

[0048] For example, according to Figure 2B The interface 200B shown may also include a prompt 234 (also known as a candidate prompt) in message 222. Such a prompt 234 can indicate the video effect of the video content to be generated. Thus, the user 140 can quickly understand the direction of video content generation, thereby enabling rapid adjustment of the generated video content's effect.

[0049] Furthermore, user 140 can confirm such a prompt 234 to designate it as the target prompt. In some examples, prompt 234 may indicate "make the ball move up and down". Thus, user 140 can confirm such a "make the ball move up and down" video effect to trigger the generation of video content from the virtual object. As an example, user 140 can trigger the confirmation of the video effect by sending a confirmation message. In some examples, user 140 may also refrain from interaction as a confirmation action. For example, if no interaction is received from user 140 within a preset time period after message 231 is presented, electronic device 110 can confirm the generation of a video corresponding to the "make the ball move up and down" video effect.

[0050] Alternatively, user 140 can also trigger the editing of such a prompt item 234 to determine the edited prompt item 234 as the target prompt item. For example, user 140 can trigger the sending of a message instructing "move the ball left and right" to determine "move the ball left and right" as the target prompt item. As another example, user 140 can trigger the sending of messages instructing "move the ball forward and backward" to determine "move the ball forward and backward" as the target prompt item.

[0051] Alternatively or additionally, if the first response content includes an image, the first response content may also include multiple candidate prompts. Then, user 140 can select a first prompt from these multiple candidate prompts to trigger the selection of the first prompt as the target prompt. Alternatively, user 140 can also trigger the editing of a second prompt from these multiple candidate prompts to trigger the editing of the second prompt as the target prompt.

[0052] In some embodiments, user 140 may also trigger a second regeneration request associated with at least one such candidate prompt. Then, electronic device 110 may present at least one updated prompt generated based on the second regeneration request. As an example, user 140 may trigger a second regeneration request associated with prompt 234; for instance, the user may trigger the sending of messages instructing "readjust the ball's movement," "make the ball move in a different way," etc. Thus, electronic device 110 may present corresponding response content from the virtual object to present at least one updated prompt generated based on messages such as "readjust the ball's movement," "make the ball move in a different way," etc.

[0053] Therefore, user 140 can confirm a third prompt among at least one such update prompt to designate the confirmed second prompt as a reference prompt. Alternatively or additionally, user 140 can also trigger a regeneration request associated with at least one such update prompt to further trigger the regeneration of the prompt, and so on, as described in the embodiments of this disclosure. Based on this approach, embodiments of this disclosure can support users in flexibly and efficiently adjusting the candidate images and / or candidate prompts included in the first response content, thereby improving the quality of video generation and enhancing user interaction efficiency.

[0054] In some embodiments, to improve the interaction efficiency between the virtual object and the user 140, the electronic device 110 may provide a message from the virtual object before generating the image 232, and present a placeholder element indicating the image generation status in such a message. For example, according to Figure 2C The interface 200C shown allows the electronic device 110 to display a placeholder element 233 to indicate an image generation status of "image generation in progress" and to display the progress of image generation (e.g., in percentage form). Thus, embodiments of this disclosure can support interaction between the user and virtual objects before image 232 is generated.

[0055] In some examples, according to Figure 2D In the interface 200D shown, user 140 can trigger the sending of message 223 (e.g., "Generate video") to trigger confirmation of generating a video effect corresponding to "making the ball move up and down" based on image 232. Thus, electronic device 110 can provide message 235 (also known as, second response content) from the virtual object, which may include video content generated based on video generation information.

[0056] In some examples, the electronic device 110 may retain image 232 in message 235 until video generation is complete. Then, once video generation is complete, the electronic device 110 can replace image 232 in such message 235 with the corresponding video. Based on this approach, embodiments of this disclosure can help user 140 more conveniently adjust the video effects of the video to be generated.

[0057] In some embodiments, during video generation, user 140 can continue to interact with the virtual object. Further, upon completion of video generation, electronic device 110 can provide a notification message from the virtual object to indicate that the video has been successfully generated. As an example, according to... Figure 2E As shown in the interface 200E, the electronic device 110 can provide a reminder message 236 from the virtual object, such reminder message 236 can indicate that the task of generating video based on image 232 has been completed.

[0058] In some examples, such a reminder message 236 may include a reference 237 to message 222 (e.g., a reference to "turn the image into multiple small balls and make it move"). Such a reference 237 helps user 140 understand the reminder message 236 more efficiently. Furthermore, such a reminder message 236 may include a viewing entry 238 (e.g., "view generated content"). User 140 can trigger the presentation of the generated video content by triggering the viewing entry 238. Thus, embodiments of this disclosure enable users to quickly view the generated video content through the viewing entry in the reminder message during interaction with virtual objects, thereby improving the efficiency of interface interaction between the user and the interactive interface.

[0059] As an example, according to Figure 2F In the interface 200F shown, user 140 can trigger the viewing entry 238 to slide a message in the session interface to the displayed message 235. For example, electronic device 110 can slide such message 235 to a preset position displayed in the session interface. In some examples, message 235 may include generated video content 240. Such video content 240 may be generated based on image 232 and has a "ball moving up and down" video effect. In some examples, user 140 can trigger the playback of such video content 240 in the interactive interface by triggering the playback control 241, thereby improving the efficiency of interface interaction.

[0060] In some examples, such video content 240 may be generated using a model. For example, electronic device 110 may input image 232 and cue item 234 (or other cue items generated based on cue item 234) into a model to obtain video content 240 generated by the model. As an example, such a model may include any suitable image generation model, such as a generative model, which may output a video that satisfies the cue item based on the input image and cue item. As an example, such a model may be deployed in electronic device 110 and / or server 130. As an example, such a model may also be deployed in other terminals or servers. Furthermore, electronic device 110 may invoke such a model through a communication connection with other terminals or servers. Thus, embodiments of this disclosure can improve the quality of video content 240.

[0061] In some embodiments, based on messages 221 and 222, the electronic device 110 can provide multiple candidate images from virtual objects to enhance interactivity. The following will combine... Figure 2G To describe at least one such candidate image includes multiple specific implementations.

[0062] As an example, according to Figure 2G The interface 200G shown allows the electronic device 110 to provide messages 251 from virtual objects. These messages 251 can be response messages from the virtual object to messages 221 and 222. For example... Figure 2G As shown, such a message 251 may include multiple generated candidate images (e.g., image 252-1, image 252-2, etc., which may be individually or collectively referred to as image 252). As an example, the generation process of such image 252 can be found in the generation process of image 232 described above, and will not be repeated here. It should be understood that this disclosure is not intended to limit the number of multiple generated images.

[0063] As an example, image 252-1 could be an image with three balls added to message 221, and image 252-2 could be an image with two balls added to message 221. Furthermore, user 140 can select a target image from a plurality of images 252 to trigger the generation of a video corresponding to the target image. In some examples, electronic device 110 can generate a corresponding video based on a specified image (e.g., image 251-1, etc.) even if user 140 does not interact within a specified time period (e.g., within a specified duration of presenting message 251). Based on this approach, embodiments of this disclosure can improve the richness of video generation and can further improve the quality of video generation.

[0064] In some embodiments, user 140 may select one or more images from such images 252 as one or more reference images for video generation. Furthermore, such one or more reference images may be used to determine one or more video frames of video content. For example, user 140 may trigger the determination of image 252-1 as a reference image for user video generation. Furthermore, electronic device 110 may provide video content from a virtual object with image 252-1 as one video frame. For example, user 140 may trigger the determination of images 252-1 and 252-2 as multiple reference images for user video generation. Furthermore, electronic device 110 may provide video content from a virtual object with images 252-1 and 252-2 as video frames. Thus, embodiments of this disclosure can improve the quality of video generation.

[0065] In some embodiments, different images 252 can be associated with different cue items to be associated with different video effects. For example, such as Figure 2GAs shown, image 252-1 can be associated with cue 253-1 "the ball moves up and down," while image 252-2 can be associated with cue 253-2 "the ball moves left and right." In some examples, electronic device 110 can generate video based on image 252-1 and cue 253-1. In other examples, video is generated based on image 252-2 and cue 253-2.

[0066] In some examples, electronic device 110 can adjust such a video generation process based on messages from user 140. For example, user 140 can trigger video generation based on image 252-1 and prompt 253-2. For example, user 140 can also trigger video generation based on image 252-2 and prompt 253-1. For example, user 140 can trigger the sending of adjustment messages to customize such a video generation process. For instance, user 140 can send the message "Add five balls" to instruct the regeneration of image 252. As another example, user 140 can send the message "Move the balls in the first image back and forth" to trigger video generation based on image 252-1 and the prompt "Move the balls back and forth." Thus, embodiments of this disclosure can further enhance the richness of generated videos.

[0067] In some embodiments, the electronic device 110 may provide multiple videos from virtual objects based on messages from the user 140, thereby enabling embodiments of this disclosure to further improve video generation quality. The following will be combined with... Figure 2H To be continued Figure 2J The following describes some embodiments of this disclosure.

[0068] For example, according to Figure 2H As shown in interface 200H, electronic device 110 can present message 224 from user 140 (e.g., "Show the scene from author 1's poem 'XXX', and provide a corresponding video for each line of the poem"). Based on this, electronic device 110 can provide a video corresponding to each line of the poem 'XXX'. As an example, the user's input may include message 224. In some examples, the user's input may also include extended information determined based on message 224. For example, the user's input may also include the full text of the poem 'XXX'.

[0069] For example, the poem "XXX" could include "aaaaa", "bbbbbb", "ccccc", "ddddd". Furthermore, according to... Figure 2IAs shown in the interface 200I, the electronic device 110 can present a message 261 (also known as, first response content) from a virtual object. As an example, message 261 may include a first set of images generated based on "aaaaa", a second set of images generated based on "bbbbb", and so on. Thus, the electronic device 110 can determine first video generation information based on the first set of images to generate first video content (e.g., video A) corresponding to "aaaaa", such video A may include at least a first reference image for video generation (e.g., an image determined by the user in the first set of images or an image that the user triggers to regenerate and confirm, etc.).

[0070] Alternatively or additionally, the electronic device 110 may determine second video generation information based on the second set of images for generating second video content (e.g., video B) corresponding to “bbbbb”, such video B may include at least a second reference image for video generation (e.g., an image determined by the user in the second set of images or an image that the user triggers to regenerate and confirm, etc.).

[0071] For ease of explanation, the embodiments of this disclosure are illustrated by taking the example of electronic device 110 message 261 including an image generated based on each line of the poem "XXX". Specific implementations of multiple images corresponding to each line of the poem can be found in the embodiments of this disclosure, and will not be elaborated upon here.

[0072] Based on this, according to Figure 2H The interface 200H shown may include a message 261 that includes an image 262-1 corresponding to “aaaaa”, an image 262-2 corresponding to “bbbbb”, etc. Furthermore, the electronic device 110 may provide a video generated based on image 262-1 (e.g., video A), a video generated based on image 262-2 (e.g., video B), etc.

[0073] As an example, the generation process of such image 262 can be found in the detailed description of the generation process of image 232 described above. Alternatively or additionally, the generation process of such videos A, B, etc., can be found in the detailed description of the generation process of video content 240 described above. Embodiments of this disclosure will not be repeated here.

[0074] Therefore, the embodiments of this disclosure can enhance the richness of interaction with virtual objects.

[0075] In some embodiments, the electronic device 110 can merge videos such as Video A, Video B, etc., corresponding to each line of the poem "XXX" to obtain a merged video corresponding to the poem "XXX". For example, the electronic device 110 can determine second video generation information based on message 261, and such second video generation information can also be used to generate a merged video corresponding to the poem "XXX". Such a merged video may at least include the first video content (e.g., Video A or Video B, etc.).

[0076] As an example, such video synthesis can be achieved using any suitable video generation tool; for instance, videos A, B, etc., can be spliced ​​together to obtain such a merged video. Alternatively, electronic device 110 can input videos A, B, etc., into a video processing model (e.g., a generative model, which can, for example, synthesize multiple input videos) to obtain the merged video output by the video processing model.

[0077] In some embodiments, such merged videos can also be provided based on a user's merge request. Specifically, electronic device 110 can trigger the merging of videos corresponding to each line of the poem "XXX" based on a merge request from user 140. For example, user 140 can trigger the sending of messages indicating "merge videos" or "generate complete video" to initiate such video merging.

[0078] As an example, according to Figure 2J The interface 200J shown allows the electronic device 110 to be associated with message 261, providing multiple interactive items (e.g., interactive item 281-1, interactive item 281-2, interactive item 281-3, interactive item 281-4, etc., which can be individually or collectively referred to as interactive item 281) to support convenient interaction between user 140 and the virtual object. As an example, such interactive item 281 can indicate a request to merge multiple videos in message 261. For example, user 140 can select interactive item 281-1 (e.g., "Generate complete video") to trigger a request to send merged videos to the virtual object. Thus, embodiments of this disclosure can further enhance the richness of video generation.

[0079] It should be understood that the specific content items mentioned in the embodiments of this disclosure (e.g., the specific content of an image, the specific content of a video, the specific content of a prompt, the specific content of a message, the number of specific objects, the image number, etc.) are all exemplary only, and this disclosure is not intended to limit them.

[0080] Based on this approach, embodiments of this disclosure can provide at least one candidate image generated based on user input before providing video content from virtual objects. This allows for improved quality of video generation information used for video generation through user interaction with the at least one candidate image. Consequently, embodiments of this disclosure can generate higher-quality video content based on higher-quality video generation information.

[0081] Therefore, embodiments of this disclosure can present at least one candidate image generated based on input content to the user during the video generation process, thereby improving the quality of video generation based on the user's interaction with at least one candidate image.

[0082] Example process

[0083] Figure 3 A flowchart illustrating an example of a process 300 for generating video content according to some embodiments of the present disclosure is shown. Process 300 can be implemented at an electronic device 110. Reference is made below. Figure 1 Describe the process 300.

[0084] like Figure 3 As shown in box 310, electronic device 110 obtains user input through an interactive interface with a virtual object.

[0085] In box 320, electronic device 110 presents first response content from a virtual object in an interactive interface, the first response content including at least one candidate image generated based on the input content.

[0086] In box 330, electronic device 110 determines video generation information based on an interactive operation associated with the first response content, the video generation information including at least a reference image for generating the video.

[0087] In box 340, electronic device 110 provides a second response content from a virtual object, the second response content including video content generated based on video generation information.

[0088] In some embodiments, determining video generation information based on an interaction operation associated with the first response content includes: in response to receiving a selection of a first image among at least one candidate image, determining the first image as a reference image.

[0089] In this way, embodiments of the present disclosure can support users to flexibly select at least one candidate image included in the first response content to trigger video generation, thereby improving the quality of video generation.

[0090] In some embodiments, determining video generation information based on an interactive operation associated with the first response content includes: receiving a first regeneration request associated with at least one candidate image; presenting at least one updated image generated based on the first regeneration request; and, in response to receiving confirmation of a second image in the at least one updated image, determining the second image as a reference image.

[0091] In this way, embodiments of this disclosure can support user-triggered image regeneration, thereby improving the quality of the images referenced for video generation. Furthermore, embodiments of this disclosure can improve video generation quality by allowing the user to trigger image regeneration instead of triggering video regeneration.

[0092] In some embodiments, the first response content further includes at least one candidate cue for generating a video, and the video generation information further includes a target cue determined based on the at least one candidate cue.

[0093] In this way, embodiments of the present disclosure can help users learn more about video generation before the video is generated by presenting at least one candidate prompt for video generation, thereby further improving the quality of video generation.

[0094] In some embodiments, process 300 further includes: determining the first prompt as the target prompt in response to receiving a selection of a first prompt among at least one candidate prompt; or determining an edited second prompt as the target prompt based on an editing operation on a second prompt among at least one candidate prompt.

[0095] In this way, embodiments of the present disclosure can help users efficiently adjust the video effects generated by the video through user interaction with at least one such candidate prompt, and further improve the quality of the generated video.

[0096] In some embodiments, process 300 further includes: receiving a second regeneration request associated with at least one candidate prompt; presenting at least one updated prompt generated based on the second regeneration request; and, in response to receiving confirmation of a third prompt among the at least one updated prompt, identifying the third prompt as the target prompt.

[0097] In this way, embodiments of the present disclosure can improve the quality of the prompts referenced in video generation by supporting users to regenerate prompts for video generation, thereby improving the quality of video generation.

[0098] In some embodiments, the video generation information includes a plurality of reference images determined based on the first response content, and the plurality of reference images are used to determine a plurality of video frames in the video content.

[0099] In this way, embodiments of the present disclosure can further constrain the direction of video generation by referring to multiple reference images determined based on the first response content during the video generation process, thereby further improving the quality of video generation.

[0100] In some embodiments, at least one candidate image is a first set of images generated based on a first portion of the input content, and process 300 further includes: presenting a second set of images generated based on a second portion of the input content in a first response content.

[0101] In this way, embodiments of the present disclosure can enable users to provide more images with fewer interactions with virtual objects, thereby improving the efficiency of user-virtual object interaction.

[0102] In some embodiments, the video generation information is first video generation information, the video content is first video content, and the reference image is first reference image. The process 300 further includes: determining second video generation information based on the first response content, the second video generation information including at least a second reference image for video generation; and providing second video content generated based on the second video generation information.

[0103] In this way, embodiments of the present disclosure can further improve the interaction efficiency between users and virtual objects by supporting the provision of multiple video contents in the first response content.

[0104] In some embodiments, providing second video content generated based on second video generation information includes: providing second video content independent of first video content; or providing first video content, wherein the second video content is a video segment of the first video content.

[0105] In this way, embodiments of this disclosure can support the flexible provision of multiple independent or merged video contents corresponding to user input, thereby improving the richness of video generation.

[0106] In some embodiments, providing second response content from a virtual object includes: presenting a notification message in an interactive interface in response to the generation of video content, the notification message including an entry point for viewing the video content; and presenting the video content in response to triggering the viewing entry point.

[0107] In this way, embodiments of this disclosure enable users to quickly view generated video content through a viewing entry in a notification message during interaction with virtual objects, thereby improving the efficiency of user-interactive interface interaction.

[0108] Example devices and equipment

[0109] Embodiments of this disclosure also provide corresponding apparatus for implementing the above methods or processes. Figure 4 A schematic structural block diagram of an example apparatus 400 for generating video content according to certain embodiments of the present disclosure is shown. Apparatus 400 may be implemented as or included in electronic device 110. Various modules / components in apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0110] like Figure 4 As shown, the device 400 includes an acquisition module 410 configured to acquire user input via an interactive interface with a virtual object; a first presentation module 420 configured to present first response content from the virtual object in the interactive interface, the first response content including at least one candidate image generated based on the input content; a first determination module 430 configured to determine video generation information based on an interactive operation associated with the first response content, the video generation information including at least a reference image for generating a video; and a providing module 440 configured to provide second response content from the virtual object, the second response content including video content generated based on the video generation information.

[0111] In some embodiments, the first determining module 430 is further configured to: determine the first image as a reference image in response to receiving a selection of a first image among at least one candidate image.

[0112] In some embodiments, the first determining module 430 is further configured to: receive a first regeneration request associated with at least one candidate image; present at least one updated image generated based on the first regeneration request; and determine the second image as a reference image in response to receiving confirmation of a second image in the at least one updated image.

[0113] In some embodiments, the first response content further includes at least one candidate cue for generating a video, and the video generation information further includes a target cue determined based on the at least one candidate cue.

[0114] In some embodiments, the apparatus 400 further includes a second determining module configured to: determine the first prompt as a target prompt in response to receiving a selection of a first prompt among at least one candidate prompt; or determine an edited second prompt as a target prompt based on an editing operation of a second prompt among at least one candidate prompt.

[0115] In some embodiments, the apparatus 400 further includes a third determining module configured to: receive a second regeneration request associated with at least one candidate prompt; present at least one updated prompt generated based on the second regeneration request; and determine the third prompt as the target prompt in response to receiving confirmation of a third prompt among the at least one updated prompt.

[0116] In some embodiments, the video generation information includes a plurality of reference images determined based on the first response content, and the plurality of reference images are used to determine a plurality of video frames in the video content.

[0117] In some embodiments, at least one candidate image is a first set of images generated based on a first portion of the input content, and the apparatus 400 further includes a second presentation module configured to: present a second set of images generated based on a second portion of the input content in the first response content.

[0118] In some embodiments, the video generation information is first video generation information, the video content is first video content, and the reference image is first reference image. The apparatus 400 further includes a fourth determining module configured to: determine second video generation information based on the first response content, the second video generation information including at least a second reference image for video generation; and provide second video content generated based on the second video generation information.

[0119] In some embodiments, the second presentation module is further configured to: provide second video content independent of the first video content; or provide the first video content, wherein the second video content is a video segment of the first video content.

[0120] In some embodiments, the providing module 440 is further configured to: in response to the generation of video content, present a reminder message in the interactive interface, the reminder message including an entry point for viewing the video content; and in response to triggering the viewing entry point, present the video content.

[0121] The modules included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the modules in device 400 can be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASHCs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0122] Figure 5A block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. Figure 5 The electronic device 500 shown can be used to achieve Figure 1 Electronic devices 110 or Figure 4 Device 400.

[0123] like Figure 5 As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, at least one processor 510 or processing unit, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processor 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0124] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data and can be accessed within electronic device 500.

[0125] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0126] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0127] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0128] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the methods described above.

[0129] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0130] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0131] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0132] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0133] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for generating video content, comprising: User input is obtained through an interactive interface with a virtual object, and the input at least indicates a request to generate a video; In the interactive interface, a first response content from the virtual object is presented, the first response content including at least one candidate image generated based on the input content; Based on the interactive operation associated with the first response content, video generation information is obtained, the video generation information including at least a reference image for generating the video, the reference image being obtained based on the at least one candidate image; as well as A second response content is provided from the virtual object, the second response content including video content generated based on the video generation information.

2. The method of claim 1, wherein determining the video generation information based on the interaction operation associated with the first response content includes: In response to receiving a selection of a first image among the at least one candidate image, the first image is determined as the reference image.

3. The method of claim 1, wherein determining the video generation information based on the interaction operation associated with the first response content includes: Receive a first regeneration request associated with the at least one candidate image; Present at least one updated image generated based on the first regeneration request; as well as In response to receiving confirmation of the second image in the at least one updated image, the second image is determined as the reference image.

4. The method of claim 1, wherein the first response content further includes at least one candidate cue for generating a video, and the video generation information further includes a target cue determined based on the at least one candidate cue.

5. The method according to claim 4, further comprising: In response to receiving a selection of a first prompt among the at least one candidate prompts, the first prompt is determined as the target prompt. or Based on the editing operation of the second prompt item among the at least one candidate prompt items, the edited second prompt item is determined as the target prompt item.

6. The method according to claim 5, further comprising: Receive a second regeneration request associated with the at least one candidate suggestion; Present at least one update prompt item generated based on the second regeneration request; as well as In response to receiving confirmation of a third prompt among the at least one update prompt, the third prompt is identified as the target prompt.

7. The method of claim 1, wherein the video generation information includes a plurality of reference images determined based on the first response content, and the plurality of reference images are used to determine a plurality of video frames in the video content.

8. The method of claim 1, wherein the at least one candidate image is a first set of images generated based on a first portion of the input content, the method further comprising: In the first response content, a second set of images generated based on a second part of the input content is presented.

9. The method according to claim 8, wherein the video generation information is first video generation information, the video content is first video content, and the reference image is a first reference image, the method further comprising: Based on the first response content, second video generation information is determined, and the second video generation information includes at least a second reference image for video generation. as well as Provide second video content generated based on the second video generation information.

10. The method of claim 9, wherein providing the second video content generated based on the second video generation information comprises: Provide second video content that is independent of the first video content; or The first video content is provided, and the second video content is a video segment from the first video content.

11. The method of claim 1, wherein providing the second response content from the virtual object comprises: In response to the generation of the video content, a reminder message is displayed in the interactive interface, the reminder message including an entry point for viewing the video content; as well as In response to the triggering of the viewing entry, the video content is displayed.

12. An apparatus for generating video content, comprising: The acquisition module is configured to acquire user input via an interactive interface with a virtual object, the input indicating at least a request to generate video; A first presentation module is configured to present first response content from the virtual object in the interactive interface, the first response content including at least one candidate image generated based on the input content; A first determining module is configured to obtain video generation information based on an interactive operation associated with the first response content. The video generation information includes at least a reference image for generating the video, the reference image being obtained based on the at least one candidate image. as well as A providing module is configured to provide second response content from the virtual object, the second response content including video content generated based on the video generation information.

13. An electronic device, comprising: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 11 when executed by the at least one processor.

14. A computer-readable storage medium having stored thereon computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 11.

15. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 11.

Images