A video generation method, an electronic device, a readable storage medium and a chip

Abstract

This application provides a video generation method, an electronic device, a readable storage medium, and a chip. The method, upon receiving a user instruction, saves tasks generated based on the user instruction to a task queue. By executing tasks in the task queue, a video matching the user instruction is generated; thus, this application provides a novel method for video generation. Furthermore, the execution order of tasks in the task queue follows the order in which the user instructions associated with those tasks are received. This ensures that the electronic device executes the task corresponding to the first received user instruction in the task queue after completing the task corresponding to that first received user instruction, thereby generating a video that meets the user's requirements.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to a video generation method, an electronic device, a readable storage medium, and a chip. Background Technology

[0002] Currently, many electronic devices are equipped with cameras, allowing users to take photos and videos. Alternatively, electronic devices can acquire images or videos from the internet or other devices. With the continuous development of smart terminals, users' demands and experiences regarding images and videos are constantly increasing. In certain scenarios, users need to generate videos on related topics based on images or videos to facilitate viewing the content. Summary of the Invention

[0003] This application provides a video generation method, an electronic device, a readable storage medium, and a chip that can generate videos that meet user needs and improve user experience.

[0004] Firstly, a video generation method is provided, the method comprising:

[0005] In response to a first user instruction, a first task is generated, wherein the first task is used to instruct the search for a first source material; the first task is saved to a task queue, and the execution order of tasks in the task queue is according to the order in which the user instructions associated with the received tasks are executed; if the first task group number of the first task does not exist in the task queue, the first task is executed, and the first task group number is used to distinguish the first task; if the first task is completed, in response to a second user instruction, a second task is generated, wherein the second task is used to instruct the generation of a first video based on the first source material; the second task is saved to a task queue; if the second task group number of the second task does not exist in the task queue, the second task is executed to obtain the first video, and the second task group number is used to distinguish the second task.

[0006] In the above technical solution, after receiving a user instruction (e.g., a first user instruction), the electronic device saves the task generated based on the user instruction (e.g., a first task) to a task queue. Then, by executing the tasks in the task queue, a video (e.g., a first video) matching the user instruction is generated. Thus, this application provides a novel method for generating videos. Furthermore, the execution order of the tasks in the task queue follows the order in which the user instructions associated with the task are received. This ensures that the electronic device executes multiple tasks corresponding to multiple user instructions in the task queue in the order in which the user inputs the user instructions. It also ensures that the electronic device executes the task corresponding to the first received user instruction in the task queue after completing the task corresponding to that first received user instruction in the task queue, thereby generating a video that meets the user's needs and improving the user experience.

[0007] Furthermore, before executing a task in the task queue, the electronic device needs to check whether the task group number is occupied (for example, check whether the first task group number of the first task exists in the task queue, and check whether the second task group number of the second task exists in the task queue). If the task group number is not occupied (i.e., if the first task group number of the first task does not exist in the task queue, or if the second task group number of the second task does not exist in the task queue), the task in the task queue is executed, which can ensure that the tasks in the task queue are executed normally.

[0008] In conjunction with the first aspect, in certain implementations of the first aspect, in response to a first user instruction, a first task is generated, including:

[0009] In response to a first user instruction, a first command is generated, wherein the first command carries a first session identifier and first semantic information corresponding to the first user instruction; based on the first command, a first task is generated.

[0010] In this embodiment of the application, the electronic device can generate a first command based on a first user instruction, and generate a first task based on the first command, which can quickly convert user instructions into tasks.

[0011] In conjunction with the first aspect, in certain implementations of the first aspect, a first task is generated based on a first command, including:

[0012] Based on the first command, the system creates first task input data, first task configuration data, first result notifier, and first task sequence number for the first task. The first task input data includes a first session identifier, first semantic information, and a first task identifier, obtained from the first command. The first configuration data includes a first task name, a first task group number, a first timeout threshold, and the address of a first snapshot cache, all obtained from the first command. The first result notifier records the execution result of the first task, and the first task sequence number is obtained from the first command. The system then generates the first task based on the first task input data, first task configuration data, first result notifier, and first task sequence number.

[0013] In this embodiment of the application, the electronic device can create first task input data, first task configuration data, first result notification device, and first task sequence number for the first task. Then, based on the first task input data, first task configuration data, first result notification device, and first task sequence number, the first task is generated. Moreover, when generating other tasks (such as the second task), they can also be generated in the above manner, allowing for flexible configuration of different tasks.

[0014] In conjunction with the first aspect, in some implementations of the first aspect, after executing the second task to obtain the first video when it is detected that the second task group number of the second task does not exist in the task queue, the method further includes:

[0015] If the second task is stopped, it is determined that the first execution result of the second task is invalid. The first execution result is used to indicate the result of the second task being completed. The first execution result includes the first video. The first video is rolled back to obtain the second execution result of the second task. The rollback process is used to indicate that the first execution result is restored to the state before the second task was executed.

[0016] In this embodiment of the application, when the second task is detected to be stopped, the electronic device can execute the second task normally, and then roll back the first video to obtain the second execution result of the second task. In this way, when a situation arises that requires stopping the task during the execution of the task, the electronic device can follow the original task execution process without affecting the original task execution process.

[0017] In conjunction with the first aspect, some implementations of the first aspect also include the following methods:

[0018] Display a first interface, which includes a stop control; in response to an operation on the stop control, generate a stop command for the second task; based on the stop command for the second task, detect that the second task has been stopped.

[0019] In conjunction with the first aspect, in some implementations of the first aspect, after saving the second task to the task queue, the method further includes:

[0020] During the execution of the second task, a third user instruction is received, and a third task is generated based on the third user instruction. The third task is used to instruct the editing of the first video. The third task is saved to the task queue. If the third task group number of the third task is detected in the task queue, the third task in the task queue is cleared. The second interface is displayed to indicate that the third task is occupied.

[0021] In this embodiment of the application, when a third task group number of a third task is detected in the task queue, the third task in the task queue is cleaned up, which can simplify the number of tasks in the task queue.

[0022] In conjunction with the first aspect, some implementations of the first aspect also include the following methods:

[0023] If the third task group number is determined to be the same as the second task group number, the third task group number of the third task is detected in the task queue.

[0024] In conjunction with the first aspect, in some implementations of the first aspect, after saving the second task to the task queue, the method further includes:

[0025] If the second task is detected to be interrupted, the second task in the task queue is cleaned up.

[0026] Display a third interface, which includes a first pop-up window; in response to an operation on the first pop-up window, generate a second command, which instructs the interrupted second task to be re-issued; save the second task to the task queue.

[0027] In conjunction with the first aspect, in some implementations of the first aspect, the method is applied to an electronic device including a task queue module and an editing module, the editing module being used to perform a second task and save the second task to the task queue, including:

[0028] The second task is saved to the task queue in the task queue module; after the second task is saved to the task queue module, the method further includes: detecting that the second task is interrupted if the task queue module fails to call the clip module.

[0029] In conjunction with the first aspect, in some implementations of the first aspect, after executing the second task to obtain the first video when it is detected that the second task group number of the second task does not exist in the task queue, the method further includes:

[0030] Upon receiving a third user instruction, a third task is generated based on the instruction. The third task is used to instruct the editing of the first video. The third task is saved to the task queue. If the execution duration of the third task is detected to be greater than or equal to the second timeout threshold, the third task in the task queue is cleaned up and an exception log is generated.

[0031] In conjunction with the first aspect, in some implementations of the first aspect, the method is applied to an electronic device including a task queue module and a clipping module to perform a second task, including:

[0032] The task queue module sends an instruction to the editing module to execute the second task; the editing module executes the second task based on the instruction.

[0033] In conjunction with the first aspect, in some implementations of the first aspect, the method is applied to an electronic device including a session state module and a snapshot cache module. The session state module is used to save the first material and the first video, and the snapshot cache module is used to perform snapshot processing on the first material to obtain snapshot information of the first material, and to perform snapshot processing on the first video to obtain snapshot information of the first video.

[0034] In a second aspect, an electronic device is provided for performing the method provided in the first aspect. Specifically, the electronic device may include a processing unit for performing any possible implementation of the first aspect.

[0035] Thirdly, an electronic device is provided, comprising: one or more processors; one or more memories; wherein the one or more memories store one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform any possible implementation of the method in the first aspect described above.

[0036] Fourthly, a computer-readable storage medium is provided, including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect.

[0037] Fifthly, a chip is provided, including a memory for storing instructions; and a processor for retrieving and executing the instructions from the memory, causing an electronic device on which the chip is mounted to perform the method described in the first aspect above. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of this application.

[0039] Figure 2 This is a schematic diagram of the software system of the electronic device 100 according to an embodiment of this application.

[0040] Figure 3 This is a set of graphical user interfaces provided in the embodiments of this application.

[0041] Figure 4 This is another set of graphical user interfaces provided in the embodiments of this application.

[0042] Figure 5 This is another set of graphical user interfaces provided in the embodiments of this application.

[0043] Figure 6 This is another set of graphical user interfaces provided in the embodiments of this application.

[0044] Figure 7 This is another set of graphical user interfaces provided in the embodiments of this application.

[0045] Figure 8 This is another set of graphical user interfaces provided in the embodiments of this application.

[0046] Figure 9 This is another set of graphical user interfaces provided in the embodiments of this application.

[0047] Figure 10 This is a timing diagram of a material search task creation method provided in an embodiment of this application.

[0048] Figure 11 This is a timing diagram of an embodiment of the present application for performing a material search task.

[0049] Figure 12 This is a timing diagram of an embodiment of the present application for performing a material search task.

[0050] Figure 13 This is a timing diagram of an embodiment of the present application for performing a material search task.

[0051] Figure 14 This is a timing diagram showing how an electronic device stops performing a material search task, as provided in an embodiment of this application.

[0052] Figure 15 This is a timing diagram of an electronic device performing a video saving task, as provided in an embodiment of this application.

[0053] Figure 16 This is a timing diagram of an electronic device performing a material search task, as provided in an embodiment of this application.

[0054] Figure 17 This is a timing diagram of another electronic device performing a video generation task, as provided in an embodiment of this application.

[0055] Figure 18 This is a schematic flowchart illustrating a video generation method provided in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "plural" or "multiple" refers to two or more than two.

[0057] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0058] The video generation method provided in this application is applied to electronic devices. Electronic devices can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Electronic devices can be mobile phones, smart TVs, wearable devices, tablets, computers with wireless transceiver capabilities, virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technologies or device forms used in the electronic devices.

[0059] Figure 1 This is a schematic diagram of the structure of an electronic device 100 (taking a mobile phone as an example) provided in an embodiment of this application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, an antenna 1, an antenna 2, a mobile communication module 140, a wireless communication module 150, an audio module 160, a sensor module 170, a camera 181, a display screen 182, etc. The sensor module 170 may include a pressure sensor 170A, a gyroscope sensor 170B, etc.

[0060] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0061] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0062] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0063] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0064] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0065] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 140, wireless communication module 150, modem processor, and baseband processor.

[0066] Electronic device 100 implements display functions through a GPU, a display screen 182, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 182 and the application processor. The GPU performs mathematical and geometric calculations and is used for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0067] Display screen 182 is used to display images, videos, etc. Display screen 182 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Mini LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 182, where N is a positive integer greater than 1.

[0068] Electronic device 100 can perform shooting functions through ISP, camera 181, video codec, GPU, display screen 182 and application processor.

[0069] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.

[0070] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121.

[0071] Electronic device 100 can implement audio functions through audio module 160 and application processor, such as music playback and recording.

[0072] The pressure sensor 170A is used to sense pressure signals and can convert pressure signals into electrical signals.

[0073] The gyroscope sensor 170B can be used to determine the motion attitude of the electronic device 100.

[0074] This concludes the introduction to the hardware structure of electronic device 100. It is understandable that... Figure 1 The components included in the hardware structure shown do not constitute a specific limitation on the electronic device 100. The electronic device 100 may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0075] In addition, an operating system runs on top of these components. Examples include Apple's iOS, Google's Android, and Microsoft's Windows. Applications can be installed and run on this operating system.

[0076] The operating system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software system of electronic device 100.

[0077] Figure 2This is a schematic diagram of the software system of an electronic device 100 according to an embodiment of this application. The software system includes several layers, each with a clear role and division of labor, and the layers communicate with each other through software interfaces. In some embodiments, the software system is divided from top to bottom into an application layer, a media middleware framework layer, an application framework layer, a system library, and a hardware layer.

[0078] The application layer can include a series of application packages, such as camera, gallery, calendar, map, navigation, language model, editing application, YOYO Assistant, etc. When these application packages are run, they can access the various service modules provided by the media platform framework layer through the application programming interface (API) and execute the corresponding business logic.

[0079] For example, YOYO Assistant can receive user commands, call a voice model to analyze the user commands, the voice model returns the analysis results to YOYO Assistant, and YOYO Assistant calls service modules in the media platform framework layer, such as the editing module, to perform editing tasks based on the analysis results.

[0080] The media middle platform framework layer is a software layer positioned between the application layer and the application framework layer. This layer can include modules such as authoring, task configuration, tasks, task queues, result notification, timeout detector, session state, snapshot caching, task input, task output, and editing.

[0081] The creation module is used to create tasks related to video generation and publish these tasks to the task queue module. Tasks related to video generation can include material search tasks, video generation tasks, character confirmation tasks, etc.

[0082] The task configuration module is used to create task configuration data, which includes the task name, the task group number to which the task belongs, and the preset timeout duration of the task.

[0083] The task input module is used to create task input data, which includes task identifiers, session identifiers, and semantic information. For the specific meanings of task identifiers, session identifiers, and voice information, please refer to other embodiments, which will not be repeated here.

[0084] The snapshot caching module is used to create snapshot caches of media footage. For a detailed explanation of the specific meaning of snapshot caching for media footage, please refer to other embodiments; it will not be repeated here.

[0085] The result notifyer is used to store the execution results of tasks. The result notifyer can inform the YOYO assistant in the application layer of the execution results of tasks.

[0086] The task module is used to create tasks. The created task includes task input data, task configuration data, result notification, and task sequence number.

[0087] The timeout detector is used to detect whether a task execution has timed out.

[0088] The session state module is used to store the execution results of tasks. For example, if the execution result of a material search task is that several materials were found in the image library, the session state module is used to store those materials.

[0089] The task output module is used to create task output data, which includes the task's execution status, such as normal, stopped, or occupied.

[0090] The editing module is used to perform tasks related to video generation. For example, the editing module performs tasks such as generating video, saving video, and changing music.

[0091] The application framework layer, or simply the framework layer, provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes a set of predefined functions.

[0092] The application framework layer can include window managers, content providers, view systems, resource managers, etc.

[0093] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0094] Content providers are used to store and retrieve data and make that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, etc.

[0095] A view system includes visual controls, such as controls for displaying text and controls for displaying images, and can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon can include views for displaying text and views for displaying images.

[0096] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0097] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0098] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0099] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0100] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0101] A 2D graphics engine is a graphics engine for 2D drawing.

[0102] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0103] Understandable, Figure 2 The layers in the software system illustrated, and the components contained in each layer, do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer layers than illustrated, and each layer may include more or fewer components.

[0104] It is understood that, in order to implement the video generation method in the embodiments of this application, the electronic device includes hardware and / or software modules that perform various functions. Based on the algorithm steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments.

[0105] It should be noted that although the embodiments of this application are described using the Android system as an example, the basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0106] The video generation method provided in the embodiments of this application will now be described in further detail with reference to the accompanying drawings.

[0107] In some embodiments, a mobile phone can provide image search and video creation functions through a voice assistant application (hereinafter referred to as a voice assistant), such as YOYO. However, this is not a limitation in practice. For example, a mobile phone can also provide image search and video creation functions through other applications, such as dedicated video editing applications and gallery applications.

[0108] The following text primarily uses a voice assistant as an example to illustrate the solution of this application. It is understood that the implementation principle is similar for other applications, and will not be elaborated upon in the embodiments of this application.

[0109] A mobile phone can provide at least one of the following methods to trigger the phone to start searching for images.

[0110] Method 1: Wake up the voice assistant and enter the image search request.

[0111] The wake-up methods for the voice assistant include, but are not limited to, at least one of the following: Wake-up method 1, keyword wake-up: the phone can wake up the voice assistant when it recognizes the user's input of a wake-up voice containing keywords, such as "Hello YOYO"; Wake-up method 2, the phone can wake up the voice assistant when it detects the user's long press of the power button; Wake-up method 3, breath wake-up: the phone can wake up the voice assistant when it detects the user raising the phone and pointing their mouth towards the microphone. This application does not specifically limit the wake-up methods.

[0112] Taking the wake-up method of inputting the voice prompt "Hello YOYO" as an example, see [link / reference]. Figure 3 When the phone displays interface 301, in response to the input of the wake-up voice "Hello YOYO", the phone can wake up the voice assistant. After waking up the voice assistant, the phone can display interface 302. Interface 302 includes a prompt card 3021 for the voice assistant, indicating that the voice assistant has been woken up.

[0113] After the phone activates the voice assistant using the above method, the voice assistant is now active. While the voice assistant is active, the phone can receive voice input from the user. For example, the user can input the voice command "image of my second daughter dancing." Alternatively, the phone can also receive text input from the user, such as... Figure 3 The prompt card 3021 in the interface 301 shown includes a keyboard control 3022. In response to the user's trigger operation (such as a click operation) on the keyboard control 3022, the mobile phone can provide a keyboard for the user to input text.

[0114] In response to a user's input of a voice or text instruction to search for an image (i.e., inputting an image search request), the mobile phone can display a dialog box of the voice assistant (hereinafter referred to as the dialog box), which includes the image search request.

[0115] Taking the image search prompt "image of the second daughter dancing" as an example, see below. Figure 3 When the phone displays interface 302, in response to the user's voice input "image of the second daughter dancing", the phone can display interface 303, which includes a voice assistant dialog box 3031. The dialog box 3031 includes the dialogue "image of the second daughter dancing" 3032.

[0116] In addition, in response to a user's input of an image search request, the phone can also begin searching for images in the gallery app that match the search request, thus triggering the image search to begin.

[0117] Therefore, it should be noted that in this embodiment, the format of the image search request input by the user via text or voice can be arbitrary. Taking searching for images of the second daughter dancing as an example, the search request could be "images of the second daughter dancing," "search for images of the second daughter dancing," "including images of the second daughter dancing," etc. It is understood that creating videos typically requires finding images. Therefore, the format of the image search request could also be "generate / create a video of...", such as generating a birthday greeting video for Jimmy, generating a video of a trip to Beijing, etc. In practice, the mobile phone can limit the voice or text image search request to one or more of the above fixed formats, such as the fixed format "generate a video of...", to facilitate the identification of the user's needs. This will not be repeated below.

[0118] Method 2: Enter the image search request after entering the voice assistant's dialog box.

[0119] An icon for the voice assistant is displayed on the phone's home screen. In response to the user's action of clicking the voice assistant icon, such as tapping it, the phone can display a dialog box.

[0120] See Figure 4 The phone can display interface 401. Interface 401 is the phone's desktop. Interface 401 includes an icon 4011 for the voice assistant. In response to the user's click on icon 4011, the phone can display interface 402, which includes a dialog box 4021.

[0121] In dialog box 4021, the user can also enter a search request. In response to entering a search request in the dialog box, the phone can also display the search request in the dialog box, and search for image materials, thus triggering the start of the search.

[0122] Taking the example of the search prompt "image of the second daughter dancing," see below. Figure 4 When the phone displays interface 402, in response to the user's voice input "image of the second daughter dancing", the phone can display interface 403 (the same as interface 303 mentioned above), which also includes a dialog box 4031. The dialog box 4031 further includes the dialogue "image of the second daughter dancing" 4032.

[0123] It should be noted that the dialog box in Method 1 (e.g., dialog box 3031) and the dialog box in Method 2 (e.g., dialog box 4031) can be the same dialog box. Therefore, in the dialog box of Method 1, the mobile phone can also respond to the user's input of an image search request and start the image search. Further details will not be elaborated here.

[0124] It should be understood that the above Figure 3 and Figure 4 The illustrated user interface of the phone triggering the image search is merely illustrative and does not constitute any limitation on the user interface of the electronic device triggering the image search in the video generation method provided in this application embodiment. For example, in some implementations, in response to a user's triggering operation on any creative recommendation card displayed on the phone's user interface, such as a click operation, the phone can display a video creation dialog box (also simply referred to as a dialog box) and start a video creation conversation in the dialog box. Any user interface can be the desktop, the negative one screen, the lock screen, the application interface of a video application, the application interface of a chat application, etc. In the following, the display of a creative recommendation card on the phone's desktop is mainly used as an example to illustrate the solution of this application. The creative recommendation card displayed on the phone's user interface can be created in the following way: the phone displays the creative recommendation card after recognizing that at least one of the current time, current location, and images in the gallery application meets the conditions for video creation.

[0125] Below, in conjunction with Figures 5 to 9 A schematic diagram illustrating a scenario of the video generation method provided in the embodiments of this application.

[0126] In the embodiments of this application, during the process of video generation by the electronic device, the electronic device may perform tasks related to video generation without specific limitations. For example, tasks related to video generation may include, but are not limited to, one or more of the following tasks: material search task, character confirmation task, material addition task, material deletion task, material selection task, video generation task, template change task, music change task, duration change task, and video saving task.

[0127] During the execution of the above tasks by the electronic device, the execution result of each task can be one of the following states: task completed (Scenario 1), task stopped (Scenario 2), task error reported (Scenario 3), task timeout (Scenario 4), and task interrupted (Scenario 5). Of course, in other embodiments, the execution result of each task can also be other states, and this application embodiment does not limit this.

[0128] The user interfaces of the electronic devices involved in scenarios one through five above will be described below.

[0129] Scenario 1: During the execution of the video generation method provided in the embodiments of this application by the electronic device, the electronic device can perform the task normally, and the task does not stop, time out, interrupt, or malfunction during the execution of the task.

[0130] In one example, taking the scenario where a voice assistant in an electronic device receives a user's voice and directly generates a target video based on searched materials, this application's embodiment of the video generation method relates to the user interface of the electronic device. Please refer to... Figure 4 Users can communicate via voice or text. Figure 4 After the text "Help me generate a video of my second daughter dancing" is entered in dialog box 4031 of the user interface 403 shown, the electronic device displays... Figure 5 The user interface 501 is shown. (As shown...) Figure 5 The user interface 501 displays a dialog box 5011 containing the text "Searching for materials for you..." (5013) and a stop control 5014. The stop control 5014 is used to trigger the electronic device to stop the current material search task. When the electronic device displays the user interface 501 to the user and the user does not trigger the stop control 5014, the electronic device displays search results. These search results may include the text "Selected materials for the second daughter" (5022) displayed in the dialog box 5021 of the user interface 502, and thumbnails of the displayed materials (5023). The thumbnails 5023 display a "View All" control (50321), such as... Figure 5 The "View All" control 50321 indicates that 15 out of 20 clips have been selected. Optionally, the user can also increase or decrease the number of clips displayed in the thumbnail 5023 by triggering the "View All" control 50321.

[0131] In electronic devices display such as Figure 5 After the user interface 502 is displayed, the user can trigger the video generation control 50232 shown in the material card 5023 of the user interface 502 to trigger the electronic device to generate the target video. Specifically, after the user triggers the video generation control 50232, the electronic device displays as follows: Figure 5 The user interface 503 is shown. The dialog box 5031 displayed in user interface 503 shows the text "Generate Video" 5032, the text "Analyzing Footage..." 5033, and a stop control 5034. The text 5033 indicates the progress of the video production, and the stop control 5034 is used to trigger the electronic device to stop the current video generation task. For example, the content in the text "Analyzing Footage..." 5033 can also be replaced with "Processing, please wait..." etc.

[0132] When the electronic device displays user interface 503 to the user, and the user does not trigger the stop control 5034 displayed in user interface 503, the electronic device will display to the user as follows: Figure 5 The user interface 504 is shown. The dialog box 5041 displayed in the user interface 504 shows the execution result corresponding to the video generation task. The execution result may include the text 5042 displayed in the dialog box 5041 of the user interface 504, "The video has been completed. Come and enjoy the little cutie's dance~", and a thumbnail of the target video 5043.

[0133] After the video is created, the electronic device can perform editing functions such as saving and deleting the video.

[0134] In some implementations, the thumbnail 5043 of the target video can also display editing controls. For example, the editing controls include at least one of the following: such as a "Save" control 50431, a "Delete" control, a "Like" control, a "Dislike" control, etc., displayed in the user interface 504.

[0135] Scenario 2: During the execution of the video generation method provided in the embodiments of this application, the electronic device stops performing the task.

[0136] In one example, taking the situation where the voice assistant in the electronic device encounters a situation where it stops performing a task to search for materials after receiving the user's voice as an example, the user interface of the electronic device involved in the video generation method provided in this application embodiment is introduced.

[0137] In practical applications, when electronic devices display the above to users Figure 5 After the user interface 501 is displayed, if the user needs to stop the electronic device from performing the task of searching for materials, the user can input a stop command to the voice assistant via voice or text. For example, the user can click... Figure 6 The user interface 601 shows a stop control 6014. In response to the user clicking the stop control 6014, the electronic device displays... Figure 6 The user interface 602 shown has a dialog box 6021 displaying the text "This answer has stopped" 6022.

[0138] Scenario 3: During the execution of the video generation method provided in the embodiments of this application by the electronic device, an error occurs in the task performed by the electronic device.

[0139] In one example, after the voice assistant in the electronic device receives the user's voice, while performing the task of generating a video, the electronic device also performs the task of saving the video. In this case, because the electronic device is performing the task of generating the video, it cannot perform the task of saving the video. Thus, the task of saving the video that the electronic device is about to perform has encountered an error. This example illustrates the user interface of the electronic device involved in the video generation method provided in this application embodiment.

[0140] In practical applications, in Figure 5 The user interface 503 displays the text "Analyzing material..." followed by the text 5033, and in... Figure 5 Before the target video thumbnail is displayed in dialog box 5041 in the user interface 504, the user... Figure 7 After the user inputs the text "Save Video" via voice in dialog box 7011 of the user interface 701, the electronic device responds to the user's voice input and displays the following: Figure 7 The user interface 702 shown displays the text 7021 that reads "The video is not yet finished and cannot be saved".

[0141] Scenario 4: During the execution of the video generation method provided in the embodiments of this application by the electronic device, the task executed by the electronic device times out.

[0142] In one example, taking the case where a timeout occurs during the process of a voice assistant in an electronic device receiving a user's voice and performing a task to search for materials, this application's embodiment of the video generation method describes the user interface of the electronic device involved.

[0143] In practical applications, in Figure 5 When the user interface 501 displays the text "Searching for materials for you...", it means the electronic device is performing a material search task. If the electronic device times out during the material search task, it will display... Figure 8 The user interface 801 displays the text "Finding material has timed out" 8013.

[0144] Scenario 5: During the execution of the video generation method provided in the embodiments of this application by an electronic device, the task performed by the electronic device is interrupted.

[0145] In one example, taking the interruption that occurred during the process of the voice assistant in the electronic device receiving the user's voice and performing the task of generating a video as an example, this application describes the user interface of the electronic device involved in the video generation method.

[0146] In practical applications, such as in electronic device displays... Figure 5After the user interface 502 is displayed, the user can trigger the video generation control 50232 shown in the material card 5023 of the user interface 502 to trigger the electronic device to perform the task of generating video. If the electronic device is interrupted during the process of performing the video generation task, the electronic device displays... Figure 9 The user interface 901 shown includes a pop-up window 9011, which contains a control 9012. The user can click the control 9012, and the electronic device can resume the interrupted video generation task in response to the user's click. Specifically, after the user clicks the control 9012, the electronic device displays... Figure 9 The user interface 902 shown includes a pop-up window 9021, which includes a control 9022. The user can click the control 9022, and the electronic device responds to this click by displaying... Figure 5 The electronic device has user interfaces 503 and 504, and can display a thumbnail 5043 of the prepared target video in user interface 504.

[0147] It should be understood that the above Figures 5 to 9 The user interface shown is merely illustrative and does not constitute any limitation on the user interface of the electronic device to which the video generation method provided in this application embodiment is adapted. For example, the above... Figures 5 to 9 The materials shown can be still images, animated images, or videos, etc.

[0148] The execution process of the electronic devices involved in scenarios one through five above will be described below.

[0149] Scenario 1 involves the execution process of an electronic device generating a video. During this process, the electronic device needs to create a material search task, execute the material search task, create a video generation task, and execute the video generation task.

[0150] The following describes the implementation process of creating a material search task using electronic devices in Scenario 1. Figure 10 This is a timing diagram of a material search task creation method provided in an embodiment of this application.

[0151] S1001, YOYO Assistant receives user voice 1.

[0152] It should be understood that User Voice 1 is the conversational input content received by YOYO Assistant from the user. For example, User Voice 1 could be "Make a video of my second daughter dancing from childhood to adulthood". Figure 4 The user interface 403 shows options such as "Help me generate a video of my second daughter dancing".

[0153] In some embodiments, in addition to voice input, users can also input conversational content into the YOYO Assistant through other means, which are not limited in this application embodiment. For example, a user can send the conversational content "Make a video of my second daughter dancing from childhood to adulthood" to the YOYO Assistant via text input.

[0154] In practice, users can input voice or text through the conversation interface provided by YOYO Assistant on their electronic devices. The conversation interface provided by YOYO Assistant can be like... Figure 4 The user interface 402 is shown.

[0155] S1002, after receiving user voice 1, YOYO Assistant generates analysis instruction 1 (carrying user voice 1) based on user voice 1.

[0156] For example, user voice 1 could be Figure 4 The text 4032 shown is "Help me generate a video of my second daughter dancing". Analysis instruction 1 is used to instruct semantic analysis and intent analysis of user voice 1 to extract user intent corresponding to user voice 1 and information such as time, place, person and event corresponding to user voice 1.

[0157] S1003, YOYO Assistant sends analysis command 1 to the speech model.

[0158] It should be understood that a language model can be a neural network model, and a speech model can perform semantic analysis and intent analysis on the text or speech input by the user.

[0159] The language model can be a module located in YOYO Assistant, or it can be a module located outside of YOYO Assistant; there are no specific limitations on this.

[0160] S1004, after receiving the analysis instruction 1, the speech model analyzes and processes the user's speech 1 and generates analysis result 1.

[0161] It should be understood that analysis result 1 includes creative intent. Figure 1 Semantic information 1. Creative intent refers to the purpose of a user's creation. For example, based on the conversational input of "making a video of my second daughter dancing from childhood to adulthood," a speech model can determine the user's creative intent. Figure 1For creating a video. Semantic information 1 includes information such as the time, location, people, and events corresponding to user voice 1. For example, if user voice 1 is "Help me generate a video of my second daughter dancing," semantic information 1 can include the person as "second daughter" and the event as "dancing." As another example, if user voice 1 is "Generate a video of my trip to Beijing last October," semantic information can include the time "last October," the location "Beijing," the person "me," and the event "tripping."

[0162] S1005, the speech model returns analysis result 1 to YOYO Assistant.

[0163] S1006, YOYO Assistant generates material search command 1 (carrying session identifier 1 and semantic information 1) based on analysis result 1.

[0164] It should be understood that the material search command is used to instruct the electronic device to search the gallery for materials that match the semantic information. For example, if the voice information 1 includes the person as "second daughter" and the event as "dancing", then the material search command is used to instruct the electronic device to search the gallery for images or videos of the second daughter dancing.

[0165] Session identifiers are used to identify the session page when a user inputs content into the YOYO Assistant while creating a video. Different videos created by different users have different session identifiers. For example, if user voice A instructs the creation of a video about traveling in Beijing, and user voice B instructs the creation of a video about traveling in Xi'an, the videos created by user voice A and user voice B are different. Therefore, the session identifier A corresponding to user voice A and the session identifier B corresponding to user voice B are two different session identifiers.

[0166] It should also be understood that YOYO Assistant can base its analysis on the creative intent in Result 1. Figure 1 "Create video," generating material search command 1. In other embodiments, YOYO Assistant can also generate other commands based on creative intent. For example, if the user's voice says "Save video," then the creative intent is "Save video," and YOYO Assistant generates the save video command based on the creative intent of "Save video."

[0167] S1007 After YOYO Assistant generates the material search command, it notifies the creation module, task queue module, and timeout checker in the media platform framework layer to initialize.

[0168] It should be understood that after YOYO Assistant notifies the creation module, task queue module, and timeout checker in the media platform framework layer to initialize, the creation module, task queue module, and timeout checker can perform initialization operations to put their own modules into a working state.

[0169] It should also be understood that electronic devices can notify the creation module, task queue module, and timeout checker in the media platform framework layer to initialize after activation (first power-on), after each power-on, after a system update, after a factory reset, or when the electronic device is running YOYO Assistant. This application provides an example illustrating how the electronic device running YOYO Assistant, after generating a material search command, notifies the creation module, task queue module, and timeout checker in the media platform framework layer to initialize.

[0170] It should also be understood that after the creation module, task queue module, and timeout checker in the media middle platform framework layer are initialized, a message will be returned to YOYO Assistant that the above modules have completed initialization.

[0171] In other embodiments, after YOYO Assistant generates a material search command, it can also initialize modules in the media platform framework layer other than the creation module, task queue module, and timeout checker, such as initializing the result notification device. This application embodiment does not limit this.

[0172] S1008, after receiving the message that the creation module, task queue module and timeout detector module have completed initialization, YOYO Assistant sends material search command 1 (carrying session identifier 1 and semantic information 1) to the creation module.

[0173] S1009, the creation module generates the following commands based on the material search command 1: creation task input command 1, creation material snapshot cache command 1, creation task configuration command 1, and creation material search task command 1.

[0174] S1010, the authoring module sends a task input creation instruction 1 (carrying session identifier 1 and semantic information 1) to the task input module.

[0175] S1011, after receiving the task input creation instruction 1, the task input module creates a task input 1 object (including task identifier 1, session identifier 1, and semantic information 1).

[0176] It should be understood that creating a task input instruction refers to creating a task input object related to video generation. For example, creating a task input 1 object means creating a task input 1 object for the material search task. Tasks related to video generation can be referred to in the above embodiments, and will not be repeated here. An object is an entity used in a program to describe an objective thing; a task input object is an entity used in a program to describe task input. Task input includes a task identifier, session identifier, and semantic information.

[0177] After receiving the "Create Task Input Command 1", the task input module can generate a task identifier. It should be understood that the task identifier is used to identify a task related to video generation; one task identifier identifies one task. For example, Table 1 shows the correspondence between task identifiers and tasks.

[0178] Table 1

[0179] Task Identifier Task type Add to task queue? Task group number 1 Material Search yes 1 2 Character confirmed yes 2 3 Material addition yes 3 4 Material deletion yes 4 5 Selected materials yes 5 6 Generate video yes 6 7 Change template yes 6 8 Change music yes 6 9 Adjust duration yes 6 10 Save video yes 6 11 stop no 7 12 Can interrupted connection be resumed? no 8

[0180] In this embodiment of the application, after receiving the create task input instruction 1, the task input module can generate the task identifier 1 in Table 1, and create a task input 1 object based on the session identifier 1 and semantic information 1 carried in the create task input instruction 1. The task input 1 object includes the session identifier 1, semantic information 1, and task identifier 1.

[0181] S1012, the task input module returns the task input 1 object to the creation module.

[0182] S1013, the creation module sends the creation material snapshot cache instruction 1 to the snapshot cache module.

[0183] S1014, after receiving the instruction to create material snapshot cache 1, the snapshot cache module creates the material snapshot cache 1 object.

[0184] It should be understood that a cache refers to temporary storage, and a media snapshot cache object refers to the address in temporary storage where media snapshot data is stored. A snapshot is a fully usable copy of a specified set of data, which includes an image of the corresponding data at a certain point in time (the time when the copy begins). A snapshot can be a copy of the data it represents, or it can be a replica of the data.

[0185] It should also be understood that, in the embodiments below, when starting to execute video generation-related tasks, it is first necessary to preprocess each video generation-related task, including snapshot processing, which can store the snapshot data obtained from snapshot processing in the material snapshot cache object.

[0186] In this embodiment of the application, the material snapshot cache 1 object refers to the address that stores the snapshot data of the materials searched by the material search task.

[0187] S1015, the snapshot cache module returns a material snapshot cache 1 object to the creation module.

[0188] S1016, the creation module sends the task configuration creation instruction 1 to the task configuration module.

[0189] S1017, after receiving the task configuration instruction 1, the task configuration module creates a task configuration 1 object (task configuration 1 includes task name 1, snapshot cache 1 object, task group number 1 and preset timeout duration 1).

[0190] It should be understood that the task name refers to the name of the task related to video generation. For example, when the task related to video generation is a material search task, the name of the material search task can be "material search". This name can also be other custom fields, which are not limited in this embodiment.

[0191] It should also be understood that each task related to video generation has its corresponding task group number, and multiple tasks can correspond to one group number. For example, see Table 1: the tasks for generating video, changing modules, changing music, adjusting duration, and saving video correspond to the same task group number 6. The remaining tasks in Table 1 correspond to different task group numbers. In this embodiment, task group number 1 refers to the group number corresponding to the material search task in Table 1.

[0192] The preset timeout duration refers to the timeout threshold for each task. If the execution time exceeds the preset timeout duration during task execution, the task has timed out. In this embodiment, the preset timeout duration can be configured based on device performance, and this embodiment does not limit this. In this embodiment, the preset timeout durations for different tasks can be the same or different, and this embodiment does not limit this.

[0193] S1018, the task configuration module returns a task configuration 1 object to the creation module.

[0194] S1019, the authoring module sends the Create Result Notifier instruction 1 to the Result Notifier.

[0195] S1020, The result notifier creates the result notifier 1 object corresponding to task identifier 1.

[0196] It should be understood that the result notify object can store the execution result of the task corresponding to the task identifier. For example, the result notify 1 object can store the execution result of the material search task corresponding to task identifier 1.

[0197] S1021, The result notifier returns the result notifier1 object to the authoring module.

[0198] S1022, the creation module sends a creation material search task instruction 1 to the task module (carrying a task input 1 object, a task configuration 1 object, and a result notification 1 object).

[0199] S1023, the task module creates material search task 1 (including task input 1 object, task configuration 1 object, result notification 1 object and task number 1).

[0200] For example, after receiving the instruction 1 to create a material search task, the task module can obtain the task number of the task at the end of the task queue and generate the task number of the currently created task based on that task number. For instance, if the task number at the end of the task queue is 3, meaning there are 2 tasks before the end of the queue, then the task number of the currently created task is generated as 4. Alternatively, if there are no tasks in the task queue, then the task number of the currently created task is generated as 1.

[0201] It should be understood that during the video generation process, electronic devices need to execute the material search task first and then the video generation task. Therefore, under normal circumstances, when the task module creates a material search task, there is no task in the task queue, so the task number of the material search task generated by the task module can be task number 1.

[0202] In other embodiments, the tasks created by the task module may also be referred to as task instances, etc., and this application does not limit this.

[0203] S1024, the task module returns material search task 1 (including task input 1 object, task configuration 1 object, result notification 1 object and task number 1) to the creation module.

[0204] It should be understood that steps 1001 to 1024 above specifically illustrate the implementation process of the electronic device creating a search task, and the implementation process of the electronic device creating other tasks related to video generation (such as...). Figure 12 The implementation process of creating and generating video tasks, creating and saving video tasks, etc. can be found in the implementation process of creating search tasks, which will not be repeated here.

[0205] The following describes the implementation process of the electronic device performing the material search task in Scenario 1. Figure 11 This is a timing diagram of an embodiment of the present application for performing a material search task.

[0206] S1025, after receiving the material search task 1, the creation module sends the material search task 1 (including task input 1 object, task configuration 1 object, result notification 1 object and task sequence number 1) to the task queue module.

[0207] It should be understood that, Figure 10 In step 1024, the task module returns material search task 1 to the creation module. After receiving material search task 1, the creation module sends material search task 1 to the task queue module to publish material search task 1.

[0208] S1026, After receiving the material search task 1, the task queue module checks whether task group number 1 of the material search task 1 is occupied.

[0209] The explanation of the task group number can be found in the above example, and will not be repeated here.

[0210] In implementation, the task queue module can detect whether a newly published task's task group number is occupied by checking if the task group number of an existing task in the task queue is the same as the task group number of the newly published task. For example, referring to the task types and task group numbers shown in Table 1, assuming there is already a task to generate a video in the task queue, and the task group number corresponding to this task is 6, when the task module publishes a new task to the task queue module, for example, the new task to save a video, which also has a task group number of 6, then the task queue module detects that task group number 6 is occupied. Conversely, assuming there is already a task to search for materials in the task queue, and the task group number corresponding to this task is 1, when the task module publishes a new task to the task queue module, for example, the new task to generate a video, which has a task group number of 6, then the task queue module detects that task group number 6 is not occupied.

[0211] S1027, The task queue module records the occupancy status of task group number 1 based on the detection results.

[0212] It should be understood that during the video generation process of the electronic device, before the task module publishes the material search task to the task queue module, there are no other tasks in the task queue. Therefore, the task group number 1 of the material search task 1 detected by the task queue module in step 1026 is not occupied, and the occupancy status of the task group number 1 recorded in step 1027 is "unoccupied".

[0213] The implementation process for setting the task group number to "occupied" can be found in the following example, and will not be repeated here.

[0214] S1028, when the task queue module determines that the occupancy status of task group number 1 is unoccupied, it sends an instruction to the timeout detector to start timing for material search task 1.

[0215] It should be understood that the task queue module sends a timeout detection instruction to the timeout detector to check if a task has timed out. The implementation process for detecting task timeouts can be found in the following embodiment, and will not be repeated here.

[0216] S1029, the task queue module adds material search task 1 to the end of the task queue.

[0217] It should be understood that when a new task is published to the task queue module through the task module, the task queue module can add the task to the end of the task queue. The newly added task will be executed after the tasks before the end of the queue have been completed.

[0218] It should also be understood that during the video generation process of the electronic device, before the task module publishes the material search task to the task queue module, there are no other tasks in the task queue. Therefore, the task queue module adding material search task 1 to the end of the task queue means that material search task 1 is at the beginning of the task queue.

[0219] S1030, the task queue module sends task number 1 of material search task 1 to the creation module.

[0220] It should be understood that when the task queue module is about to start scheduling and executing a task, it can send the task sequence number of the task to be executed to the creation module so that the creation module can know the task to be executed.

[0221] S1031, the creation module records the current task number as task number 1.

[0222] S1032, the creation module sends the execution result 1 of the material search task 1 corresponding to task number 1 to the YOYO assistant based on task number 1 (execution result 1 indicates that material search task 1 is being executed in the task queue).

[0223] S1033, YOYO Assistant displays interface 1, including "Searching for materials", based on execution result 1.

[0224] For example, interface 1 can be referred to Figure 5 The user interface 501 is shown. The text 5013 in the user interface 1 reads "Searching for materials for you". The text 5013 can also be "Searching for materials", "Searching for materials", etc., and this application embodiment does not limit it.

[0225] S1034, the task queue module schedules and executes the first task in the task queue (i.e., material search task 1).

[0226] S1035, the task queue module sends the execution instruction for material search task 1 to the task module.

[0227] S1036, the task module receives the execution instruction for material search task 1 and performs preprocessing on material search task 1.

[0228] It should be understood that when the task module begins executing tasks related to video generation, it first needs to perform preprocessing on each task. Preprocessing includes snapshot processing, which stores the snapshot data obtained from the snapshot processing in a media snapshot cache object. The implementation process of snapshot processing can be referred to S1037 to S1039. Of course, in other embodiments, preprocessing may also include other processing, and this application embodiment does not limit this.

[0229] S1037, the task module sends a snapshot processing instruction to the snapshot cache module.

[0230] S1038, after receiving the snapshot processing instruction, the snapshot cache module retrieves the material set from the session state module.

[0231] It should be understood that the session state module stores relevant materials obtained after the task module completes tasks related to video generation. For example, after completing the material search task, the task module can find relevant materials and sends them to the session state module for storage. Similarly, after completing the video generation task, the task module can generate a video and sends it to the session state module for storage.

[0232] In the implementation, after the task module executes search task 1, the task module can store the searched materials in the session state module. After receiving the snapshot processing instruction, the snapshot cache module can obtain the set of materials searched by material search task 1 from the session state module. Then, the snapshot cache module performs snapshot processing on the set of materials to generate snapshot information of the set of materials.

[0233] S1039, the snapshot caching module performs snapshot processing on the material set after obtaining the material set, and generates snapshot information of the material set.

[0234] It should be understood that a snapshot is a fully usable copy of a specified data set, including an image of the corresponding data at a specific point in time (the time the copy begins). Snapshot information for the media set is used to support the functionality of stopping a task, allowing for a rollback to the state prior to task processing after the task is stopped.

[0235] S1040, the task module checks whether the material search task 1 has been interrupted.

[0236] It should be understood that when the task module executes tasks corresponding to certain task group numbers, it needs to call the editing module in the media platform framework layer to execute the task. For example, referring to Table 1, when the task module executes the task corresponding to task group number 6, it needs to call the editing module to execute the task corresponding to task group number 6. When the task module executes tasks with group numbers 1 to 5 and 7 to 8, it does not need to call the editing module; the task module executes the task itself.

[0237] In implementation, the task module can detect whether a task is interrupted by checking if the editing module can be called. For example, when the task module executes the material search task 1 corresponding to task group number 1, it does not need to call the editing module; therefore, the detection result could be that the material search task 1 is not interrupted. As another example, if the editing module has been uninstalled or encountered an error and cannot be called, when the task module executes the video generation task 6 corresponding to task group number 6, the task execution module would normally need to call the editing module to execute video generation task 6. However, since the editing module has been uninstalled or encountered an error and cannot be called, the detection result could be that video generation task 1 is interrupted.

[0238] S1041, if the task module detects that the material search task 1 is not interrupted, it executes the material search task 1 according to the task input 1 object in the material search task 1.

[0239] In implementation, if the task module detects that the material search task 1 is not interrupted, it can send a request to the task input module to obtain the task input object corresponding to the material search task 1. The task input module returns the task input object to the task module. Based on the task identifier 1, session identifier 1, and semantic information 1 included in the task input object, the task module executes the material search task 1. For example, if the task identifier 1 corresponds to the material search task, the session in the session identifier 1 is "make a video of the second daughter dancing from childhood to adulthood," and the voice information 1 includes the person as "second daughter" and the event as "dancing," then the task module can search the image library for images and / or videos containing the second daughter dancing based on the above information.

[0240] S1042, the task module generates a task output object creation instruction (carrying the completed task status) based on the task status when executing the task.

[0241] It should be understood that a task status may include one or more of the following: task completed, task error, task timeout, task occupied, task stopped, etc.

[0242] In this embodiment of the application, the task status of material search task 1 is task completed.

[0243] S1043, the task module sends a command to the task output module to create a task output object (carrying the status of the completed task).

[0244] It should be understood that the task output object may include the task status of the task related to video generation.

[0245] S1044, The task output module creates a task output object (including the task status of completed material search task 1).

[0246] S1045, the task output module returns a task output object (carrying the status of the completed task) to the task module.

[0247] S1046, The task module generates execution result 2 (including multiple images) based on the task output object.

[0248] In this embodiment of the application, the task output corresponds to the task status of the material search task 1 being completed. After receiving the task output object, the task module can obtain multiple searched materials from the image library and generate execution result 2 based on the multiple searched materials.

[0249] S1047, the task module sends an update session status command (carrying multiple images) to the session status module.

[0250] S1048, the session status module saves or updates materials.

[0251] It should be understood that when the data set of materials stored in the session state module is empty, the session state module will save multiple materials to the session state module after receiving a session state command. When the data set of materials stored in the session state module is not empty, the session state module will update the data set in the session state module after receiving a session state command. For example, if the session state module originally stored 30 photos of Li Si traveling in Beijing, and the session state command received by the session state module includes 20 photos of his second daughter dancing, then the session state module will update the 30 photos of Li Si traveling in Beijing to the 20 photos of his second daughter dancing. As another example, if the session state module originally stored 30 photos of Li Si traveling in Beijing, and the session state command received includes 5 deleted photos of Li Si traveling in Beijing, then the session state module will update the 30 photos of Li Si traveling in Beijing to 25 photos of Li Si traveling in Beijing.

[0252] S1049, the task module checks whether material search task 1 has stopped.

[0253] In implementation, the user is Figure 6 When the stop control 6014 is clicked in the interface 601 shown, or when the user inputs a stop command via voice, the task module detects that the material search task 1 has stopped; otherwise, the task module detects that the material search task 1 has not stopped.

[0254] In this embodiment of the application, the user did not... Figure 6 Clicking the stop control 6014 in the shown interface 601, or receiving a stop command via voice input, causes the task detection module to detect that the material search task 1 has not stopped.

[0255] S1050, if the task module detects that the material search task 1 has not stopped, confirm that the execution result 2 has taken effect.

[0256] It should be understood that Execution Result 2 includes multiple media files searched when the task module performed the media search task.

[0257] S1051, the task module sends execution result 2 (carrying multiple images) to the result notifier based on the result notifier 1 object.

[0258] It should be understood that the task module sends the execution result to the result notification device so that the result notification device can notify the YOYO Assistant, so that the YOYO Assistant can know the execution result of the task and display the corresponding interface.

[0259] S1052, The result notifier records the execution result 2 in the result notifier 1 object.

[0260] S1053, the result notification device sends execution result 2 (carrying multiple images) to YOYO Assistant.

[0261] S1054, YOYO Assistant displays interface 2, which includes multiple materials, based on execution result 2.

[0262] For example, interface 2 can be referenced. Figure 5 The user interface 502 shown can contain multiple images, which can be 5023 within the user interface 502.

[0263] S1055, the task module sends a task output object to the task queue module (the task output object includes the task status of completed material search task 1).

[0264] S1056, the task queue module cleans up tasks based on the task output object.

[0265] It should be understood that the task queue module can clean up tasks based on the task status of the tasks included in the task output object. For example, if the task status of material search task 1 is "completed," then the task queue module will clean up completed material search task 1, and material search task 1 will not be added to the task queue. As another example, if the task status of material search task 1 is "timeout," then the task queue module will clean up material search task 1 in the timeout state, and material search task 1 will not be added to the task queue.

[0266] S1057, the task queue module sends a command to the timeout detector to stop timing the material search task 1.

[0267] In this embodiment of the application, during the execution of a task, if the timeout detector does not detect that the execution time of the task has expired, the task queue module sends an instruction to the timeout detector to stop timing the task. After receiving the stop timing instruction, the timeout detector stops timing the material search task 1.

[0268] It should be understood that steps 1025 to 1057 above specifically describe the implementation process of the electronic device performing the search task, and the implementation process of the electronic device performing other tasks related to video generation (such as...). Figure 13 The implementation process of executing the video generation task and the video saving task can be referred to the implementation process of executing the search task, which will not be repeated here.

[0269] The following describes the implementation process of creating a material search task using electronic devices in Scenario 1. Figure 12 This is a timing diagram of an embodiment of the present application for performing a material search task.

[0270] S1201, YOYO Assistant receives user voice 2.

[0271] It should be understood that User Voice 1 is the conversational input content received by the YOYO Assistant from the user. The explanation of the conversational input content has been described in the above embodiments and will not be repeated here.

[0272] In this embodiment of the application, user voice 2 refers to the voice input by the user in the dialogue interface provided by YOYO Assistant in the electronic device, which says "generate video".

[0273] In other embodiments, the user can also click Figure 5 The user interface 502 shows a video generation control 5023. In response to the user clicking the control 5023, the electronic device generates the analysis instruction 2 in step S1202.

[0274] S1202, after receiving user voice 2, YOYO Assistant generates analysis instruction 2 (carrying user voice 2) based on user voice 2.

[0275] For example, user voice 2 could be Figure 5 The text 5031 for "Generate Video" is shown in interface 503. Analysis instruction 2 is used to instruct semantic analysis and intent analysis of user voice 2 to extract user intent corresponding to user voice 2 and information such as time, location, person, and event corresponding to user voice 2.

[0276] S1203, YOYO Assistant sends analysis command 2 to the speech model.

[0277] S1204, after receiving the analysis instruction 2, the speech model analyzes and processes the user's speech 2 and generates analysis result 2.

[0278] It should be understood that analysis result 2 includes creative intent. Figure 2 And semantic information 2. For example, based on the conversational input of "generate video", the speech model can determine the user's creative intent. Figure 2 To generate a video.

[0279] It should also be understood that when a user enters conversational input on the same conversation page of YOYO Assistant (i.e., the user enters different voice or text commands, but the conversation identifier generated by YOYO Assistant is the same), the semantic information corresponding to different user voices in the analysis results generated by the speech model is the same. For example, if a user first enters "Help me generate a video of my second daughter dancing" (user voice 1), and then enters "Generate video" (user voice 2), since the purpose of user voice 2 is also to generate a video of the second daughter dancing, the conversation identifiers of user voice 1 and user voice 2 are the same. Therefore, the semantic information 1 of user voice 1 and the semantic information 2 of user voice 2 in the analysis results generated by the speech model are the same. An explanation of semantic information 2 can be found in semantic information 1, and will not be repeated here.

[0280] S1205, the speech model returns analysis result 2 to the YOYO assistant.

[0281] S1206, YOYO Assistant generates video generation command 2 (carrying session identifier 1 and semantic information 2) based on analysis result 2.

[0282] It should be understood that the video generation command is used to instruct the electronic device to synthesize a video from the material found in the library that matches semantic information 2.

[0283] S1207, YOYO Assistant sends video generation command 2 (carrying session identifier 1 and semantic information 2) to the authoring module.

[0284] S1208, the creation module generates the following commands based on the video generation command 2: creation task input command 2, creation material snapshot cache command 2, creation task configuration command 2, and creation material search task command 2.

[0285] S1209, the authoring module sends a task input creation instruction 2 (carrying session identifier 1 and semantic information 2) to the task input module.

[0286] S1210, the task input module creates a task input 2 object (including task identifier 2, session identifier 1, and semantic information 2).

[0287] For example, creating a task input 2 object means creating a task input 2 object to generate a video task. In this embodiment, the task identifier 2 in the task input 2 object can refer to the task identifiers for generating video tasks in Table 1.

[0288] S1211, the task input module returns the task input 2 object to the creation module.

[0289] S1212, the creation module sends the creation material snapshot cache instruction 2 to the snapshot cache module.

[0290] S1213, the snapshot cache module creates a material snapshot cache 2 object.

[0291] In this embodiment of the application, the material snapshot cache 2 object refers to the address that stores the snapshot data of the video generated by the video generation task.

[0292] S1214, the snapshot cache module returns the material snapshot cache 2 object to the creation module.

[0293] S1215, the creation module sends the task configuration creation instruction 2 to the task configuration module.

[0294] S1216, The task configuration module creates a task configuration 2 object (task configuration 2 includes task name 2, snapshot cache 2 object, task group number 2 and preset timeout duration 2).

[0295] For example, the name of task name 2 can be "Generate Video", or it can be other custom fields, which are not limited in this embodiment of the application.

[0296] Task group number 2 can be found in Table 1, which shows the task group number corresponding to the video generation task.

[0297] The preset timeout duration 2 refers to the timeout threshold for generating the video task. The preset timeout duration 2 can be the same as or different from the preset timeout duration 1. This application embodiment does not limit this.

[0298] S1217, The task configuration module returns the task configuration 2 object to the creation module.

[0299] S1218, the authoring module sends the Create Result Notifier instruction 2 to the Result Notifier.

[0300] S1219, The result notifier creates the result notifier 2 object corresponding to task identifier 2.

[0301] For example, the result notification object 2 can store the execution result of the video generation task corresponding to task identifier 2.

[0302] S1220, the result notifyer returns the result notifyer2 object to the authoring module.

[0303] S1221, the authoring module sends a video creation task instruction 2 (carrying a task input 2 object, a task configuration 2 object, and a result notification 2 object) to the task module.

[0304] S1222, the task module creates and generates video task 2 (including task input 2 object, task configuration 2 object, result notifier 2 object and task number 2).

[0305] It should be understood that the process of creating the video generation task 2 is the same as the process of creating the search material task 1, and will not be repeated here.

[0306] S1223, the task module returns the generated video task 2 (including task input 2 object, task configuration 2 object, result notification 2 object and task number 2) to the authoring module.

[0307] The following describes the implementation process of the electronic device performing the video generation task in Scenario 1. Figure 13 This is a timing diagram of an embodiment of the present application for performing a video generation task.

[0308] S1225, the creation module sends the video generation task 2 to the task queue module.

[0309] It should be understood that, Figure 12 In step 1223, the task module returns the video generation task 2 to the creation module. After receiving the video generation task 2, the creation module sends the video generation task 2 to the task queue module to publish the video generation task 2.

[0310] S1226 After receiving the video generation task 2, the task queue module checks whether the task group number 2 of the video generation task 2 is occupied.

[0311] The implementation method for detecting whether task group number 2 of video generation task 2 is occupied can be referred to the above embodiment, and will not be repeated here.

[0312] S1227, The task queue module records the occupancy status of task group number 2 based on the detection results.

[0313] This application embodiment describes the process of an electronic device performing a video generation task in Scenario 1 above. In Scenario 1, task group number 2 for generating video task 2 is not occupied, so the occupation status of task group number 2 recorded in step 1327 is "unoccupied".

[0314] S1228, when the task queue module determines that the occupancy status of task group number 2 is unoccupied, it sends an instruction to the timeout detector to start timing for the video generation task 2.

[0315] S1229, the task queue module will add the generated video task 2 to the end of the task queue.

[0316] It should be understood that when a new task is published to the task queue module through the task module, the task queue module can add the task to the end of the task queue. The newly added task will be executed after the tasks before the end of the queue have been completed.

[0317] In this embodiment of the application, when the task module publishes the video generation task 2 to the task queue module, the search material task 1 in the task queue has been completed. Therefore, the video generation task 2 is located at the first position in the task queue.

[0318] S1230, the task queue module sends task number 2 of generating video task 2 to the creation module.

[0319] S1231, the creation module records the current task number as task number 2.

[0320] S1232, the creation module sends the execution result 3 of the video generation task 2 corresponding to task number 2 to the YOYO assistant based on task number 2 (execution result 3 indicates that the video generation task 2 is being executed in the task queue).

[0321] S1233, YOYO Assistant displays interface 3, including "Analyzing materials", based on execution result 3.

[0322] For example, interface 3 can be referenced. Figure 5 The interface 503 shown includes the text 5033 that reads "Analyzing material".

[0323] S1234, the task queue module schedules and executes the first task in the task queue (i.e., video generation task 2).

[0324] S1235, the task queue module sends the execution instruction to the task module to generate video task 2.

[0325] S1236, the task module receives the execution instruction for generating video task 2 and performs preprocessing on generating video task 2.

[0326] S1237, the task module sends a snapshot processing instruction to the snapshot cache module.

[0327] S1238, the snapshot cache module obtains a collection of materials (which can be a complete video) from the session state module.

[0328] In this embodiment of the application, after the task module has completed the video generation task 2, the task module stores the generated video in the session state module. After receiving the snapshot processing instruction, the snapshot cache module can obtain the video generated by video task 2 from the session state module. Then, the snapshot cache module performs snapshot processing on the video to generate snapshot information of the material set (the snapshot information of the material set can be the snapshot information of the complete video).

[0329] S1239, the snapshot caching module, upon obtaining the material set, performs snapshot processing on the material set to generate snapshot information for the finished video.

[0330] The material set in this application embodiment can be a complete video. The snapshot information of the complete video may include the video template information, music information, duration, cover art, etc.

[0331] S1240, the task module detects whether the video generation task 2 is interrupted.

[0332] The implementation method for detecting whether the video generation task 2 is interrupted can be found in the above embodiment, and will not be repeated here.

[0333] In this embodiment of the application, when the task module executes the video generation task 2, it can call the editing module to execute the task. Therefore, the detection result of the task module in detecting whether the video generation task 2 is interrupted is that the task 2 is not interrupted.

[0334] S1241, if the task module detects that the video generation task 2 is not interrupted, it sends an instruction to the editing module to execute the video generation task 2 (the instruction carries the task input 2 object).

[0335] S1242, after receiving the instruction to execute video generation task 2, the editing module retrieves the material set from the session state module.

[0336] S1243, the editing module executes the video generation task 2 based on the material collection and task input 2 object.

[0337] S1244, the editing module sends the progress of generating video task 2 to the task module.

[0338] It should be understood that the editing module can send the progress of generating video task 2 to the task module at preset intervals. This application embodiment does not limit the specific data for the preset interval.

[0339] The progress of video generation task 2 can be expressed as a percentage, such as 50% or 60%, or as a fraction, such as 4 / 5 or 1 / 3. This application embodiment does not limit this.

[0340] S1245, the task module checks whether video generation task 2 has stopped.

[0341] It should be understood that, in the event of a user inputting a stop command, the task module can detect whether video generation task 2 has stopped.

[0342] S1246, the task module sends the progress of generating video task 2 to the result notification device.

[0343] S1247, The result notification device records the progress of video generation task 2.

[0344] S1248, the result notification device sends the progress of generating video task 2 to YOYO Assistant.

[0345] S1249, YOYO Assistant displays interface 4, including "Analyzing materials", based on the progress of video task 2.

[0346] For example, interface 4 can be referenced. Figure 5 The interface 503 shown includes the text 5033 that reads "Analyzing material".

[0347] S1250, the task module sends the progress of generating video task 2 to the task queue module.

[0348] S1251, the task queue module generates an instruction to update the timing of task 2 based on the progress of video task 2, and sends the instruction to the timeout detector.

[0349] For example, assuming the preset timeout duration 2 is 20ms, after receiving the start timing instruction sent by the task queue module via S1228, the timeout detector starts timing the execution time of the video generation task 2 from 0ms. Originally, if the video generation task 2 has not been completed or the editing module has not sent the progress of the video generation task 2 to the task module at 20ms, the timeout detector sends a timeout notification to the task queue module, notifying that the video generation task 2 has timed out. However, if the task queue module receives the progress of the video generation task 2 sent by the task module via S1250 at 10ms, the task queue module generates an instruction to update the timing of task 2 based on the progress of the video generation task 2. This update timing instruction is used to instruct the timeout detector to restart timing the video generation task 2 from 10ms. If the video generation task 2 has not been completed or the editing module has not sent the progress of the video generation task 2 to the task module at 30ms, the timeout detector sends a timeout notification to the task queue module, notifying that the video generation task 2 has timed out.

[0350] S1252, the editing module sends the status and execution result of video generation task 2 to the task module.

[0351] It should be understood that the status of video generation task 2 can be completed, stopped, or abnormal. For example, if the editing module encounters an error, malfunction, or freezes, it cannot execute video task 2 normally. In this case, the editing module sends the status of video generation task 2 to the task module as abnormal or stopped. The execution result of abnormal or stopped status can be program error or program stoppage. Alternatively, if the editing module successfully completes video generation task 2, it sends the status of video generation task 2 to the task module as completed. The result of normal execution is the generated video.

[0352] In this embodiment, the editing module sends the status of video generation task 2 to the task module as completed, and the execution result is the video generated after video generation task 2 has been executed normally.

[0353] S1253, the task module generates a task creation output instruction (carrying the completed task status) based on the task status of generating video task 2.

[0354] S1254, the task module sends a task output creation instruction (carrying the status of completed tasks) to the task output module.

[0355] S1255, The task output module creates a task output object (the task output object includes the task status of generating video task 2 which has been completed).

[0356] S1256, The task output object model returns a task output object (carrying the status of the completed task) to the task module.

[0357] S1257, the task module generates execution result 4 (including the complete video) based on the task output object.

[0358] S1258, the task module sends an update session status command (carrying the full video) to the session status module.

[0359] S1259, Session status module saves or updates materials.

[0360] For example, the session state module saves the final video generated after video generation task 2 has been successfully executed.

[0361] S1260, the task module checks whether video generation task 2 has stopped.

[0362] The implementation method for detecting whether video generation task 2 has stopped can be referred to the above embodiment, and will not be repeated here.

[0363] In this embodiment of the application, the user did not... Figure 5Clicking the stop control 5034 in the shown interface 503, or receiving a stop command via voice input, causes the task detection module to detect that the video generation task 2 has not stopped.

[0364] S1261, if the task module detects that the video generation task 2 has not stopped, it confirms that the execution result 4 is effective.

[0365] S1262, the task module sends execution result 4 (carrying the full video) to the result notifier based on the result notifier 2 object.

[0366] S1263, The result notifier records the execution result 4 in the result notifier1 object.

[0367] S1264, the result notification device sends execution result 4 (carrying the full video) to YOYO Assistant.

[0368] S1265, YOYO Assistant displays an interface including the complete video based on the execution result 4.

[0369] For example, interface 5 can be referenced. Figure 5 The interface shown is 504. For the finished video, please refer to video 5043 in interface 504.

[0370] S1266, the task module sends a task output object to the task queue module (the task output object includes the task status of the completed video generation task 2).

[0371] S1267, the task queue module cleans up tasks based on the task output object.

[0372] For example, if the task status of generating video task 2 is completed, then the task queue module will clean up the completed generating video task 2, and the task queue will not include generating video task 2.

[0373] S1268, the task queue module sends a stop timing command to the timeout detector for generating video task 2.

[0374] In this embodiment of the application, during the execution of a task by the task module, if the timeout detector does not detect that the execution time of the task has expired, the task queue module sends an instruction to the timeout detector to stop timing the task. After receiving the stop timing instruction, the timeout detector stops timing the video generation task 2.

[0375] The following section describes the execution process of the electronic devices involved in Scenario 2 above.

[0376] Scenario 2 involves an electronic device ceasing task execution when a user issues a stop command via voice or text input during video generation. This application embodiment illustrates this by demonstrating the electronic device ceasing to perform a material search task. Figure 14 This is a timing diagram showing how an electronic device stops performing a material search task, as provided in an embodiment of this application.

[0377] S1401, the creation module sends material search task 1 to the task queue module.

[0378] S1402, after the task queue module receives the material search task 1, it checks whether the task group number 1 of the material search task 1 is occupied.

[0379] S1403, the task queue module records the occupancy status of task group number 1 based on the detection results.

[0380] S1404, when the task queue module determines that the occupancy status of task group number 1 is unoccupied, it sends an instruction to the timeout detector to start timing for material search task 1.

[0381] S1405, the task queue module adds material search task 1 to the end of the task queue.

[0382] S1406, the task queue module sends task number 1 of material search task 1 to the creation module.

[0383] S1407, the creation module records the current task number as task number 1.

[0384] S1408, the creation module sends the execution result 1 of the material search task 1 corresponding to task number 1 to the YOYO assistant based on task number 1 (execution result 1 indicates that material search task 1 is being executed in the task queue).

[0385] S1409, YOYO Assistant displays interface 1, including "Searching for materials," based on execution result 1.

[0386] S1410, the task queue module schedules and executes the first task in the task queue (i.e., material search task 1).

[0387] S1411, the task queue module sends the execution instruction for material search task 1 to the task module.

[0388] S1412, the task module receives the execution instruction for material search task 1 and performs preprocessing on material search task 1.

[0389] S1413, the task module sends a snapshot processing instruction to the snapshot cache module.

[0390] S1414, the snapshot cache module obtains the media collection from the session state module.

[0391] S1415, the snapshot caching module performs snapshot processing on the material set after obtaining the material set, and generates snapshot information of the material set.

[0392] S1416, YOYO Assistant detected the user's operation on the stop control.

[0393] For example, users can... Figure 6 When the user clicks on control 6014 in the interface 601 shown, YOYO Assistant can detect the user's operation on the stop control.

[0394] In other embodiments, users can also input the same stop command to the YOYO Assistant via voice input or text input. After receiving the command, the YOYO Assistant can also display an interface 6 including "This answer has stopped". This application embodiment does not limit this.

[0395] S1417, YOYO Assistant responds to the user's operation on the stop control and displays an interface 6 including "This answer has stopped".

[0396] For example, interface 6 can be referenced. Figure 6 The interface 602 shown includes the text 6022 that reads "This answer has stopped".

[0397] S1418, YOYO Assistant sends a stop command to the creation module to stop the current task.

[0398] In this embodiment of the application, the current task refers to material search task 1.

[0399] S1419, the creation module sends the task number 1 of the current task to the task queue module.

[0400] In this embodiment of the application, task number 1 refers to the task number of material search task 1.

[0401] S1420, the task queue module queries the material search task 1 corresponding to task number 1.

[0402] S1421, the task queue module sends material search task 1 to the task module.

[0403] S1422, the task module changes the status of material search task 1 to stopped.

[0404] S1423, the task module checks whether the material search task 1 has been interrupted.

[0405] S1424, if the task module detects that the material search task 1 is not interrupted, it executes the material search task 1 according to the task input 1 object in the material search task 1.

[0406] It should be understood that the implementation process of the task module executing material search task 1 can be referred to other embodiments, and will not be repeated here.

[0407] S1425, the task module generates a task creation output instruction (carrying the status of completed tasks) based on the task status when executing the task.

[0408] It should be understood that, in this embodiment of the application, although the task module has changed the state of material search task 1 to the stopped state through S1422, the task module still needs to execute material search task 1 normally. Therefore, the task state of material search task 1 in S1425 is the completed state.

[0409] S1426, the task module sends a task output creation command (carrying the status of completed tasks) to the task output module.

[0410] S1427, The task output module creates a task output object (including the task status of completed material search task 1).

[0411] S1428, The task output module returns a task output object (carrying the status of the completed task) to the task module.

[0412] S1429, The task module generates execution result 2 (including multiple images) based on the task output object.

[0413] S1430, the task module sends an update session status command (carrying multiple images) to the session status module.

[0414] S1431, the session status module saves or updates materials.

[0415] S1432, the task module checks whether the material search task 1 has stopped.

[0416] In this embodiment of the application, the user is Figure 6 When the stop control 6014 is clicked in the interface 601 shown, or after the user inputs a stop command via voice, the task module detects that the material search task 1 has stopped.

[0417] S1433, if the task module detects that the material search task 1 has stopped, it confirms that the execution result 2 is not effective.

[0418] S1434, the task module updates the status of material search task 1 in the task output object to the stopped state.

[0419] It should be understood that the task module updates the status of material search task 1 from completed to stopped.

[0420] S1435, the task module sends execution result 5 (carrying stop status information) to the result notifier based on the result notifier 1 object.

[0421] S1436, The result notifier records the execution result 5 in the result notifier1 object.

[0422] S1437, the task module sends a rollback command to the snapshot cache module.

[0423] In this embodiment, after YOYO Assistant detects the user's operation on the stop control in S1418, the task module executes the material search task 1 normally in S1424 and saves the materials searched by material search task 1 in the session state module. For example, the session state module saves 20 pictures of the second daughter dancing. If the task module detects in S1432 that material search task 1 has stopped, it needs to restore the data in the session state module to the state before the task module executed material search task 1 through rollback processing. For example, the 20 pictures of the second daughter dancing saved in the session state module are restored to a state where the material data is empty.

[0424] In implementation, the task module can send a rollback command to the snapshot cache module. The snapshot cache module performs rollback processing based on the snapshot information of the material collection, restoring the data in the session state module to the state before the task module executed the material search task 1.

[0425] S1438, after receiving the rollback instruction, the snapshot cache module performs rollback processing based on the snapshot information of the material collection.

[0426] S1439, the task module sends a task output object to the task queue module (the task output object includes the task status that the material search task 1 has stopped).

[0427] S1440, the task queue module cleans up tasks based on the task output object.

[0428] For example, the task queue module can clear material search task 1 that is in a stopped state in the task queue.

[0429] S1441, the task queue module sends a command to the timeout detector to stop timing the material search task 1.

[0430] In this embodiment of the application, during the execution of a task, if the timeout detector does not detect that the execution time of the task has expired, the task queue module sends an instruction to the timeout detector to stop timing the task. After receiving the stop timing instruction, the timeout detector stops timing the material search task 1.

[0431] It should be understood that Figure 14 This describes the process of stopping the execution of task 1 (material search task 1) while the electronic device is performing it. For the process of stopping other video generation-related tasks (such as video generation or saving tasks), please refer to [reference needed]. Figure 14 .

[0432] The execution process of the electronic devices involved in Scenario 3 above will be described below.

[0433] Scenario 3 involves a situation where an electronic device encounters a task error while performing a video generation-related task (this could be due to the task group number being occupied). This application embodiment uses the example of an electronic device issuing a task to save a video to the task queue, where the task group number for the video saving task is already occupied, as an example for illustration. Figure 15 This is a timing diagram of an electronic device performing a video saving task, as provided in an embodiment of this application.

[0434] S1501, the creation module sends the video saving task 3 to the task queue module.

[0435] It should be understood that before the creation module sends the save video task 3 to the task queue module, the creation module needs to create the save video task 3. The implementation process of the creation module creating the save video task 3 can be found in the above embodiment, and will not be repeated here.

[0436] S1502, after the task queue module receives the video saving task 3, it checks whether the task group number of the video saving task 3 is occupied.

[0437] In implementation, the task queue module can detect whether a newly published task's task group number is already in use by checking if the task group number of an existing task in the task queue is the same as the task group number of the newly published task. For example, referring to the task types and task group numbers shown in Table 1, assuming that a task to generate video already exists in the task queue, and the task group number corresponding to this task is 6, when the task module publishes a new task to the task queue module, for example, the newly published task is to save video, and the task group number corresponding to the save video task is also 6, then the task queue module detects that task group number 6 is already in use.

[0438] S1503, when the task queue module detects that the task group number of video task 3 has been occupied, it records the task status of the task group number of video task 3 as occupied.

[0439] S1504, the task queue module sends a notification to the authoring module that the publishing and saving video task 3 failed (carrying the "occupied" occupancy status information).

[0440] S1505, the creation module sends the task group number of video task 3 to YOYO Assistant, indicating that the task is occupied.

[0441] S1506, YOYO Assistant displays interface 7 based on the occupancy status information.

[0442] For example, interface 7 can be referenced. Figure 7 The interface 702 shown includes the text 7021 that reads "The video has not been completed and cannot be saved".

[0443] It should be understood that after YOYO Assistant displays interface 7 via S1506, the task queue module can clean up the saved video tasks that have been occupied.

[0444] It should also be understood that Figure 15 This describes the process of detecting that the task group number for saving video task 3 is already in use during the electronic device's process of issuing and saving video task 3. For the process of detecting that the task group number for other tasks is already in use during the electronic device's issuance of other tasks, please refer to [link to documentation]. Figure 15 .

[0445] The execution process of the electronic devices involved in Scenario 4 above will be described below.

[0446] Scenario 4 involves an electronic device experiencing a timeout while performing a video generation-related task. This embodiment uses the example of an electronic device experiencing a timeout while performing material search task 1 to illustrate the scenario. Figure 16 This is a timing diagram of an electronic device performing a material search task, as provided in an embodiment of this application.

[0447] S1601, the creation module sends material search task 1 to the task queue module.

[0448] S1602, after the task queue module receives the material search task 1, it checks whether the task group number 1 of the material search task 1 is occupied.

[0449] S1603, the task queue module records the occupancy status of task group number 1 based on the detection results.

[0450] S1604, when the task queue module determines that the occupancy status of task group number 1 is unoccupied, it sends an instruction to the timeout detector to start timing for material search task 1.

[0451] It should be understood that after receiving the instruction to start timing for material search task 1, the timeout detector begins timing for material search task 1.

[0452] S1605, the task queue module adds material search task 1 to the end of the task queue.

[0453] S1606, the task queue module sends task number 1 of material search task 1 to the creation module.

[0454] S1607, the creation module records the current task number as task number 1.

[0455] S1608, the creation module sends the execution result 1 of the material search task 1 corresponding to task number 1 to the YOYO assistant based on task number 1 (execution result 1 indicates that material search task 1 is being executed in the task queue).

[0456] S1609, YOYO Assistant displays interface 1, including "Searching for materials", based on execution result 1.

[0457] S1610, the task queue module schedules and executes the first task in the task queue (i.e., material search task 1).

[0458] S1611, the task queue module sends the execution instruction for material search task 1 to the task module.

[0459] S1612, the task module receives the execution instruction for material search task 1 and performs preprocessing on material search task 1.

[0460] S1613, the task module sends a snapshot processing instruction to the snapshot cache module.

[0461] S1614, the snapshot cache module obtains the media set from the session state module.

[0462] S1615, the snapshot caching module performs snapshot processing on the material set after obtaining the material set, and generates snapshot information of the material set.

[0463] S1616, the task module checks whether the material search task 1 has been interrupted.

[0464] S1617, if the task module detects that the material search task 1 is not interrupted, it executes the material search task 1 according to the task input 1 object in the material search task 1.

[0465] S1618, the timeout detection module confirms that the execution time of material search task 1 is greater than or equal to the preset timeout duration.

[0466] For example, assuming the preset timeout duration is 30ms, after the timeout detection module receives the instruction to start timing for material search task 1 and starts timing for material search task 1 (for example, starting timing from 0ms), the timeout detection module sends a timeout notification to the task module at 30ms or 31ms.

[0467] S1619, the timeout detection module sends a timeout notification to the task module.

[0468] S1620, the task module generates a task creation output instruction (carrying the timed-out task status) based on the task status when executing the task.

[0469] S1621, the task module sends a task output creation command (carrying the timed-out task status) to the task output module.

[0470] S1622, The task output module creates a task output object (including the task status of material search task 1 having timed out).

[0471] S1623, The task output module returns a task output object (carrying the timed-out task status) to the task module.

[0472] S1624, the task module generates execution result 6 (including timeout status) based on the task output object.

[0473] S1625, the task module checks whether material search task 1 has stopped.

[0474] S1626, if the task module detects that the material search task 1 has not stopped, confirm that the execution result 6 has taken effect.

[0475] S1627, the task module sends execution result 6 (carrying timeout status) to the result notifier based on the result notifier 1 object.

[0476] S1628, The result notifier records the execution result 6 in the result notifier1 object.

[0477] S1629, the result notifier sends execution result 6 (with timeout status) to YOYO Assistant.

[0478] S1630, YOYO Assistant displays interface 7 based on execution result 6.

[0479] For example, interface 7 can be referenced. Figure 8 The interface 801 shown displays the text "Finding material has timed out" 8013.

[0480] S1631, the task module sends a task output object to the task queue module (the task output object includes the task status of material search task 1 which has timed out).

[0481] S1632, the task queue module cleans up tasks based on the task output object.

[0482] It should be understood that the task queue module cleans up the search material task 1 that has timed out in the task queue.

[0483] S1633, the timeout detector generates an exception log.

[0484] It should be understood that the exception log includes the reason for the task timeout, the faulty program node, etc.

[0485] It should be understood that Figure 16 This implementation covers the timeout process for Task 1 (searching for source materials). For implementations of timeouts for other video generation-related tasks, please refer to [link / reference]. Figure 16 .

[0486] The execution process of the electronic devices involved in Scenario 5 above will be described below.

[0487] Scenario 5 involves a situation where an electronic device experiences a task interruption while performing a video generation-related task. This embodiment uses the example of an electronic device experiencing a task interruption while performing video generation task 2 as an illustration. Figure 17 This is a timing diagram of another electronic device performing a video generation task, as provided in an embodiment of this application.

[0488] S1701, the creation module sends the video generation task 2 to the task queue module.

[0489] S1702, after the task queue module receives the video generation task 2, it checks whether the task group number 2 of the video generation task 2 is occupied.

[0490] S1703, the task queue module records the occupancy status of task group number 2 based on the detection results.

[0491] S1704, when the task queue module determines that the occupancy status of task group number 2 is unoccupied, it sends a command to the timeout detector to start timing for the video generation task 2.

[0492] S1705, the task queue module will add video task 2 to the end of the task queue.

[0493] S1706, the task queue module sends task number 2 of generating video task 2 to the creation module.

[0494] S1707, the creation module records the current task number as task number 2.

[0495] S1708, the creation module sends the execution result 3 of the video generation task 2 corresponding to task number 2 to the YOYO assistant based on task number 2 (execution result 3 indicates that the video generation task 2 is being executed in the task queue).

[0496] S1709, YOYO Assistant displays interface 3, including "Searching for materials", based on execution result 3.

[0497] S1710, the task queue module schedules and executes the first task in the task queue (i.e., video generation task 2).

[0498] S1711, the task queue module sends the execution instruction to the task module to generate video task 2.

[0499] S1712, the task module receives the execution instruction for generating video task 2 and performs preprocessing on generating video task 2.

[0500] S1713, the task module sends a snapshot processing instruction to the snapshot cache module.

[0501] S1714, the snapshot cache module obtains the media collection from the session state module.

[0502] S1715, the snapshot caching module performs snapshot processing on the material set after obtaining the material set, and generates snapshot information of the material set.

[0503] S1716, the task module detects whether the video generation task 2 is interrupted.

[0504] It should be understood that when the task module executes tasks corresponding to certain task group numbers, it needs to call the editing module in the media platform framework layer to execute the task. For example, referring to Table 1, when the task module executes the task corresponding to task group number 6, it needs to call the editing module to execute the task corresponding to task group number 6. When the task module executes tasks with group numbers 1 to 5 and 7 to 8, it does not need to call the editing module; the task module executes the task itself.

[0505] In implementation, the task module can detect whether a task is interrupted by checking if the editing module can be called. For example, if the editing module has been uninstalled or cannot be called due to an error, when the task module executes the video generation task 6 corresponding to task group number 6 shown in Table 1, the task execution module should normally call the editing module to execute the video generation task 6. However, since the editing module has been uninstalled or cannot be called due to an error, the detection result could be that the video generation task 1 is interrupted.

[0506] S1717, when the task module detects that the video generation task 2 has been interrupted, it generates a task creation output command (carrying the interrupted task status) based on the task status when the task was executed.

[0507] S1718, the task module sends a task output creation command (carrying the status of the interrupted task) to the task output module.

[0508] S1719, The task output module creates a task output object (including the task status of generating video task 2 which has been interrupted).

[0509] S1720, the task output module returns a task output object (carrying the status of the interrupted task) to the task module.

[0510] S1721, the task module generates execution result 7 (including interrupt status) based on the task output object.

[0511] S1722, the task module checks whether video generation task 2 has stopped.

[0512] S1723, if the task module detects that the video generation task 2 has not stopped, it confirms that the execution result 7 is effective.

[0513] S1724, the task module sends execution result 7 (carrying interrupt status) to the result notifier based on the result notifier 2 object.

[0514] S1725, The result notifier records the execution result 7 in the result notifier2 object.

[0515] S1726, the result notifier sends execution result 7 (carrying interrupt status) to YOYO Assistant.

[0516] S1727, YOYO Assistant displays interface 8, including pop-up window 1, based on execution result 7.

[0517] For example, interface 8 can be referenced. Figure 9 The pop-up window 1 shown in interface 901 can be referenced from pop-up window 9011 included in interface 901.

[0518] S1728, the task module sends a task output object to the task queue module (the task output object includes the task status of the interrupted video generation task 2).

[0519] S1729, The task queue module cleans up tasks based on the task output object.

[0520] S1730, the task queue module sends a stop timing command to the timeout detector for the video generation task 2.

[0521] S1731, the task queue module records the task sequence number of the currently interrupted task.

[0522] It should be understood that the task number in this embodiment is the task number for generating video task 2.

[0523] S1732, YOYO Assistant receives user actions on pop-up window 1.

[0524] For example, the user's operation on pop-up window 1 could be that the user can... Figure 9 The operation of clicking control 9012 in the shown interface 901.

[0525] S1733, YOYO Assistant responds to the operation of pop-up window 1 and generates a continuation instruction for the current task.

[0526] It should be understood that the current task refers to generating video task 2.

[0527] S1734, YOYO Assistant sends a continuation command to the task queue module.

[0528] In some embodiments, the YOYO Assistant can send a continuation instruction to the task module, which will then republish the video generation task 2 in the task queue.

[0529] S1735, the task queue module redistributes the interrupted task based on the recorded task sequence number of the currently interrupted task.

[0530] It should be understood that the task queue module can republish the interrupted video generation task 2, after which the electronic device can execute it. Figure 13 Steps S1226 to S1268 in the above.

[0531] It should be understood that Figure 17 This describes the implementation process for an interruption that occurs during the execution of video generation task 2 by the electronic device. The implementation process for interruptions when the electronic device is performing other video generation-related tasks (such as the task corresponding to task group number 6 shown in Table 1) can be found in [reference needed]. Figure 17 .

[0532] The implementation process of a video generation method provided in the embodiments of this application will be described below.

[0533] Please refer to Figure 18 , Figure 18 This is a schematic flowchart illustrating a video generation method provided in an embodiment of this application. This method can be executed by an electronic device, or by a processor or chip within the electronic device; this embodiment does not impose any limitations. For ease of description, an electronic device is used as an example to illustrate the method in detail.

[0534] S1801, the electronic device responds to the first user instruction and generates a first task, wherein the first task is used to instruct the search for a first material.

[0535] It should be understood that the first user instruction can be given by the user via voice or text. Figure 4 In the dialog box 4031 of the YOYO Assistant shown in the user interface 403, the text "Help me generate a video of my second daughter dancing" is entered 4032. Other video generation commands can also be entered by the user, such as "Help me generate a video of my trip to Beijing," etc. The user can enter the first user command via voice or text; this embodiment does not limit this.

[0536] The first task can be the material search task shown in Table 1. The first material can be an image or video. For example, if the first user instruction is "Help me generate a video of my second daughter dancing," the first material can be an image or video including the second daughter dancing. The first material can be referenced... Figure 5 The materials for the second daughter were found in the interface 502 shown.

[0537] In this embodiment of the application, a first user instruction can be input into the speech model, and a first task can be generated through the language model.

[0538] S1802, the electronic device saves the first task to the task queue. The execution order of the tasks in the task queue is based on the order of the user instructions associated with the received tasks.

[0539] For example, users can communicate via voice or text. Figure 4 After the user enters the text "Help me generate a video of my second daughter dancing" in dialog box 4031 of the user interface 403 shown, the electronic device can generate a material search task based on the user's instruction, and then save the material search task to a task queue. The electronic device executes the tasks in the task queue in the order they are arranged. For example, if task one and task two are arranged in the task queue in order, the electronic device will execute task one first, and then execute task two after task one is completed.

[0540] It should be understood that the order of tasks in the task queue is based on the user's instructions. For example, if the user first enters the instruction "Help me generate a video of my second daughter dancing" and then enters the instruction "Generate video", then the material search task generated based on the instruction "Help me generate a video of my second daughter dancing" will be ranked before the video generation task generated based on the instruction "Generate video".

[0541] S1803, the electronic device executes the first task if it detects that the first task group number of the first task does not exist in the task queue.

[0542] It should be understood that the first task group number is used to distinguish the first task. For example, the first task group number can be referred to the task group numbers shown in Table 1. Assuming that the first task is a material search task, its task group number is 1, after the electronic device saves the first task in the task queue, it checks whether task group number 1 exists in the task queue. If task group number 1 does not exist, the material search task is executed.

[0543] S1804, after the first task is completed, the electronic device generates a second task in response to a second user instruction. The second task is used to instruct the generation of a first video based on the first material.

[0544] For example, an electronic device can display [the following information] after the search for materials has been completed. Figure 5 The user interface 502 includes a video generation control 50232. The user can click on the video control 50232. The user's operation can be regarded as the user inputting a second user instruction to the electronic device. The electronic device responds to the second user instruction and generates a video generation task.

[0545] The first video can be used as a reference. Figure 5 The interface 504 shows 5043.

[0546] S1805, the electronic device saves the second task to the task queue.

[0547] It should be understood that in the task queue, the second task is listed after the first task.

[0548] S1806, if the electronic device detects that there is no second task group number for the second task in the task queue, it executes the second task to obtain the first video.

[0549] It should be understood that the second task group number is used to distinguish the second task. For example, the second task group number can be referred to the task group numbers shown in Table 1. Assuming that the second task is to generate a video, and its task group number is 6, after the electronic device saves the second task in the task queue, it checks whether task group number 6 exists in the task queue. If task group number 6 does not exist, the video generation task is executed to obtain the first video.

[0550] In the above technical solution, after receiving a user instruction (e.g., a first user instruction), the electronic device saves the task generated based on the user instruction (e.g., a first task) to a task queue. Then, by executing the tasks in the task queue, a video (e.g., a first video) matching the user instruction is generated. Thus, this application provides a novel method for generating videos. Furthermore, the execution order of the tasks in the task queue follows the order in which the user instructions associated with the task are received. This ensures that the electronic device executes multiple tasks corresponding to multiple user instructions in the task queue in the order in which the user inputs the user instructions. It also ensures that the electronic device executes the task corresponding to the first received user instruction in the task queue after completing the task corresponding to that first received user instruction in the task queue, thereby generating a video that meets the user's needs and improving the user experience.

[0551] Furthermore, before executing a task in the task queue, the electronic device needs to check whether the task group number is occupied (for example, check whether the first task group number of the first task exists in the task queue, and check whether the second task group number of the second task exists in the task queue). If the task group number is not occupied (i.e., if the first task group number of the first task does not exist in the task queue, or if the second task group number of the second task does not exist in the task queue), the task in the task queue is executed, which can ensure that the tasks in the task queue are executed normally.

[0552] In some embodiments, the electronic device generates a first task in response to a first user instruction, including:

[0553] The electronic device responds to a first user instruction by generating a first command, wherein the first command carries a first session identifier and first semantic information corresponding to the first user instruction; and generates a first task based on the first command.

[0554] It should be understood that the session identifier is used to identify the session page when a user inputs session content into the YOYO Assistant while creating a video. Different users create different videos, each with a different session identifier. For an explanation of the session identifier, please refer to the above embodiment; it will not be repeated here.

[0555] Semantic information includes information such as the time, location, person, and event corresponding to the user's voice. A detailed explanation of semantic information can be found in the above embodiment, and will not be repeated here.

[0556] In this embodiment of the application, the first user instruction may be: Figure 10 The user's voice 1 shown in the image, the first command could be... Figure 10The material search command 1 shown can have the following first session identifier: Figure 10 The session identifier 1 shown in the figure may have the following first semantic information: Figure 10 The semantic information shown is 1.

[0557] It should also be understood that electronic devices can input the first user instruction into the language model to generate the first command, and then input the first command into the creation module in the media platform framework layer to generate the first task.

[0558] In other embodiments, the electronic device may generate other tasks related to video generation, such as the video generation task shown in Table 1, the video saving task, etc., based on the implementation of the embodiments of this application.

[0559] In this embodiment of the application, the electronic device can generate a first command based on a first user instruction, and generate a first task based on the first command, which can quickly convert user instructions into tasks.

[0560] In some embodiments, the electronic device generates a first task based on a first command, including:

[0561] The electronic device, based on a first command, creates first task input data, first task configuration data, first result notification, and first task sequence number for a first task. The first task input data includes a first session identifier, first semantic information, and a first task identifier, which is obtained based on the first command. The first configuration data includes a first task name, a first task group number, a first timeout threshold, and the address of a first snapshot cache, all obtained based on the first command. The first result notification records the execution result of the first task, and the first task sequence number is obtained based on the first command. The first task is then generated based on the first task input data, first task configuration data, first result notification, and first task sequence number.

[0562] It should be understood that the implementation process of the electronic device creating the first task input data, the first task configuration data, the first result notification device, and the first task sequence number of the first task, as well as the implementation process of generating the first task based on the first task input data, the first task configuration data, the first result notification device, and the first task sequence number, can be referred to the implementation process of S1009 to S1024, and will not be repeated here.

[0563] In other embodiments, the electronic device may generate other tasks related to video generation, such as the video generation task shown in Table 1, the video saving task, etc., based on the implementation of the embodiments of this application.

[0564] In this embodiment of the application, the electronic device can create first task input data, first task configuration data, first result notification device, and first task sequence number for the first task. Then, based on the first task input data, first task configuration data, first result notification device, and first task sequence number, the first task is generated. Moreover, when generating other tasks (such as the second task), they can also be generated in the above manner, allowing for flexible configuration of different tasks.

[0565] In some embodiments, after the electronic device executes the second task to obtain the first video when it detects that the second task group number of the second task does not exist in the task queue, the method further includes:

[0566] When the electronic device detects that the second task has been stopped, it determines that the first execution result of the second task is invalid. The first execution result is used to indicate the result of the completion of the second task and includes the first video. The first video is rolled back to obtain the second execution result of the second task. The rollback process is used to indicate that the first execution result is restored to the state before the second task was executed.

[0567] It should be understood that the first execution result of the second task refers to the result after the second task is executed normally, and the first execution result includes the generated video. The second execution result refers to the result when the second task is stopped, and the generated video is not included in the second execution result.

[0568] In the implementation, for example, please refer to Figure 14 The electronic device can detect whether the task group number of material search task 1 is occupied through S14002. If it is detected that the task group number of material search task 1 is not occupied (i.e., if it is detected that there is no unoccupied task group number of material search task 1 in the task queue), the electronic device also needs to detect whether material search task 1 has stopped through S1432. If it is detected that material search task 1 has stopped through S1432, it is confirmed through S1433 that the execution result of material search task 1 is invalid (the execution result is that the required material was found, such as "picture of the second daughter dancing"). Then, through S1437 and S1438, the material in the execution result of material search task 1 is rolled back, and the electronic device can restore the data before material search task 1 was executed.

[0569] The above describes the implementation method of this application embodiment using the material search task as an example. When the task is the second task (i.e., the video generation task), the electronic device's implementation method is the same as the above implementation method when it detects that the task has been stopped, and will not be repeated here. It should also be understood that when the task is one of the tasks in Table 1 of the above embodiments, the electronic device's implementation method is the same as the above implementation method when it detects that the task has been stopped, and will not be repeated here.

[0570] In this embodiment of the application, when the second task is detected to be stopped, the electronic device can execute the second task normally, and then roll back the first video to obtain the second execution result of the second task. In this way, when a situation arises that requires stopping the task during the execution of the task, the electronic device can follow the original task execution process without affecting the original task execution process.

[0571] In some embodiments, the method further includes:

[0572] Display a first interface, which includes a stop control; in response to an operation on the stop control, generate a stop command for the second task; based on the stop command for the second task, detect that the second task has been stopped.

[0573] It should be understood that the first interface can be used as a reference. Figure 5 For interface 503, the stop control can be found in control 5034 within interface 503.

[0574] In implementation, the user can click on control 5034. In response to the user's click on control 5034, the electronic device generates a stop command to halt the second task. Upon receiving the stop command, the electronic device can detect that the second task has been stopped.

[0575] In some embodiments, after the electronic device saves the second task to the task queue, the method further includes:

[0576] During the execution of the second task, the electronic device receives instructions from a third user and generates a third task based on those instructions. The third task is used to instruct the editing of the first video. The third task is saved to the task queue. If a third task group number for the third task is detected in the task queue, the third task in the task queue is cleared. A second interface is displayed to indicate that the third task is occupied.

[0577] The third task can be the task corresponding to task identifiers 6 to 10 in Table 1, and the third task can mark the first video.

[0578] In implementation, for example, during the execution of the video generation task (second task), the electronic device receives a third user instruction (e.g., user voice input to generate video). Based on the third user instruction, the electronic device generates a video saving task (third task) and saves the third task to the task queue. At this time, the task queue contains both the video generation task and the video saving task. The task group number for both the video generation task and the video saving task is 6. Therefore, the task group number for the video saving task already exists in the task queue. So, when the electronic device checks whether the task group number for the video saving task is occupied, the detection result is "occupied." The electronic device can then clear the video saving task whose task group number is occupied and display... Figure 7 The interface 7021 shown is the second interface. The second interface displays the text "The video has not been completed and cannot be saved" 7021. This text 7021 is used to indicate that the third task is occupied.

[0579] In this embodiment of the application, when a third task group number of a third task is detected in the task queue, the third task in the task queue is cleaned up, which can simplify the number of tasks in the task queue.

[0580] When the electronic device determines that the third task group number is the same as the second task group number, it detects that the third task group number of the third task exists in the task queue.

[0581] For example, if the second task group number is 6 as shown in Table 1 and the third task group number is also 6 as shown in Table 1, then the electronic device can determine that the third task group number is the same as the second task group number, and the electronic device can detect that the third task group number of the third task exists in the task queue.

[0582] In some embodiments, after the electronic device saves the second task to the task queue, the method further includes:

[0583] When the electronic device detects that the second task has been interrupted, it clears the second task in the task queue; displays a third interface, which includes a first pop-up window; in response to the operation on the first pop-up window, it generates a second command, which is used to instruct the interrupted second task to be re-issued; and saves the second task to the task queue.

[0584] It should be understood that the third interface can be used as a reference. Figure 9 The first pop-up window shown in interface 901 can be referenced from 9011 in interface 901.

[0585] It should also be understood that when electronic devices perform certain tasks, such as the task corresponding to task group number 6 in Table 1, the electronic device needs to call the internal editing application to execute the task. If the editing module has been uninstalled or an error occurs that prevents it from being called, the task will be interrupted.

[0586] Implementation in progress, please refer to Figure 17 Assuming the editing module has been uninstalled or malfunctions, the electronic device can... Figure 17 Step 1716 can detect that the second task has been interrupted, after which the electronic device can clear the second task in the task queue through step S1729. The electronic device can also display... Figure 9 The electronic device can also display the following after the user clicks on 9011 in the first pop-up window 901 shown on the interface 901: Figure 9 The pop-up window 9021 shown in the middle interface 902 allows the user to click on the control 9022 in the pop-up window 9021. In response to the user's operation on the control 9022, a second command is generated. The second command can be... Figure 17 The continuation command mentioned in S1733 is used to instruct the interrupted second task to be reissued.

[0587] In some embodiments, the method is applied to an electronic device including a task queue module and a clipping module, wherein the clipping module is used to perform a second task and save the second task to the task queue, including:

[0588] The electronic device saves the second task to the task queue in the task queue module; after the second task is saved to the task queue in the task queue module, the method further includes: detecting that the second task is interrupted if the task queue module fails to call the clipping module.

[0589] It should be understood that if the editing module has been uninstalled or an error occurs, the editing module cannot be called, and the task queue module may fail to call the editing module.

[0590] In some embodiments, after the electronic device executes the second task to obtain the first video when it detects that the second task group number of the second task does not exist in the task queue, the method further includes:

[0591] The electronic device receives a third user instruction and generates a third task based on the instruction. The third task is used to instruct the editing of the first video. The third task is saved to the task queue. If the execution duration of the third task is detected to be greater than or equal to the second timeout threshold, the third task in the task queue is cleaned up and an exception log is generated.

[0592] In this embodiment of the application, the electronic device can perform timeout detection on tasks, and if a task has timed out, it can clean up the timed-out task and generate an exception log.

[0593] In implementation, it is assumed that when the electronic device detects whether the third task (which may be a video saving task) has timed out, it can detect whether the execution time of the third task is greater than or equal to the timeout threshold. If the execution time of the third task is greater than or equal to the timeout threshold, the third task has timed out.

[0594] In some embodiments, the method is applied to an electronic device including a task queue module and a clipping module to perform a second task, including:

[0595] The electronic device task queue module sends an instruction to the editing module to execute the second task; the editing module executes the second task based on the instruction.

[0596] For example, please refer to Figure 13 The electronic device can send an instruction to the editing module via S1241 to execute the video generation task 2. After receiving the instruction, the editing module can execute the video generation task.

[0597] In some embodiments, the method is applied to an electronic device including a session state module and a snapshot cache module. The session state module is used to save a first material and a first video, and the snapshot cache module is used to perform snapshot processing on the first material to obtain snapshot information of the first material, and to perform snapshot processing on the first video to obtain snapshot information of the first video.

[0598] For example, please refer to Figure 13 After the electronic device completes the task, the image or material generated by the task can be saved in the session state module through S1258. The snapshot cache module can perform snapshot processing on the image or video saved in the session state module to obtain the snapshot information of the image or video.

[0599] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0600] This application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar to those of the related embodiments described above, and will not be repeated here.

[0601] This application provides a readable storage medium containing instructions that, when executed by an electronic device, cause the electronic device to perform the technical solution described in the above embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0602] This application provides a chip for executing instructions. When the chip is running, it executes the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0603] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0604] It should be understood that the term "embodiment" used throughout the specification means that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, various embodiments throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0605] In this application, the use of singular pronouns to denote "one or more" rather than "one and only one," unless otherwise specified. In this application, unless otherwise specified, "at least one" is intended to mean "one or more," and "more than" is intended to mean "two or more."

[0606] In this document, the terms "at least one of..." or "at least one of..." refer to all or any combination of the listed items. For example, "at least one of A, B, and C" can mean: A exists alone, B exists alone, C exists alone, A and B exist simultaneously, B and C exist simultaneously, and A, B, and C exist simultaneously. A can be singular or plural, B can be singular or plural, and C can be singular or plural.

[0607] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0608] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0609] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0610] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0611] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0612] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0613] The same or similar parts between the various embodiments in this application can be referred to mutually. In the various embodiments of this application, and in the various implementation methods / methods / implementations within each embodiment, unless otherwise specified or logically conflicting, the terminology and / or descriptions between different embodiments and between the various implementation methods / methods / implementations within each embodiment are consistent and can be mutually referenced. The technical features in different embodiments and the various implementation methods / methods / implementations within each embodiment can be combined according to their inherent logical relationships to form new embodiments, implementation methods, methods, or implementation approaches. The above-described embodiments of this application do not constitute a limitation on the scope of protection of this application.

[0614] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims. In conclusion, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method of video generation, the method comprising: include: In response to a first user instruction, a first command is generated, wherein the first command carries a first session identifier and first semantic information corresponding to the first user instruction; Based on the first command, the first task input data, the first task configuration data, the first result notification, and the first task sequence number of the first task are created. The first task input data includes the first session identifier, the first semantic information, and the first task identifier, which is obtained based on the first command. The first configuration data includes the first task name, the first task group number, the first timeout threshold, and the address of the first snapshot cache, which are obtained based on the first command. The first result notification is used to record the execution result of the first task, and the first task sequence number is obtained based on the first command. The first task is generated based on the first task input data, the first task configuration data, the first result notifier, and the first task sequence number, wherein the first task is used to indicate the search for the first material; The first task is saved to a task queue, and the execution order of the tasks in the task queue is according to the order in which the user instructions associated with the task are received. If the first task group number of the first task does not exist in the task queue, the first task is executed, whereby the first task group number is used to distinguish the first task. Upon completion of the first task, in response to a second user instruction, a second task is generated, which instructs the generation of a first video based on the first material. Save the second task to the task queue; If the second task group number of the second task does not exist in the task queue, the second task is executed to obtain the first video. The second task group number is used to distinguish the second task.

2. The method according to claim 1, characterized in that, If the second task group number of the second task does not exist in the task queue, and the second task is executed to obtain the first video, the method further includes: If the second task is detected to have been stopped, it is determined that the first execution result of the second task is invalid. The first execution result is used to indicate the result of the completion of the second task, and the first execution result includes the first video. The first video is rolled back to obtain the second execution result of the second task. The rollback process is used to indicate that the first execution result is restored to the state before the second task was executed.

3. The method according to claim 2, characterized in that, The method further includes: Display a first interface, which includes a stop control; In response to an operation on the stop control, a stop command for the second task is generated; Based on the stop command of the second task, it was detected that the second task had been stopped.

4. The method according to claim 1, characterized in that, After saving the second task to the task queue, the method further includes: During the execution of the second task, a third user instruction is received, and a third task is generated based on the third user instruction, the third task being used to instruct the editing of the first video; Save the third task to the task queue; If the third task group number of the third task is detected in the task queue, the third task in the task queue is cleaned up. A second interface is displayed, which indicates that the third task is occupied.

5. The method according to claim 4, characterized in that, The method further includes: If it is determined that the third task group number is the same as the second task group number, the third task group number of the third task is detected in the task queue.

6. The method according to claim 1, characterized in that, After saving the second task to the task queue, the method further includes: If the interruption of the second task is detected, the second task in the task queue is cleaned up; A third interface is displayed, which includes a first pop-up window; In response to the operation of the first pop-up window, a second command is generated, which is used to instruct the interrupted second task to be re-published; Save the second task to the task queue.

7. The method according to claim 6, characterized in that, The method is applied to an electronic device including a task queue module and an editing module, wherein the editing module is used to execute the second task, and saving the second task to the task queue includes: Save the second task to the task queue in the task queue module; After the second task is saved to the task queue in the task queue module, the method further includes: If the task queue module fails to call the editing module, the second task is detected to be interrupted.

8. The method according to claim 1, characterized in that, If the second task group number of the second task does not exist in the task queue, and the second task is executed to obtain the first video, the method further includes: Upon receiving a third user instruction, and based on the third user instruction, a third task is generated, the third task being used to instruct the editing of the first video; Save the third task to the task queue; If the execution time of the third task is detected to be greater than or equal to the second timeout threshold, the third task in the task queue is cleaned up and an exception log is generated.

9. The method according to any one of claims 1 to 8, characterized in that, The method is applied to an electronic device including a task queue module and a clipping module, wherein executing the second task includes: The task queue module sends an instruction to the editing module to execute the second task; The editing module executes the second task based on the instructions of the second task.

10. The method according to any one of claims 1 to 8, characterized in that, The method is applied to an electronic device including a session state module and a snapshot cache module. The session state module is used to save the first material and the first video. The snapshot cache module is used to perform snapshot processing on the first material to obtain snapshot information of the first material, and to perform snapshot processing on the first video to obtain snapshot information of the first video.

11. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 10.

12. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 10.

Images