Video generation method, electronic equipment, readable storage medium and chip

Abstract

The invention provides a video generation method, electronic equipment, a readable storage medium and a chip. According to the method, after the user instruction is received, the task generated based on the user instruction is stored in the task queue, and the video matched with the user instruction is generated by executing the task in the task queue, namely, the invention provides a brand-new method for generating the video. Besides, the execution sequence of the tasks in the task queue is executed according to the sequence of the received user instructions associated with the tasks, so that the electronic equipment can be ensured to execute the tasks corresponding to the user instructions received first in the task queue after executing the tasks corresponding to the user instructions received first in the task queue. And then executing a task corresponding to the user instruction received after the first received user instruction in the task queue, thereby generating a video meeting user requirements.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a video generation method, electronic equipment, a readable storage medium, and a chip. Background Art

[0002] Currently, many electronic devices are equipped with cameras. Users can use these cameras to take photos and videos, or they can obtain images or videos from the internet or other electronic devices. With the continuous development of smart devices, users' demand for and experience with images and videos is also increasing. In some scenarios, users need to generate videos on related topics based on images or videos, so that they can easily view the highlights of the videos. Summary of the Invention

[0003] The present application provides a video generation method, electronic device, readable storage medium and chip, which can generate videos that meet user needs and improve user experience.

[0004] In a first aspect, a video generation method is provided, the method comprising:

[0005] In response to a first user instruction, a first task is generated, wherein the first task is used to instruct a search for a first material; the first task is saved in a task queue, and the execution order of tasks in the task queue is executed in the order of the received user instructions associated with the tasks; when it is detected that the first task group number of the first task does not exist in the task queue, the first task is executed, and the first task group number is used to distinguish the first task; when the first task is completed, a second task is generated in response to a second user instruction, and the second task is used to instruct generation of a first video based on the first material; the second task is saved in the task queue; when it is detected that the second task group number of the second task does not exist in the task queue, the second task is executed to obtain the first video, and the second task group number is used to distinguish the second task.

[0006] In the above technical solution, after the electronic device receives a user instruction (for example, a first user instruction), it saves the task (for example, the first task) generated based on the user instruction into a task queue. Thereafter, by executing the task in the task queue, a video (for example, the first video) that matches the user instruction is generated. That is, the present application provides a new method for generating a video. In addition, the execution order of the tasks in the above-mentioned task queue is executed in the order of the received user instructions associated with the task. In this way, it can be ensured that the execution order of the multiple tasks corresponding to the multiple user instructions in the task queue by the electronic device is executed in the order of the user instructions input by the user, and it can be ensured that after the electronic device completes the execution of the task corresponding to the first received user instruction in the task queue, it executes the task corresponding to the user instruction received after the first received user instruction in the task queue, thereby generating a video that meets the user's needs and improves the user experience.

[0007] Moreover, before executing a task in a task queue, the electronic device needs to detect whether the task group number of the task is occupied (for example, detecting whether there is a first task group number of the first task in the task queue, and detecting whether there is a second task group number of the second task in the task queue). If it is detected that the task group number of the task is not occupied (that is, when it is detected that there is no first task group number of the first task in the task queue, or when it is detected that there is no second task group number of the second task in the task queue), the task in the task queue is executed, which can ensure that the tasks in the task queue are executed normally.

[0008] In conjunction with the first aspect, in certain implementations of the first aspect, generating a first task in response to a first user instruction includes:

[0009] In response to a first user instruction, a first command is generated, wherein the first command carries a first session identifier and first semantic information corresponding to the first user instruction; and based on the first command, a first task is generated.

[0010] In the embodiment of the present application, the electronic device can generate a first command according to a first user instruction, and generate a first task based on the first command, so that the user instruction can be quickly converted into a task.

[0011] In conjunction with the first aspect, in certain implementations of the first aspect, generating the first task based on the first command includes:

[0012] Based on the first command, first task input data of the first task, first task configuration data of the first task, a first result notifier of the first task and a first task sequence number of the first task are created, wherein the first task input data includes a first session identifier, first semantic information and a first task identifier, the first task identifier is obtained based on the first command, the first configuration data includes a first task name, a first task group number, a first timeout threshold and an address of a first snapshot cache, the first task name, the first task group number, the first timeout threshold and the address of the first snapshot cache are obtained based on the first command, the first result notifier is used to record the execution result of the first task, and the first task sequence number is obtained based on the first command; the first task is generated according to the first task input data, the first task configuration data, the first result notifier and the first task sequence number.

[0013] In an embodiment of the present application, the electronic device can create first task input data of the first task, first task configuration data of the first task, a first result notifier of the first task and a first task serial number of the first task, and then generate the first task based on the first task input data, the first task configuration data, the first result notifier and the first task serial number. Moreover, when generating other tasks (such as the second task), they can also be generated in the above manner, and different tasks can be flexibly configured.

[0014] In conjunction with the first aspect, in certain implementations of the first aspect, after detecting that the second task group number of the second task does not exist in the task queue and executing the second task to obtain the first video, the method further includes:

[0015] When it is detected that the second task is stopped, it is determined that the first execution result of the second task is not effective. The first execution result is used to indicate the result of the completion of the second task, and the first execution result includes the first video; the first video is rolled back to obtain the second execution result of the second task, and the rollback process is used to indicate that the first execution result is restored to the state before the second task is executed.

[0016] In an embodiment of the present application, when it is detected that the second task is stopped, the electronic device can execute the second task normally, and then roll back the first video to obtain a second execution result of the second task. In this way, when a situation arises during the execution of the task that requires stopping the task, the electronic device can follow the original task execution process without affecting the original task execution process.

[0017] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes:

[0018] A first interface is displayed, wherein the first interface includes a stop control; in response to an operation on the stop control, a stop instruction for the second task is generated; and based on the stop instruction for the second task, it is detected that the second task is stopped.

[0019] In combination with the first aspect, in some implementations of the first aspect, after saving the second task to the task queue, the method further includes:

[0020] During the execution of the second task, a third user instruction is received, and based on the third user instruction, a third task is generated, and the third task is used to instruct the editing of the first video; the third task is saved in the task queue; when the third task group number of the third task is detected in the task queue, the third task in the task queue is cleared; and a second interface is displayed, and the second interface is used to indicate that the third task is occupied.

[0021] In the embodiment of the present application, when it is detected that the third task group number of the third task exists in the task queue, the third task in the task queue is cleared, which can simplify the number of tasks in the task queue.

[0022] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes:

[0023] When it is determined that the third task group number is the same as the second task group number, it is detected that the third task group number of the third task exists in the task queue.

[0024] In combination with the first aspect, in some implementations of the first aspect, after saving the second task to the task queue, the method further includes:

[0025] When it is detected that the second task is interrupted, clearing the second task in the task queue;

[0026] A third interface is displayed, which includes a first pop-up window. In response to an operation on the first pop-up window, a second command is generated, where the second command is used to instruct the interrupted second task to be re-issued. The second task is saved in a task queue.

[0027] In conjunction with the first aspect, in certain implementations of the first aspect, the method is applied to an electronic device including a task queue module and a clipping module, wherein the clipping module is configured to execute the second task and save the second task to the task queue, including:

[0028] The second task is saved in the task queue of the task queue module; after the second task is saved in the task queue of the task queue module, the method further includes: if the task queue module fails to call the clipping module, detecting that the second task is interrupted.

[0029] In conjunction with the first aspect, in certain implementations of the first aspect, after detecting that the second task group number of the second task does not exist in the task queue and executing the second task to obtain the first video, the method further includes:

[0030] A third user instruction is received, and based on the third user instruction, a third task is generated, where the third task is used to instruct editing of the first video; the third task is saved in a task queue; and when it is detected that the execution time of the third task is greater than or equal to a second timeout threshold, the third task in the task queue is cleared and an exception log is generated.

[0031] In conjunction with the first aspect, in certain implementations of the first aspect, the method is applied to an electronic device including a task queue module and a clipping module, and performing the second task includes:

[0032] The task queue module sends an instruction to execute the second task to the editing module; and the editing module executes the second task based on the instruction of the second task.

[0033] In combination with the first aspect, in certain implementations of the first aspect, the method is applied to an electronic device including a session state module and a snapshot cache module, the session state module is used to save the first material and the first video, the snapshot cache module is used to perform snapshot processing on the first material to obtain snapshot information of the first material, and to perform snapshot processing on the first video to obtain snapshot information of the first video.

[0034] In a second aspect, an electronic device is provided, wherein the electronic device is configured to execute the method provided in the first aspect. Specifically, the electronic device may include a processing unit configured to execute any possible implementation of the first aspect.

[0035] In a third aspect, an electronic device is provided, comprising: one or more processors; one or more memories; the one or more memories storing one or more computer programs, the one or more computer programs comprising instructions, which, when executed by the one or more processors, enable the electronic device to execute a method in any possible implementation of the first aspect.

[0036] In a fourth aspect, a computer-readable storage medium is provided, comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method described in the first aspect.

[0037] In a fifth aspect, a chip is provided, comprising a memory for storing instructions; and a processor for calling and executing instructions from the memory, so that an electronic device equipped with the chip executes the method described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 1 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application.

[0039] Figure 2 Schematic diagram of the software system of the electronic device 100 according to an embodiment of the present application.

[0040] Figure 3 It is a set of graphical user interfaces provided by the embodiments of this application.

[0041] Figure 4 This is another set of graphical user interfaces provided by the embodiments of the present application.

[0042] Figure 5 This is another set of graphical user interfaces provided by the embodiments of the present application.

[0043] Figure 6 This is another set of graphical user interfaces provided by the embodiments of the present application.

[0044] Figure 7 This is another set of graphical user interfaces provided by the embodiments of the present application.

[0045] Figure 8 This is another set of graphical user interfaces provided by the embodiments of the present application.

[0046] Figure 9 This is another set of graphical user interfaces provided by the embodiments of the present application.

[0047] Figure 10 This is a timing diagram of creating a material search task provided by an embodiment of the present application.

[0048] Figure 11 This is a timing diagram for executing a material search task provided in an embodiment of the present application.

[0049] Figure 12 This is a timing diagram for executing a material search task provided in an embodiment of the present application.

[0050] Figure 13 This is a timing diagram for executing a material search task provided in an embodiment of the present application.

[0051] Figure 14 This is a timing diagram of an electronic device stopping executing a material search task provided by an embodiment of the present application.

[0052] Figure 15 This is a timing diagram of an electronic device performing a video saving task provided by an embodiment of the present application.

[0053] Figure 16 This is a timing diagram of an electronic device performing a material search task provided by an embodiment of the present application.

[0054] Figure 17 This is a timing diagram of another electronic device performing a video generation task provided by an embodiment of the present application.

[0055] Figure 18 This is a schematic flowchart of a video generation method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" or "multiple" refers to two or more than two.

[0057] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0058] The video generation method provided in the embodiments of the present application is applied to electronic devices. The electronic device may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The electronic device may be an electronic device (mobile phone), a smart TV, a wearable device, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless terminal in industrial control (industrial control), a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in smart grid (smart grid), a wireless terminal in transportation safety (transportation safety), a wireless terminal in a smart city (smart city), a wireless terminal in a smart home (smart home), etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the electronic device.

[0059] Figure 1 1 is a schematic diagram of the structure of an electronic device 100 (taking a mobile phone as an example) provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, an antenna 1, an antenna 2, a mobile communication module 140, a wireless communication module 150, an audio module 160, a sensor module 170, a camera 181, a display screen 182, etc. The sensor module 170 may include a pressure sensor 170A, a gyroscope sensor 170B, etc.

[0060] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0061] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0062] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0063] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0064] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0065] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 140, the wireless communication module 150, the modem processor and the baseband processor.

[0066] Electronic device 100 implements display functionality through a GPU, display screen 182, and an application processor. The GPU is a microprocessor for image processing that connects display screen 182 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0067] Display screen 182 is used to display images, videos, and the like. Display screen 182 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 182, where N is a positive integer greater than one.

[0068] The electronic device 100 can implement a shooting function through an ISP, a camera 181, a video codec, a GPU, a display screen 182, and an application processor.

[0069] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.

[0070] The internal memory 121 may be used to store computer executable program codes, which include instructions. The processor 110 executes the instructions stored in the internal memory 121 to execute various functional applications and data processing of the electronic device 100.

[0071] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 160 and the application processor.

[0072] The pressure sensor 170A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0073] The gyro sensor 170B may be used to determine the motion posture of the electronic device 100 .

[0074] This concludes the introduction to the hardware structure of the electronic device 100. It is understood that Figure 1 The components included in the illustrated hardware structure do not constitute a specific limitation on the electronic device 100. The electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in the figure may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0075] Furthermore, operating systems run on the above components, such as the iOS operating system developed by Apple, the Android open-source operating system developed by Google, and the Windows operating system developed by Microsoft. Application programs can be installed and run on these operating systems.

[0076] The operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software system of the electronic device 100.

[0077] Figure 2Schematic diagram of the software system of electronic device 100 according to an embodiment of the present application. The software system comprises several layers, each with distinct roles and divisions of labor, and communication between layers via software interfaces. In some embodiments, the software system is divided from top to bottom into an application layer, a media platform framework layer, an application framework layer, a system library layer, and a hardware layer.

[0078] The application layer can include a series of application packages, such as camera, gallery, calendar, map, navigation, language model, editing application, YOYO assistant, etc. When these application packages are running, they can access the various service modules provided by the media platform framework layer through the application programming interface (API) and execute corresponding services.

[0079] For example, YOYO Assistant can receive user instructions, call the voice model to analyze the user instructions, and the voice model returns the analysis results to YOYO Assistant. Based on the analysis results, YOYO Assistant calls the service module in the media middle platform framework layer, such as the editing module, to perform the editing business.

[0080] The media platform framework layer is a software layer located between the application layer and the application framework layer. It can include an authoring module, a task configuration module, a task module, a task queue module, a result notifier, a timeout detector, a session state module, a snapshot cache module, a task input module, a task output module, and an editing module.

[0081] The creation module is used to create tasks related to video generation and publish the created tasks to the task queue module. Tasks related to video generation can be material search tasks, video generation tasks, character confirmation tasks, etc.

[0082] The task configuration module is used to create configuration data for a task, which includes the name of the task, the task group number to which the task belongs, the preset timeout duration of the task, etc.

[0083] The task input module is used to create input data for the task. The input data includes a task identifier, a session identifier, and semantic information. For the specific meanings of the task identifier, session identifier, and voice information, please refer to other embodiments and will not be repeated here.

[0084] The snapshot cache module is used to create a snapshot cache of the material. The specific meaning of the snapshot cache of the material can be referred to other embodiments and will not be repeated here.

[0085] The result notifier is used to store the execution result of the task. The result of the task can be notified to the YOYO assistant in the application layer through the result notifier.

[0086] The task module is used to create tasks. The created tasks include task input data, task configuration data, result notifier and task sequence number, etc.

[0087] The timeout detector is used to detect whether the task execution has timed out.

[0088] The session state module is used to store the results of task execution. For example, if the result of a material search task is that several materials are found in the gallery, the session state module is used to store these materials.

[0089] The task output module is used to create the output data of the task. The output data includes the execution status of the task, which can be normal, stopped, occupied, etc.

[0090] The editing module is used to perform tasks related to video generation. For example, the editing module performs tasks such as generating a video, saving a video, and changing music.

[0091] The application framework layer, also referred to as the framework layer, provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0092] The application framework layer may include a window manager, content provider, view system, resource manager, etc.

[0093] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0094] Content providers are used to store and retrieve data and make it accessible to applications. These data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0095] The view system includes visual controls, such as those for displaying text and images. It can be used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0096] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0097] The system library can include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0098] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0099] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0100] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0101] A 2D graphics engine is a drawing engine for 2D drawings.

[0102] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0103] It is understandable that Figure 2 The layers in the software system shown and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer layers than shown, and each layer may include more or fewer components.

[0104] It is understandable that, in order to implement the video generation method in the embodiment of the present application, the electronic device includes hardware and / or software modules that perform the corresponding functions. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments.

[0105] It should be noted that although the embodiments of the present application are described using the Android system as an example, its basic principles are also applicable to electronic devices based on operating systems such as iOS or Windows.

[0106] The video generation method provided by the embodiment of the present application is described in detail below with reference to the accompanying drawings.

[0107] In some embodiments, a mobile phone can provide functions of image search and video creation through a voice assistant application (hereinafter simply referred to as the voice assistant), such as YOYO. In practice, it is not limited thereto. Exemplarily, the mobile phone can also provide functions of image search and video creation through other applications, such as a dedicated video editing application, a gallery application, etc.

[0108] Hereinafter, mainly taking the voice assistant as an example, the solution of the present application will be described. It can be understood that if it is other applications, the implementation principle is similar, and it will not be elaborated in the embodiments of the present application.

[0109] The mobile phone can provide at least one of the following ways to trigger the mobile phone to start image search.

[0110] Way 1: After waking up the voice assistant, input an image search request.

[0111] Among them, the waking-up methods of the voice assistant include but are not limited to at least one of the following: Waking-up method 1, keyword waking-up: When the mobile phone recognizes that the user inputs a waking-up voice including a keyword, such as "Hello YOYO", the voice assistant can be woken up; Waking-up method 2, when the mobile phone detects that the user long-presses the power button, the voice assistant can be woken up; Waking-up method 3, breath waking-up: When the mobile phone detects that the user raises the mobile phone and puts the mouth against the microphone, the voice assistant can be woken up. The embodiments of the present application do not specifically limit the waking-up methods.

[0112] Taking the waking-up voice as "Hello YOYO" as an example, refer to Figure 3 , in the case of the mobile phone display interface 301, in response to the input waking-up voice "Hello YOYO", the mobile phone can wake up the voice assistant. After waking up the voice assistant, the mobile phone can display interface 302. Interface 302 includes a prompt card 3021 of the voice assistant, indicating that the voice assistant has been woken up.

[0113] After the mobile phone wakes up the voice assistant through the above method, the voice assistant is in the waking-up state. In the case where the voice assistant is in the waking-up state, the mobile phone can receive the user's voice input. For example, the user can input the voice "Images of the second daughter dancing". Or, the mobile phone can also receive the user's text input. For example, Figure 3 as shown in the prompt card 3021 in interface 301, which includes a keyboard control 3022. In response to the user's triggering operation (such as a click operation) on the keyboard control 3022, the mobile phone can provide a keyboard for the user to input text.

[0114] In response to the user inputting voice or text indicating image search (i.e., inputting an image search request), the mobile phone can display a dialog box of the voice assistant (hereinafter simply referred to as the dialog box), and the dialog box includes the image search request.

[0115] Taking the voice indicating image search as "Images of the second daughter dancing" as an example, continue to refer to Figure 3 In the case where the mobile phone displays interface 302, in response to the user inputting the voice "image of the second daughter dancing", the mobile phone can display interface 303, which includes a dialog box 3031 of the voice assistant. The dialog box 3031 includes a dialogue "image of the second daughter dancing" 3032.

[0116] Furthermore, in response to the user inputting a picture search request, the mobile phone may also start searching the gallery application for image materials that match the picture search request, ie, triggering the start of the picture search.

[0117] At this point, it should be noted that: in the embodiment of the present application, the format of the image search request input by the user through text or voice can be arbitrary. Taking the search for images of the second daughter dancing as an example, the search request can be "images of the second daughter dancing", "search for images of the second daughter dancing", "including images of the second daughter dancing", etc. It is understandable that creating videos usually requires finding images. Therefore, the format of the image search request can also be "generate / create a video of...", such as generating a birthday blessing video for Jimmy, generating a video of a trip to Beijing, etc. In practice, the mobile phone can limit the voice or text image search request to one or more of the above-mentioned fixed formats, such as a fixed format of "generate a video of...", so as to facilitate identification of user needs. In the following, this will not be repeated.

[0118] Method 2: Enter the voice assistant dialog box and enter the image search request.

[0119] A voice assistant icon is displayed on the desktop of the mobile phone. In response to a user triggering operation on the voice assistant icon, such as a click operation, the mobile phone can display a dialog box.

[0120] See also Figure 4 , the mobile phone can display interface 401. Interface 401 is the desktop of the mobile phone. Interface 401 includes a voice assistant icon 4011. In response to the user clicking icon 4011, the mobile phone can display interface 402, which includes a dialog box 4021.

[0121] In dialog box 4021, the user can also enter a search request. In response to the input indicating the search request in the dialog box, the mobile phone can also display the search request in the dialog box and search for image materials, that is, trigger the start of the search.

[0122] Still taking the voice command to search for images as "images of the second daughter dancing" as an example, continue to refer to Figure 4 In the case where the mobile phone displays interface 402, in response to the user inputting the voice "image of the second daughter dancing", the mobile phone may display interface 403 (same as interface 303 above), which also includes a dialog box 4031. Dialog box 4031 further includes a dialogue "image of the second daughter dancing" 4032.

[0123] It should be noted that the dialog box in the above-mentioned method 1 (such as dialog box 3031) and the dialog box in the method 2 (such as dialog box 4031) can be the same dialog box. Therefore, in the dialog box of method 1, the mobile phone can also start searching for images in response to the user inputting the image search request. This will not be repeated here.

[0124] It should be understood that the above Figure 3 and Figure 4 The user interface of the mobile phone shown for triggering the mobile phone to start searching for images is for illustration only and does not constitute any limitation on the user interface for triggering the electronic device to start searching for images involved in the video generation method provided in the embodiment of the present application. For example, in some implementations, in response to a user triggering an operation, such as a click operation, on a creation recommendation card displayed on any user interface of the mobile phone, the mobile phone may display a dialog box for video creation (also referred to as a dialog box for short) and start a video creation conversation in the dialog box. Any user interface may be a desktop, a negative screen, a lock screen interface, an application interface of a video application, an application interface of a chat application, and the like. In the following, the present application scheme is mainly explained by taking the mobile phone displaying a creation recommendation card on the desktop as an example. The creation recommendation card displayed on the user interface of the mobile phone can be created in the following way: after the mobile phone recognizes that at least one of the current time, current location, and images in the gallery application meets the conditions for video creation, it displays the creation recommendation card.

[0125] Next, combine Figures 5 to 9 A schematic diagram illustrating a scenario of a video generation method provided in an embodiment of the present application.

[0126] In an embodiment of the present application, during the process of an electronic device executing video generation, the electronic device may execute tasks related to video generation without specific limitation. For example, tasks related to video generation may include, but are not limited to, one or more of the following tasks: material search task, character confirmation task, material addition task, material deletion task, material selection task, video generation task, template change task, music change task, duration change task, and video saving task.

[0127] When the electronic device is performing the above tasks, the execution result of each task can be the following states: task completion state (recorded as scenario one), task stop state (recorded as scenario two), task error state (recorded as scenario three), task timeout state (recorded as scenario four), and task interruption state (recorded as scenario five). Of course, in other embodiments, the execution result of each task can also be other states, and the embodiments of this application are not limited to this.

[0128] The following describes the user interfaces of the electronic devices involved in the above scenarios 1 to 5 respectively.

[0129] Scenario 1: During the process of an electronic device executing the video generation method provided in an embodiment of the present application, the electronic device can perform the task normally, and during the execution of the task, the task does not stop, time out, be interrupted, or have errors.

[0130] In one example, the user interface of the electronic device involved in the video generation method provided in the embodiment of the present application is introduced by taking the example of a voice assistant in an electronic device generating a target video directly based on the searched material after receiving the user's voice. Figure 4 , users can use voice or text to Figure 4 After inputting the text 4032 of "Help me generate a video of my second daughter dancing" in the dialog box 4031 of the user interface 403, the electronic device displays Figure 5 The user interface 501 is shown. Figure 5 As described, a dialog box 5011 is displayed in the user interface 501, and a text 5013 of "Searching for materials for you..." and a stop control 5014 are displayed in the dialog box 5011, wherein the stop control 5014 is used to trigger the electronic device to stop the current task of searching for materials. When the electronic device displays the user interface 501 to the user and the user does not trigger the stop control 5014, the electronic device will display the search results to the user, and the search results may include a text 5022 of "The following materials of the second daughter are selected for you" displayed in the dialog box 5021 in the user interface 502, and a thumbnail 5023 of the displayed material. The thumbnail 5023 of the material displays a view all control 50321, as shown in FIG. Figure 5 The view all control 50321 shown indicates that 15 materials out of the 20 materials are selected. Optionally, the user can also increase or delete the number of materials displayed by the thumbnails 5023 by triggering the view all control 50321.

[0131] The electronic device displays Figure 5 After the user interface 502 is displayed, the user can trigger the generate video control 50232 displayed in the material card 5023 in the user interface 502 to trigger the electronic device to generate the target video. Specifically, after the user triggers the generate video control 50232, the electronic device displays the following Figure 5 User interface 503 is shown. Dialog box 5031 displayed in user interface 503 includes text 5032 reading "Generating Video," text 5033 reading "Analyzing Material...", and a stop control 5034. Text 5033 indicates the progress of the video, and stop control 5034 is used to trigger the electronic device to stop the current video generation task. For example, the content in text 5033 reading "Analyzing Material..." can be replaced with, for example, "Processing, please wait..."

[0132] When the electronic device displays the user interface 503 to the user and the user does not trigger the stop control 5034 displayed in the user interface 503, the electronic device will display the following to the user: Figure 5 The user interface 504 is shown. The dialog box 5041 displayed in the user interface 504 displays the execution result corresponding to the generate video task, and the execution result may include a text 5042 displayed in the dialog box 5041 in the user interface 504, "The video has been produced. Come and enjoy the little cutie's dance~" and a thumbnail 5043 of the target video.

[0133] After the video is created, the electronic device can perform editing controls such as saving and deleting the video.

[0134] In some implementations, the thumbnail 5043 of the target video may also display editing controls. Exemplarily, the editing controls include at least one of the following: a "save" control 50431, a "delete" control, a "like" control, a "dislike" control, etc., as displayed in the user interface 504.

[0135] Scenario 2: During the process of the electronic device executing the video generation method provided in the embodiment of the present application, the electronic device stops executing the task.

[0136] In one example, taking the situation where a voice assistant in an electronic device receives a user voice and stops the task while performing a search for materials as an example, the user interface of the electronic device involved in the video generation method provided in an embodiment of the present application is introduced.

[0137] In practical applications, the electronic device displays the above Figure 5 After the user interface 501 is displayed, if the user needs to stop the electronic device from performing the task of searching for materials, the user can input a stop instruction to the voice assistant through voice or text. For example, the user can click Figure 6 The stop control 6014 shown in the user interface 601 in FIG. 1 is displayed. In response to the user clicking the stop control 6014, the electronic device displays Figure 6 In the user interface 602 shown, a text 6022 reading “This answer has been stopped” is displayed in a dialog box 6021 of the user interface 602 .

[0138] Scenario 3: During the process of an electronic device executing the video generation method provided in an embodiment of the present application, an error occurs in the task performed by the electronic device.

[0139] In one example, after the voice assistant in the electronic device receives the user's voice, the electronic device performs the task of saving the video during the process of performing the task of generating a video. Since the electronic device is performing the task of generating the video, it cannot perform the task of saving the video. At this time, an error occurs in the task of saving the video that the electronic device is about to perform. As an example, the user interface of the electronic device involved in the video generation method provided in the embodiment of the present application is introduced.

[0140] In practical applications, Figure 5 After the text 5033 "Analyzing material..." is displayed in the user interface 503 in Figure 5 Before the thumbnail 5043 of the target video is displayed in the dialog box 5041 in the user interface 504 in Figure 7 After the text 7012 of "Save Video" is inputted by voice in the dialog box 7011 in the user interface 701, the electronic device displays the following in response to the user's voice input: Figure 7 The user interface 702 is shown, and the text 7021 "The video has not been completed and cannot be saved" is displayed in the interface 702.

[0141] Scenario 4: During the process of an electronic device executing the video generation method provided in an embodiment of the present application, a task executed by the electronic device times out.

[0142] In one example, taking the case where a voice assistant in an electronic device receives a user voice and a timeout occurs while performing a task of searching for materials as an example, the user interface of the electronic device involved in the video generation method provided in an embodiment of the present application is introduced.

[0143] In practical applications, Figure 5 When the text 5013 "Searching for materials for you..." is displayed in the user interface 501, it means that the electronic device is performing the task of searching for materials. If a timeout occurs during the execution of the task of searching for materials, the electronic device will display Figure 8 In the user interface 801, the user interface 801 displays a text 8013 indicating that the search for materials has timed out.

[0144] Scenario 5: During the process of an electronic device executing the video generation method provided in an embodiment of the present application, the task performed by the electronic device is interrupted.

[0145] In one example, taking the case where a voice assistant in an electronic device receives a user voice and is interrupted in the process of performing a task of generating a video as an example, the user interface of the electronic device involved in the video generation method provided in an embodiment of the present application is introduced.

[0146] In practical applications, electronic devices display Figure 5After the user interface 502 is displayed, the user can trigger the generate video control 50232 displayed in the material card 5023 in the user interface 502 to trigger the electronic device to perform the task of generating a video. If the electronic device is interrupted during the process of performing the task of generating a video, the electronic device will display Figure 9 The user interface 901 shown includes a pop-up window 9011, which includes a control 9012. The user can click on the control 9012, and the electronic device can respond to the user's operation of clicking on the control 9012 to resume the interrupted task of generating a video. Specifically, after the user clicks on the control 9012, the electronic device responds to the user's operation of clicking on the control 9012 and displays Figure 9 The user interface 902 shown includes a pop-up window 9021, which includes a control 9022. The user can click on the control 9022, and the electronic device responds to the user clicking on the control 9022 by displaying the following information: Figure 5 In the user interface 503 and the user interface 504 , the electronic device can display a thumbnail 5043 of the prepared target video in the user interface 504 .

[0147] It should be understood that the above Figures 5 to 9 The user interface shown is for illustration only and does not constitute any limitation on the user interface of the electronic device to which the video generation method provided in the embodiment of the present application is adapted. Figures 5 to 9 The displayed material may be a static image, a dynamic image, or a video.

[0148] The following describes the execution processes of the electronic devices involved in the above scenarios 1 to 5 respectively.

[0149] Scenario 1 involves the execution process of an electronic device generating a video. During this execution process, the electronic device needs to create a material search task, execute the material search task, create a video generation task, and execute the video generation task.

[0150] The following describes the implementation process of creating a material search task for electronic devices in scenario 1. Figure 10 This is a timing diagram of creating a material search task provided by an embodiment of the present application.

[0151] S1001, YOYO assistant receives user voice 1.

[0152] It should be understood that user voice 1 is the conversation input content of the user received by YOYO assistant. For example, user voice 1 can be "make a video of my second daughter dancing from childhood to adulthood". Figure 4 The user interface 403 in FIG. 4 shows “Help me generate a video of my second daughter dancing” and the like.

[0153] In some embodiments, in addition to voice input, users can also use other methods to input conversational input to the YOYO Assistant, which is not limited in this embodiment of the present application. For example, a user can use text input to send the conversational input content of "making a video of my second daughter dancing from childhood to adulthood" to the YOYO Assistant.

[0154] In the implementation, the user can input voice or text in the dialogue interface provided by the YOYO assistant in the electronic device. The dialogue interface provided by the YOYO assistant can be Figure 4 User interface 402 is shown.

[0155] S1002, after receiving user voice 1, the YOYO assistant generates analysis instruction 1 (carrying user voice 1) according to user voice 1.

[0156] For example, user voice 1 can be Figure 4 The text 4032 of "Help me generate a video of my second daughter dancing" is shown. Analysis instruction 1 is used to instruct semantic analysis and intent analysis of user voice 1 to extract the user intent corresponding to user voice 1 and information such as time, place, person, and event corresponding to user voice 1.

[0157] S1003, YOYO assistant sends analysis instruction 1 to the voice model.

[0158] It should be understood that the language model can be a neural network model, and the speech model can perform semantic analysis and intent analysis on the text or speech input by the user.

[0159] The language model may be a module located in the YOYO Assistant, or the language model may be a module located outside the YOYO Assistant, which is not specifically limited.

[0160] S1004, after receiving the analysis instruction 1, the voice model analyzes and processes the user voice 1 to generate an analysis result 1.

[0161] It should be understood that the analysis result 1 includes creative intent. Figure 1 and semantic information 1. Creative intent refers to the purpose of the user's creation. For example, the voice model can determine the user's creative intent based on the conversation input content of "making a video of my second daughter dancing from childhood to adulthood". Figure 1To create a video. Semantic information 1 includes information such as the time, location, person, and event corresponding to user voice 1. For example, if user voice 1 is "Help me create a video of my second daughter dancing," semantic information 1 may include information that the person is "second daughter" and the event is "dancing." For another example, if user voice 1 is "Generate a video of my travels in Beijing last October," semantic information may include the time "last October," the location "Beijing," the person "me," and the event "travel."

[0162] S1005: The voice model returns analysis result 1 to the YOYO assistant.

[0163] S1006 , YOYO Assistant generates a material search command 1 (carrying a session identifier 1 and semantic information 1 ) based on the analysis result 1 .

[0164] It should be understood that the material search command is used to instruct the electronic device to search the image library for materials that match the semantic information. For example, if voice information 1 includes information about the person "Second Daughter" and the event "Dancing", then the material search command is used to instruct the electronic device to search the image library for pictures or videos of the Second Daughter dancing.

[0165] The session identifier is used to identify the session page when a user enters session input content into the YOYO Assistant when creating a video. Different videos created by users correspond to different session identifiers. For example, user voice A is used to instruct the creation of a video about a trip to Beijing, and user voice B is used to instruct the creation of a video about a trip to Xi'an. The videos created by user voice A and user voice B are different. Based on this, the session identifier A corresponding to user voice A and the session identifier B corresponding to user voice B are two different session identifiers.

[0166] It should also be understood that YOYO Assistant can Figure 1 "Make video" generates a material search command 1. In other embodiments, the YOYO assistant can also generate other commands based on the creative intent. For example, if the user voice is "save video", the creative intent is "save video". The YOYO assistant generates a save video command based on the creative intent of "save video".

[0167] S1007, after YOYO Assistant generates the material search command, it notifies the creation module, task queue module and timeout checker in the media middle platform framework layer to initialize.

[0168] It should be understood that after the YOYO Assistant notifies the creation module, task queue module and timeout checker in the media middle platform framework layer to initialize, the creation module, task queue module and timeout checker can perform initialization operations to put their own modules in working state.

[0169] It should also be understood that the electronic device can notify the creation module, task queue module and timeout checker in the media middle platform framework layer to initialize after activation (first power-on), or after each power-on, or system update, or factory settings restoration, or when the electronic device runs YOYO Assistant. This application uses an electronic device running YOYO Assistant, and after YOYO Assistant generates a material search command, it notifies the creation module, task queue module and timeout checker in the media middle platform framework layer to initialize for example.

[0170] It should also be understood that after the creation module, task queue module and timeout checker in the media middle platform framework layer are initialized, a message indicating that the initialization of the above modules has been completed will be returned to the YOYO assistant.

[0171] In other embodiments, after the YOYO assistant generates a material search command, it can also initialize modules in the media middle platform framework layer other than the creation module, task queue module and timeout checker, such as initializing the result notifier, etc. The embodiments of the present application are not limited to this.

[0172] S1008, after receiving the message that the creation module, task queue module and timeout detector module have completed initialization, the YOYO assistant sends a material search command 1 (carrying session identifier 1 and semantic information 1) to the creation module.

[0173] S1009 , the creation module generates a create task input instruction 1 , a create material snapshot cache instruction 1 , a create task configuration instruction 1 and a create material search task instruction 1 based on the material search command 1 .

[0174] S1010, the creation module sends a task creation input instruction 1 (carrying a session identifier 1 and semantic information 1) to the task input module.

[0175] S1011 , after receiving the create task input instruction 1 , the task input module creates a task input 1 object (including a task identifier 1 , a session identifier 1 and semantic information 1 ).

[0176] It should be understood that creating a task input instruction refers to creating a task input object for a task related to video generation. For example, creating a task input 1 object refers to creating a task input 1 object for a material search task. For tasks related to video generation, please refer to the above embodiments and will not be repeated here. An object is an entity used in a program to describe objective things, and a task input object is an entity used in a program to describe task input. Task input includes a task identifier, session identifier, and semantic information.

[0177] After receiving the create task input instruction 1, the task input module can generate a task identifier. It should be understood that the task identifier is used to identify tasks related to video generation, and one task identifier is used to identify one task. For example, Table 1 shows the correspondence between task identifiers and tasks.

[0178] Table 1

[0179] Task identifier Task Type Whether to join the task queue Task group number 1 Material Search yes 1 2 Character confirmation yes 2 3 Material addition yes 3 4 Material deletion yes 4 5 Material Selection yes 5 6 Generate Video yes 6 7 Change template yes 6 8 Change music yes 6 9 Adjust duration yes 6 10 Save Video yes 6 11 stop no 7 12 Interrupt and resume no 8

[0180] In this embodiment of the present application, after receiving the create task input instruction 1, the task input module can generate the task identifier 1 in Table 1 and create the task input 1 object based on the session identifier 1 and semantic information 1 carried in the create task input instruction 1. The task input 1 object includes the session identifier 1, semantic information 1, and task identifier 1.

[0181] S1012, the task input module returns the task input 1 object to the creation module.

[0182] S1013, the creation module sends a creation material snapshot cache instruction 1 to the snapshot cache module.

[0183] S1014 , after receiving the create material snapshot cache instruction 1 , the snapshot cache module creates a material snapshot cache 1 object.

[0184] It should be understood that a cache refers to temporary storage, and a material snapshot cache object refers to the address in temporary storage where material snapshot data is stored. A snapshot is a fully usable copy of a specified data set, consisting of an image of the corresponding data at a specific point in time (the point in time when the copy begins). A snapshot can be a copy or a replica of the data it represents.

[0185] It should also be understood that in the following embodiments, when starting to execute tasks related to video generation, it is first necessary to pre-process each task related to video generation. The pre-processing includes snapshot processing, and the snapshot data obtained by the snapshot processing can be stored in the material snapshot cache object.

[0186] In the embodiment of the present application, the material snapshot cache 1 object refers to the address storing the snapshot data of the material searched by the material search task.

[0187] S1015, the snapshot cache module returns the material snapshot cache 1 object to the creation module.

[0188] S1016, the creation module sends a creation task configuration instruction 1 to the task configuration module.

[0189] S1017, after receiving the create task configuration instruction 1, the task configuration module creates a task configuration 1 object (task configuration 1 includes task name 1, snapshot cache 1 object, task group number 1 and preset timeout duration 1).

[0190] It should be understood that the task name refers to the name of the task related to video generation. For example, when the task related to video generation is a material search task, the name of the material search task can be "Material Search". The name can also be other custom fields, which is not limited in this embodiment of the application.

[0191] It should also be understood that each task related to video generation has its own corresponding task group number, and multiple tasks can correspond to a single group number. For example, referring to Table 1, the Generate Video task, Change Module task, Change Music task, Adjust Duration task, and Save Video task correspond to the same task group number 6, while the remaining tasks in Table 1 correspond to different task group numbers. In the present embodiment, Task Group Number 1 refers to the group number corresponding to the Material Search task in Table 1.

[0192] The preset timeout duration refers to the timeout threshold for each task. During the execution of a task, if the execution duration exceeds the preset timeout duration, it means that the task has timed out. In the embodiments of the present application, the preset timeout duration can be configured based on the performance of the device, and the embodiments of the present application are not limited to this. In the embodiments of the present application, the preset timeout durations for different tasks can be the same or different, and the embodiments of the present application are not limited to this.

[0193] S1018, the task configuration module returns the task configuration 1 object to the creation module.

[0194] S1019, the creation module sends a create result notifier instruction 1 to the result notifier.

[0195] S1020 , the result notifier creates a result notifier 1 object corresponding to task identifier 1.

[0196] It should be understood that the result notifier object can store the execution result of the task corresponding to the task identifier. For example, the result notifier 1 object can store the execution result of the material search task corresponding to the task identifier 1.

[0197] S1021, the result notifier returns the result notifier 1 object to the creation module.

[0198] S1022, the creation module sends a create material search task instruction 1 (carrying a task input 1 object, a task configuration 1 object and a result notifier 1 object) to the task module.

[0199] S1023, the task module creates a material search task 1 (including a task input 1 object, a task configuration 1 object, a result notifier 1 object and a task sequence number 1).

[0200] For example, after receiving the Create Material Search Task instruction 1, the task module can obtain the task sequence number of the last task in the task queue and, based on this task sequence number, generate the task sequence number of the currently created task. For example, if the task sequence number of the last task in the task queue is 3, that is, there are two tasks to be executed before the last task, then, based on this task sequence number, the task sequence number generated for the currently created task is 4. For another example, if there are no tasks in the task queue, then the task sequence number generated for the currently created task is 1.

[0201] It should be understood that during the execution process of generating a video, the electronic device needs to first execute the material search task and then execute the video generation task. Therefore, under normal circumstances, when the task module creates a material search task, there is no task in the task queue, then the task number of the material search task generated by the task module can be task number 1.

[0202] In other embodiments, the tasks created by the task module may also be referred to as task instances, etc., which is not limited in the embodiments of the present application.

[0203] S1024, the task module returns the material search task 1 (including the task input 1 object, the task configuration 1 object, the result notifier 1 object and the task sequence number 1) to the creation module.

[0204] It should be understood that the above steps 1001 to 1024 specifically describe the implementation process of the electronic device creating a search task. The implementation process of the electronic device creating other tasks related to video generation (such as Figure 12 For the implementation process of creating a video generation task, the implementation process of creating a video saving task, etc., you can refer to the implementation process of creating a search task, which will not be repeated here.

[0205] The following describes the implementation process of the electronic device performing the material search task in scenario 1. Figure 11 This is a timing diagram for executing a material search task provided in an embodiment of the present application.

[0206] S1025, after receiving the material search task 1, the creation module sends the material search task 1 (including the task input 1 object, the task configuration 1 object, the result notifier 1 object and the task sequence number 1) to the task queue module.

[0207] It should be understood that Figure 10 In step 1024 , the task module returns the material search task 1 to the creation module. After receiving the material search task 1 , the creation module sends the material search task 1 to the task queue module to publish the material search task 1 .

[0208] S1026 , after receiving the material search task 1 , the task queue module detects whether the task group number 1 of the material search task 1 is occupied.

[0209] For the explanation of the task group number, please refer to the above embodiment and will not be repeated here.

[0210] In implementation, the task queue module can detect whether the task group number of the newly released task is occupied by detecting whether the task group number of the task already existing in the task queue is the same as the task group number of the task newly released by the task module. For example, please refer to the task type and task group number shown in Table 1. Suppose there is already a task of generating a video in the task queue, and the task group number corresponding to the task of generating a video is 6. When the task module releases a new task to the task queue module, for example, the new task released is to save the video, and the task group number corresponding to the task of saving the video is also 6, then the task queue module detects that task group number 6 is occupied. Suppose there is already a task of material search in the task queue, and the task group number corresponding to the task of material search is 1. When the task module releases a new task to the task queue module, for example, the new task released is to generate a video, and the task group number corresponding to the task of generating a video is 6, then the task queue module detects that task group number 6 is not occupied.

[0211] S1027, the task queue module records the occupancy status of task group number 1 according to the detection result.

[0212] It should be understood that during the execution process of the electronic device generating a video, before the task module publishes the material search task to the task queue module, there are no other tasks in the task queue. Therefore, the task group number 1 of the material search task 1 detected by the task queue module through step 1026 is not occupied, and the occupancy status of the task group number 1 recorded through step 1027 is "unoccupied".

[0213] The implementation process of setting the occupation status of the task group number to "occupied" can be referred to the following embodiment and will not be repeated here.

[0214] S1028 : When the task queue module determines that the occupied state of task group number 1 is unoccupied, the task queue module sends an instruction to time the material search task 1 to the timeout detector.

[0215] It should be understood that the task queue module sends the instruction to the timeout detector to time the task in order to detect whether the task has timed out. The implementation process of detecting whether the task has timed out can be referred to the following embodiment and will not be repeated here.

[0216] S1029: The task queue module adds the material search task 1 to the end of the task queue.

[0217] It should be understood that when a new task is issued to the task queue module through the task module, the task queue module can add the task to the end of the task queue, and execute the newly added task after the tasks before the end are completed.

[0218] It should also be understood that during the execution process of the electronic device generating a video, before the task module publishes the material search task to the task queue module, there are no other tasks in the task queue. Therefore, the task queue module adds the material search task 1 to the end of the task queue, which means that the material search task 1 is at the first place in the task queue.

[0219] S1030, the task queue module sends the task sequence number 1 of the material search task 1 to the creation module.

[0220] It should be understood that when the task queue module is about to start scheduling the execution of a task, it can send the task sequence number of the task to be executed to the creation module so that the creation module can understand the task to be executed.

[0221] S1031, the creation module records the current task number as task number 1.

[0222] S1032, the creation module sends the execution result 1 of the material search task 1 corresponding to the task number 1 to the YOYO assistant based on the task number 1 (the execution result 1 indicates that the material search task 1 is being executed in the task queue).

[0223] S1033, based on the execution result 1, the YOYO assistant displays the interface 1 including “Searching for materials”.

[0224] For example, interface 1 can refer to Figure 5 The user interface 501 is shown, and the text 5013 of "Looking for materials for you" in the user interface 1 can also be "Searching for materials", "Searching materials", etc., which is not limited in this embodiment of the present application.

[0225] S1034: The task queue module schedules the execution of the first task (ie, material search task 1) in the task queue.

[0226] S1035 , the task queue module sends an execution instruction of the material search task 1 to the task module.

[0227] S1036 , the task module receives the execution instruction of the material search task 1 and performs pre-processing on the material search task 1 .

[0228] It should be understood that when the task module begins executing tasks related to video generation, it first needs to perform pre-processing on each task related to video generation. The pre-processing includes snapshot processing, and the snapshot data obtained by the snapshot processing can be stored in the material snapshot cache object. The implementation process of snapshot processing can refer to S1037 to S1039. Of course, in other embodiments, the pre-processing can also include other processing, and the embodiments of the present application are not limited to this.

[0229] S1037, the task module sends a snapshot processing instruction to the snapshot cache module.

[0230] S1038: After receiving the snapshot processing instruction, the snapshot cache module obtains the material set from the session state module.

[0231] It should be understood that the session state module stores the relevant materials obtained after the task module completes the task related to video generation. For example, after the task module completes the material search task, it can search for relevant materials, and the task module sends the relevant materials to the session state module for storage. For another example, after the task module completes the video generation task, it can generate a video, and the task module sends the video to the session state module for storage.

[0232] In the implementation, after the task module executes search task 1, the task module can store the searched materials in the session state module. After the snapshot cache module receives the snapshot processing instruction, it can obtain the material collection searched by material search task 1 from the session state module. Then the snapshot cache module performs snapshot processing on the material collection to generate snapshot information of the material collection.

[0233] S1039 , when the snapshot cache module obtains the material set, it performs snapshot processing on the material set to generate snapshot information of the material set.

[0234] It should be understood that a snapshot is a fully usable copy of a specified data set, including an image of the corresponding data at a specific point in time (the point in time when the copy is started). Snapshot information of a material set is used to support the task stop function, allowing a task to be rolled back to its pre-task state after being stopped.

[0235] S1040: The task module detects whether the material search task 1 is interrupted.

[0236] It should be understood that when the task module executes tasks corresponding to some task group numbers, it needs to call the editing module in the media platform framework layer to execute the task. For example, referring to Table 1, when the task module executes the task corresponding to task group number 6, it needs to call the editing module to execute the task corresponding to task group number 6. When the task module executes task groups 1 to 5 and 7 to 8, it does not need to call the editing module, and the task module executes the task itself.

[0237] In implementation, the task module can detect whether the task is interrupted by detecting whether the editing module can be called. For example, when the task module executes the material search task 1 corresponding to the task group number 1, it does not need to call the editing module. Therefore, the detection result can be that the material search task 1 is not interrupted. For another example, when the editing module has been uninstalled or cannot be called due to an exception, when the task module executes the generate video task 6 corresponding to the task group number 6, the task execution module originally needs to call the editing module to execute the generate video task 6, but because the editing module has been uninstalled or cannot be called due to an exception, then the detection result can be that the generate video task 1 is interrupted.

[0238] S1041 , when the task module detects that the material search task 1 is not interrupted, the task module executes the material search task 1 according to the task input 1 object in the material search task 1 .

[0239] During implementation, when the task module detects that the material search task 1 is not interrupted, it can send a task input 1 object corresponding to the material search task 1 to the task input module. The task input module returns the task input 1 object to the task module. The task input module executes the material search task 1 according to the task identifier 1, session identifier 1 and semantic information 1 included in the task input 1 object. For example, the task corresponding to the task identifier 1 is the material search task, the conversation in the session identifier 1 is "make a video of the second daughter dancing from childhood to adulthood", and the voice information 1 includes the character "second daughter" and the event "dancing". Then, based on the above information, the task module can search the gallery for pictures and / or videos containing the second daughter dancing.

[0240] S1042, the task module generates a create task output object instruction (carrying the completed task status) according to the task status when executing the task.

[0241] It should be understood that the task status may include one or more of the following: task completion status, task error status, task timeout status, task occupied status, task stop status, etc.

[0242] In the embodiment of the present application, the task status of the material search task 1 is a task completion status.

[0243] S1043, the task module sends a create task output object instruction (carrying the completed task status) to the task output module.

[0244] It should be understood that the task output object may include the task status of the task related to video generation.

[0245] S1044 , the task output module creates a task output object (including the completed task status of the material search task 1 ).

[0246] S1045, the task output module returns the task output object (carrying the completed task status) to the task module.

[0247] S1046, the task module generates execution result 2 (including multiple materials) based on the task output object.

[0248] In the embodiment of the present application, the task output corresponds to the task status of the completed material search task 1. After receiving the task output object, the task module can obtain multiple searched materials from the gallery and generate execution result 2 based on the multiple searched materials.

[0249] S1047, the task module sends an update session state instruction (carrying multiple materials) to the session state module.

[0250] S1048, the session state module saves or updates the material.

[0251] It should be understood that when the data of the material set stored in the session state module is empty, the session state module saves multiple materials to the session state module after receiving the session state instruction. When the data of the material set stored in the session state module is not empty, the session state module updates the material set in the session state module after receiving the session state instruction. For example, the original session state module stores 30 photos of Li Si playing in Beijing, and the session state instruction received by the session state module carries 20 photos of the second daughter dancing. In this case, the session state module updates the 30 photos of Li Si playing in Beijing to 20 photos of the second daughter dancing. For another example, the original session state module stores 30 photos of Li Si playing in Beijing, and the session state instruction received by the session state module carries 5 deleted photos of Li Si playing in Beijing. In this case, the session state module updates the 30 photos of Li Si playing in Beijing to 25 photos of Li Si playing in Beijing.

[0252] S1049, the task module detects whether the material search task 1 is stopped.

[0253] In the implementation, the user Figure 6 In the illustrated interface 601 , after the stop control 6014 is clicked or the user inputs a stop command by voice, the task module detects that the material search task 1 has stopped. Otherwise, the task module detects that the material search task 1 has not stopped.

[0254] In this embodiment of the application, the user is not Figure 6 In the illustrated interface 601 , the stop control 6014 is clicked, or the user inputs a stop instruction by voice, and thus the task detection module detects that the material search task 1 has not been stopped.

[0255] S1050 : When the task module detects that the material search task 1 has not been stopped, it confirms that the execution result 2 is effective.

[0256] It should be understood that the execution result 2 includes multiple materials searched when the task module executes the material search task.

[0257] S1051, the task module sends the execution result 2 (carrying multiple materials) to the result notifier based on the result notifier 1 object.

[0258] It should be understood that the task module sends the execution result to the result notifier so that the result notifier can notify the YOYO assistant, so that the YOYO assistant can understand the execution result of the task and display the corresponding interface of the execution result.

[0259] S1052, the result notifier records the execution result 2 in the result notifier 1 object.

[0260] S1053, the result notifier sends the execution result 2 (carrying multiple materials) to the YOYO assistant.

[0261] S1054, YOYO Assistant displays interface 2 including multiple materials according to execution result 2.

[0262] For example, interface 2 can refer to Figure 5 In the user interface 502 shown, the multiple materials may be 5023 in the user interface 502 .

[0263] S1055 , the task module sends a task output object to the task queue module (the task output object includes the completed task status of the material search task 1 ).

[0264] S1056: The task queue module cleans up the tasks based on the task output objects.

[0265] It should be understood that the task queue module can clear tasks based on the task status of the tasks included in the task output object. For example, if the task status of Material Search Task 1 is Completed, the task queue module will clear the completed Material Search Task 1 and will not include Material Search Task 1 in the task queue. For another example, if the task status of Material Search Task 1 is Timed Out, the task queue module will clear the Material Search Task 1 that is in the Timed Out state and will not include Material Search Task 1 in the task queue.

[0266] S1057 , the task queue module sends a stop timing instruction for the material search task 1 to the timeout detector.

[0267] In an embodiment of the present application, when the task module is executing a task, the timeout detector does not detect that the execution time of the task has timed out. Therefore, the task queue module sends an instruction to the timeout detector to stop timing the task. After receiving the stop timing instruction, the timeout detector stops timing the material search task 1.

[0268] It should be understood that the above steps 1025 to 1057 specifically describe the implementation process of the electronic device performing the search task, and the implementation process of the electronic device performing other tasks related to video generation (such as Figure 13 For the implementation process of executing the task of generating a video, the implementation process of executing the task of saving a video, etc., you can refer to the implementation process of executing the search task, which will not be repeated here.

[0269] The following describes the implementation process of the electronic device creation material search task in scenario 1. Figure 12 This is a timing diagram for executing a material search task provided in an embodiment of the present application.

[0270] S1201, YOYO assistant receives user voice 2.

[0271] It should be understood that user voice 1 is the user's conversation input content received by the YOYO assistant. The explanation of the conversation input content has been explained in the above embodiment and will not be repeated here.

[0272] In the embodiment of the present application, user voice 2 refers to the voice of "generate video" input by the user in the dialogue interface provided by the YOYO assistant in the electronic device.

[0273] In other embodiments, the user can also click Figure 5 In the user interface 502 shown, a video generation control 5023 is shown. In response to the user clicking the control 5023 , the electronic device generates the analysis instruction 2 in step S1202 .

[0274] S1202, after receiving user voice 2, the YOYO assistant generates analysis instruction 2 (carrying user voice 2) based on user voice 2.

[0275] For example, user voice 2 can be Figure 5 The text 5031 of “Generate Video” is shown in the interface 503 in FIG. The analysis instruction 2 is used to instruct to perform semantic analysis and intent analysis on the user voice 2 to extract the user intent corresponding to the user voice 2 and the time, place, person and event information corresponding to the user voice 2.

[0276] S1203, YOYO assistant sends analysis instruction 2 to the voice model.

[0277] S1204, after receiving the analysis instruction 2, the voice model analyzes and processes the user voice 2 to generate an analysis result 2.

[0278] It should be understood that the analysis result 2 includes creative ideas. Figure 2 and semantic information 2. For example, the voice model can determine the user’s creative intent based on the “generate video” conversation input content. Figure 2 To generate video.

[0279] It should also be understood that when the user enters the conversation input content on the same conversation page of the YOYO assistant (that is, the user enters different voice commands or text commands, but the conversation identifier generated by the YOYO assistant is the same), the semantic information corresponding to different user voices in the analysis results generated by the voice model is the same. For example, the user first enters user voice 1 of "help me generate a video of my second daughter dancing", and then enters user voice 2 of "generate video". Since the purpose of user voice 2 is also to generate a video of my second daughter dancing, the conversation identifiers of user voice 1 and user voice 2 are the same, and thus, the semantic information 1 of user voice 1 and the semantic information 2 of user voice 2 in the analysis results generated by the voice model are the same. For the explanation of semantic information 2, please refer to semantic information 1, which will not be repeated here.

[0280] S1205: The voice model returns analysis result 2 to the YOYO assistant.

[0281] S1206, YOYO Assistant generates a video command 2 (carrying session identifier 1 and semantic information 2) based on analysis result 2.

[0282] It should be understood that the generate video command is used to instruct the electronic device to synthesize a video using materials that match the semantic information 2 and are searched in the gallery.

[0283] S1207, the YOYO assistant sends a generate video command 2 (carrying session identifier 1 and semantic information 2) to the creation module.

[0284] S1208, the creation module generates a create task input instruction 2, a create material snapshot cache instruction 2, a create task configuration instruction 2 and a create material search task instruction 2 based on the generate video command 2.

[0285] S1209, the creation module sends a task creation input instruction 2 (carrying session identifier 1 and semantic information 2) to the task input module.

[0286] S1210 , the task input module creates a task input 2 object (including task identifier 2 , session identifier 1 and semantic information 2 ).

[0287] Exemplarily, creating a task input 2 object refers to creating a task input 2 object for a generate video task. In the embodiment of the present application, the task identifier 2 in the task input 2 object can refer to the task identifier of the generate video task in Table 1.

[0288] S1211, the task input module returns the task input 2 object to the creation module.

[0289] S1212, the creation module sends a create material snapshot cache instruction 2 to the snapshot cache module.

[0290] S1213: The snapshot cache module creates a material snapshot cache 2 object.

[0291] In the embodiment of the present application, the material snapshot cache 2 object refers to the address for storing the snapshot data of the video generated by the generate video task.

[0292] S1214, the snapshot cache module returns the material snapshot cache 2 object to the creation module.

[0293] S1215, the creation module sends a creation task configuration instruction 2 to the task configuration module.

[0294] S1216, the task configuration module creates a task configuration 2 object (task configuration 2 includes task name 2, snapshot cache 2 object, task group number 2 and preset timeout duration 2).

[0295] For example, the name of task name 2 may be "Generate Video", and the name may also be other customized fields, which is not limited in the embodiment of the present application.

[0296] The task group number 2 may refer to the task group number corresponding to the generate video task shown in Table 1.

[0297] The preset timeout duration 2 refers to the timeout threshold for generating a video task. The preset timeout duration 2 may be the same as or different from the preset timeout duration 1, and this embodiment of the present application does not limit this.

[0298] S1217, the task configuration module returns the task configuration 2 object to the creation module.

[0299] S1218, the creation module sends a create result notifier instruction 2 to the result notifier.

[0300] S1219, the result notifier creates a result notifier 2 object corresponding to task identifier 2.

[0301] Exemplarily, the result notifier 2 object may store the execution result of the generate video task corresponding to the task identifier 2.

[0302] S1220, the result notifier returns the result notifier 2 object to the creation module.

[0303] S1221, the creation module sends a create-generate video task instruction 2 (carrying a task input 2 object, a task configuration 2 object, and a result notifier 2 object) to the task module.

[0304] S1222, the task module creates and generates video task 2 (including task input 2 object, task configuration 2 object, result notifier 2 object and task sequence number 2).

[0305] It should be understood that the implementation process of creating the video generation task 2 is the same as the implementation process of creating the search material task 1, and will not be repeated here.

[0306] S1223, the task module returns the generated video task 2 (including the task input 2 object, the task configuration 2 object, the result notifier 2 object and the task sequence number 2) to the creation module.

[0307] The following describes the implementation process of the electronic device performing the video generation task in scenario 1. Figure 13 This is a timing diagram of executing a video generation task provided by an embodiment of the present application.

[0308] S1225, the creation module sends the generated video task 2 to the task queue module.

[0309] It should be understood that Figure 12 In step 1223, the task module returns the generated video task 2 to the creation module. After receiving the generated video task 2, the creation module sends the generated video task 2 to the task queue module to publish the generated video task 2.

[0310] S1226, after receiving the generate video task 2, the task queue module detects whether the task group number 2 of the generate video task 2 is occupied.

[0311] Regarding the implementation method of detecting whether the task group number 2 of the video generation task 2 is occupied, reference can be made to the above embodiment and will not be repeated here.

[0312] S1227, the task queue module records the occupancy status of task group number 2 according to the detection result.

[0313] The embodiment of the present application introduces the process of an electronic device executing a task of generating a video in the above scenario one. In scenario one, task group number 2 of generating video task 2 is not occupied, so the occupancy status of task group number 2 recorded in step 1327 is "unoccupied".

[0314] S1228 , when the task queue module determines that the occupied state of task group number 2 is unoccupied, the task queue module sends an instruction to the timeout detector to time the generation video task 2 .

[0315] S1229, the task queue module adds the generated video task 2 to the end of the task queue.

[0316] It should be understood that when a new task is issued to the task queue module through the task module, the task queue module can add the task to the end of the task queue, and execute the newly added task after the tasks before the end are completed.

[0317] In the embodiment of the present application, when the task module issues the generate video task 2 to the task queue module, the search material task 1 in the task queue has been completed, and thus, the generate video task 2 is at the first place in the task queue.

[0318] S1230, the task queue module sends the task sequence number 2 of generating video task 2 to the creation module.

[0319] S1231, the creation module records the current task number as task number 2.

[0320] S1232, based on task number 2, the creation module sends the execution result 3 of the generate video task 2 corresponding to task number 2 to the YOYO assistant (execution result 3 indicates that the generate video task 2 is being executed in the task queue).

[0321] S1233, based on the execution result 3, the YOYO assistant displays the interface 3 including “Analyzing Materials”.

[0322] For example, interface 3 can refer to Figure 5 The interface 503 shown includes a text 5033 of “Material is being analyzed”.

[0323] S1234, the task queue module schedules the execution of the first task in the task queue (ie, generating video task 2).

[0324] S1235, the task queue module sends an execution instruction for generating video task 2 to the task module.

[0325] S1236, the task module receives the execution instruction of generating video task 2, and performs pre-processing on generating video task 2.

[0326] S1237, the task module sends a snapshot processing instruction to the snapshot cache module.

[0327] S1238: The snapshot cache module obtains a material set (the material set may be a complete video) from the session state module.

[0328] In an embodiment of the present application, after the task module has completed the task of generating video 2, the task module stores the generated video in the session state module. After the snapshot cache module receives the snapshot processing instruction, it can obtain the video generated by video task 2 from the session state module. The snapshot cache module then performs snapshot processing on the video to generate snapshot information of the material set (the snapshot information of the material set can be snapshot information of the complete video).

[0329] S1239, when the snapshot cache module obtains the material set, it performs snapshot processing on the material set to generate snapshot information of the finished video.

[0330] The material collection in the embodiment of the present application can be a finished video. The snapshot information of the finished video can include the template information, music information, duration, cover, etc. of the video.

[0331] S1240, the task module detects whether the video generation task 2 is interrupted.

[0332] Regarding the implementation method of detecting whether the video generation task 2 is interrupted, reference may be made to the above embodiment and will not be repeated here.

[0333] In the embodiment of the present application, when the task module executes the video generation task 2, it can call the editing module to execute the task. Therefore, the detection result of the task module detecting whether the video generation task 2 is interrupted is that the task 2 is not interrupted.

[0334] S1241, when the task module detects that the video generation task 2 is not interrupted, it sends an instruction to execute the video generation task 2 (the instruction carries the task input 2 object) to the editing module.

[0335] S1242: After receiving the instruction to execute the video generation task 2, the editing module obtains a material set from the session state module.

[0336] S1243, the editing module executes the video generation task 2 based on the material set and the task input 2 object.

[0337] S1244: The editing module sends the progress of generating video task 2 to the task module.

[0338] It should be understood that the editing module can send the progress of generating video task 2 to the task module every preset time. The embodiment of the present application does not limit the specific data of the preset time.

[0339] The progress of generating video task 2 can be expressed in the form of a percentage, for example, 50%, 60%, etc., or can be expressed in the form of a fractional value, for example, 4 / 5, 1 / 3, etc., which is not limited in this embodiment of the present application.

[0340] S1245, the task module detects whether the video generation task 2 is stopped.

[0341] It should be understood that when the user inputs a stop instruction, the task module can detect whether the generate video task 2 is stopped.

[0342] S1246, the task module sends the progress of generating video task 2 to the result notifier.

[0343] S1247, the result notifier records the progress of generating video task 2.

[0344] S1248, the result notifier sends the progress of generating video task 2 to the YOYO assistant.

[0345] S1249, YOYO Assistant displays interface 4 including "Analyzing Materials" based on the progress of video task 2.

[0346] For example, interface 4 can refer to Figure 5 The interface 503 shown includes a text 5033 of “Material is being analyzed”.

[0347] S1250, the task module sends the progress of generating video task 2 to the task queue module.

[0348] S1251: The task queue module generates an instruction for updating the timing of task 2 based on the progress of generating video task 2, and sends the instruction to the timeout detector.

[0349] For example, assuming that the preset timeout duration 2 is 20ms, after the timeout detector receives the start timing instruction sent by the task queue module through S1228, it starts to count the execution duration of the generated video task 2 from 0ms. Originally, at the 20ms, the generated video task 2 has not been completed or the editing module has not sent the progress of the generated video task 2 to the task module. Then the timeout detector sends a timeout notification to the task queue module, notifying that the generated video task 2 has timed out. However, if the task queue module receives the progress of the generated video task 2 sent by the task module through S1250 at the 10ms, then the task queue module generates an instruction to update the timing of task 2 based on the progress of the generated video task 2. The instruction to update the timing is used to instruct the timeout detector to re-time the generated video task 2 from the 10ms. Originally, at the 30ms, the generated video task 2 has not been completed or the editing module has not sent the progress of the generated video task 2 to the task module. Then the timeout detector sends a timeout notification to the task queue module, notifying that the generated video task 2 has timed out.

[0350] S1252: The editing module sends the status and execution result of generating video task 2 to the task module.

[0351] It should be understood that the status of the generated video task 2 can be a completed state, a stopped state, an abnormal state, etc. For example, when the program of the editing module is abnormal, fails or is stuck, the editing module cannot execute the video task 2 normally. At this time, the editing module sends the status of the generated video task 2 to the task module as an abnormal state or a stopped state. The execution result of the abnormal state or the stopped state can be a program abnormality or a program stop. For another example, after the editing module executes the generated video task 2 normally, the editing module sends the status of the generated video task 2 to the task module as a completed state, and the execution result of the normal state is the generated video.

[0352] In the embodiment of the present application, the editing module sends the status of generating video task 2 to the task module as completed, and the execution result is the video generated after generating video task 2 has been executed normally.

[0353] S1253, the task module generates a create task output instruction (carrying a completed task status) based on the task status of generating video task 2.

[0354] S1254, the task module sends a create task output instruction (carrying the completed task status) to the task output module.

[0355] S1255, the task output module creates a task output object (the task output object includes the task status of the generated video task 2 being completed).

[0356] S1256, the task output object model returns the task output object (carrying the completed task status) to the task module.

[0357] S1257, the task module generates execution result 4 (including the finished video) based on the task output object.

[0358] S1258, the task module sends an update session status instruction (carrying the completed video) to the session status module.

[0359] S1259, the session state module saves or updates the material.

[0360] Exemplarily, the session state module saves the finished video generated after the video generation task 2 has been executed normally.

[0361] S1260, the task module detects whether the video generation task 2 is stopped.

[0362] The implementation method of detecting whether the video generation task 2 is stopped can refer to the above embodiment and will not be repeated here.

[0363] In this embodiment of the application, the user is not Figure 5In the illustrated interface 503 , the stop control 5034 is clicked, or the user inputs a stop instruction by voice, and thus the task detection module detects that the video generation task 2 has not stopped.

[0364] S1261: When the task module detects that the video generation task 2 has not stopped, it confirms that the execution result 4 is effective.

[0365] S1262, the task module sends the execution result 4 (carrying the finished video) to the result notifier based on the result notifier 2 object.

[0366] S1263, the result notifier records the execution result 4 in the result notifier 1 object.

[0367] S1264, the result notifier sends the execution result 4 (with the finished video) to the YOYO assistant.

[0368] S1265, YOYO Assistant displays interface 5 including the finished video according to execution result 4.

[0369] For example, interface 5 can refer to Figure 5 Interface 504 is shown, and the finished video can refer to video 5043 in interface 504.

[0370] S1266, the task module sends a task output object to the task queue module (the task output object includes the task status of the generated video task 2 being completed).

[0371] S1267, the task queue module cleans up the tasks based on the task output objects.

[0372] Exemplarily, if the task status of the generate video task 2 is completed, the task queue module clears the completed generate video task 2, and the task queue does not include the generate video task 2.

[0373] S1268, the task queue module sends an instruction to the timeout detector to stop timing the video generation task 2.

[0374] In an embodiment of the present application, when the task module is executing a task, the timeout detector does not detect that the execution time of the task has timed out. Therefore, the task queue module sends an instruction to the timeout detector to stop timing the task. After receiving the stop timing instruction, the timeout detector stops timing the video generation task 2.

[0375] The following describes the execution process of the electronic device involved in the above scenario 2.

[0376] Scenario 2 involves an electronic device stopping the task of performing a video generation-related task when a user inputs a stop command by voice or text. This embodiment of the application uses an example of an electronic device stopping the task of performing a material search task. Figure 14 This is a timing diagram of an electronic device stopping executing a material search task provided by an embodiment of the present application.

[0377] S1401, the creation module sends the material search task 1 to the task queue module.

[0378] S1402: After receiving the material search task 1, the task queue module checks whether the task group number 1 of the material search task 1 is occupied.

[0379] S1403, the task queue module records the occupancy status of task group number 1 according to the detection result.

[0380] S1404 : When the task queue module determines that the occupied state of task group number 1 is unoccupied, the task queue module sends an instruction to time the material search task 1 to the timeout detector.

[0381] S1405: The task queue module adds the material search task 1 to the end of the task queue.

[0382] S1406, the task queue module sends the task sequence number 1 of the material search task 1 to the creation module.

[0383] S1407, the creation module records the current task number as task number 1.

[0384] S1408, the creation module sends the execution result 1 of the material search task 1 corresponding to the task number 1 to the YOYO assistant based on the task number 1 (the execution result 1 indicates that the material search task 1 is being executed in the task queue).

[0385] S1409, based on the execution result 1, the YOYO assistant displays the interface 1 including “Searching for materials”.

[0386] S1410 , the task queue module schedules the execution of the first task (ie, material search task 1 ) in the task queue.

[0387] S1411: The task queue module sends an execution instruction for the material search task 1 to the task module.

[0388] S1412: The task module receives the execution instruction of the material search task 1 and performs pre-processing on the material search task 1.

[0389] S1413, the task module sends a snapshot processing instruction to the snapshot cache module.

[0390] S1414: The snapshot cache module obtains the material collection from the session state module.

[0391] S1415 , when the snapshot cache module obtains the material set, it performs snapshot processing on the material set to generate snapshot information of the material set.

[0392] S1416: YOYO Assistant detects the user's operation on the stop control.

[0393] For example, the user can Figure 6 By clicking control 6014 in the shown interface 601, the YOYO assistant can detect the user's operation of the stop control.

[0394] In other embodiments, the user may also input the same stop instruction to the YOYO assistant through voice input or text input. After receiving the instruction, the YOYO assistant may also display an interface 6 including "This answer has stopped". This embodiment of the present application is not limited to this.

[0395] S1417, in response to the user's operation on the stop control, the YOYO assistant displays interface 6 including "This answer has been stopped".

[0396] For example, interface 6 can refer to Figure 6 Interface 602 is shown, and interface 602 includes text 6022 that reads “This answer has stopped.”

[0397] S1418, the YOYO assistant sends a stop instruction to the creation module to stop the current task.

[0398] In the embodiment of the present application, the current task refers to material search task 1.

[0399] S1419, the creation module sends the task sequence number 1 of the current task to the task queue module.

[0400] In the embodiment of the present application, task number 1 refers to the task number of material search task 1.

[0401] S1420 , the task queue module queries the material search task 1 corresponding to the task sequence number 1 .

[0402] S1421, the task queue module sends the material search task 1 to the task module.

[0403] S1422, the task module changes the state of the material search task 1 to a stopped state.

[0404] S1423, the task module detects whether the material search task 1 is interrupted.

[0405] S1424 , when the task module detects that the material search task 1 is not interrupted, the task module executes the material search task 1 according to the task input 1 object in the material search task 1 .

[0406] It should be understood that the implementation process of the task module performing the material search task 1 can refer to other embodiments and will not be repeated here.

[0407] S1425, the task module generates a create task output instruction (carrying the completed task status) according to the task status when executing the task.

[0408] It should be understood that in this embodiment of the application, although the task module has changed the state of the material search task 1 to the stopped state through S1422, the task module needs to normally execute the material search task 1. Therefore, the task state of the material search task 1 in S1425 is the completed state.

[0409] S1426, the task module sends a create task output instruction (carrying the completed task status) to the task output module.

[0410] S1427 , the task output module creates a task output object (including the completed task status of the material search task 1 ).

[0411] S1428, the task output module returns the task output object (carrying the completed task status) to the task module.

[0412] S1429, the task module generates execution result 2 (including multiple materials) based on the task output object.

[0413] S1430, the task module sends an update session state instruction (carrying multiple materials) to the session state module.

[0414] S1431, the session state module saves or updates the material.

[0415] S1432: The task module detects whether the material search task 1 is stopped.

[0416] In the embodiment of the present application, the user Figure 6 In the illustrated interface 601 , after the stop control 6014 is clicked or the user inputs a stop instruction via voice, the task module detects that the material search task 1 has stopped.

[0417] S1433: When the task module detects that the material search task 1 is stopped, it confirms that the execution result 2 is not effective.

[0418] S1434: The task module updates the state of the material search task 1 in the task output object to a stopped state.

[0419] It should be understood that the task module updates the status of the material search task 1 from the completed state to the stopped state.

[0420] S1435, the task module sends the execution result 5 (carrying the stop status information) to the result notifier based on the result notifier 1 object.

[0421] S1436, the result notifier records the execution result 5 in the result notifier 1 object.

[0422] S1437, the task module sends a rollback instruction to the snapshot cache module.

[0423] In the embodiment of the present application, after the YOYO assistant detects the user's operation of the stop control through S1418, the task module executes the material search task 1 normally through S1424, and saves the materials searched through the material search task 1 in the session state module. For example, the session state module saves 20 pictures of the second daughter dancing. In the case that the task module detects that the material search task 1 has been stopped through S1432, it is necessary to restore the data in the session state module to the state before the task module executes the search material task 1 through rollback processing. For example, the 20 pictures of the second daughter dancing saved in the session state module are restored to a state where the material data is empty.

[0424] In implementation, the task module can send a rollback instruction to the snapshot cache module, and the snapshot cache module performs rollback processing based on the snapshot information of the material collection to restore the data in the session state module to the state before the task module executes the search material task 1.

[0425] S1438: After receiving the rollback instruction, the snapshot cache module performs a rollback process based on the snapshot information of the material collection.

[0426] S1439: The task module sends a task output object to the task queue module (the task output object includes the task status of the material search task 1 being stopped).

[0427] S1440: The task queue module cleans up the tasks based on the task output objects.

[0428] Illustratively, the task queue module may clear the material search task 1 that is in a stopped state in the task queue.

[0429] S1441 , the task queue module sends a stop timing instruction for the material search task 1 to the timeout detector.

[0430] In an embodiment of the present application, when the task module is executing a task, the timeout detector does not detect that the execution time of the task has timed out. Therefore, the task queue module sends an instruction to the timeout detector to stop timing the task. After receiving the stop timing instruction, the timeout detector stops timing the material search task 1.

[0431] It should be understood that Figure 14 This is the process of stopping the execution of the material search task 1 during the electronic device's execution of the material search task 1. When the electronic device is performing other tasks related to video generation (such as generating a video task, saving a video task, etc.), the process of stopping the execution of the task can refer to Figure 14 .

[0432] Next, the execution process of the electronic device involved in the above scenario three is introduced.

[0433] Scenario 3 involves a situation where an electronic device encounters a task error during execution of a task related to video generation (this situation may be that the task group number is occupied). This embodiment of the application is illustrated by taking an example where the task group number of the video saving task is occupied when the electronic device issues a video saving task to a task queue. Figure 15 This is a timing diagram of an electronic device performing a video saving task provided by an embodiment of the present application.

[0434] S1501, the creation module sends the video saving task 3 to the task queue module.

[0435] It should be understood that before the creation module sends the save video task 3 to the task queue module, the creation module needs to create the save video task 3. The implementation process of the creation module creating the save video task 3 can be referred to the above embodiment and will not be repeated here.

[0436] S1502: After receiving the video saving task 3, the task queue module detects whether the task group number of the video saving task 3 is occupied.

[0437] In implementation, the task queue module can detect whether the task group number of the newly released task is occupied by detecting whether the task group number of the task already existing in the task queue is the same as the task group number of the task newly released by the task module. For example, referring to the task type and task group number shown in Table 1, assuming that there is already a task of generating a video in the task queue, the task group number corresponding to the task of generating a video is 6, when the task module releases a new task to the task queue module, for example, the new task released is to save the video, and the task group number corresponding to the task of saving the video is also 6, then the task queue module detects that task group number 6 is occupied.

[0438] S1503: When the task queue module detects that the task group number of the video task 3 is occupied, the task queue module records the task status of the task group number of the video task 3 as occupied.

[0439] S1504, the task queue module sends a notification of failure in publishing and saving video task 3 (carrying the occupancy status information of "occupied") to the creation module.

[0440] S1505: The creation module sends the occupancy status information of the video task 3 with the task group number "occupied" to the YOYO assistant.

[0441] S1506, YOYO Assistant displays interface 7 based on the occupancy status information.

[0442] For example, interface 7 can refer to Figure 7 Interface 702 is shown, and interface 702 includes text 7021 that reads “The video has not been completed and cannot be saved.”

[0443] It should be understood that after the YOYO assistant displays interface 7 through S1506, the task queue module can clear the occupied video saving tasks.

[0444] It should also be understood that Figure 15 This is the process of detecting that the task group number of the video saving task 3 is occupied when the electronic device is releasing the video saving task 3. The process of detecting that the task group number of the other task is occupied when the electronic device is releasing other tasks can be referred to Figure 15 .

[0445] Next, the execution process of the electronic device involved in the above scenario four is introduced.

[0446] Scenario 4 involves a situation where a timeout occurs when the electronic device is performing a task related to video generation. This embodiment of the application is described by taking an example where a timeout occurs when the electronic device is performing a material search task 1. Figure 16 This is a timing diagram of an electronic device performing a material search task provided by an embodiment of the present application.

[0447] S1601, the creation module sends the material search task 1 to the task queue module.

[0448] S1602: After receiving the material search task 1, the task queue module checks whether the task group number 1 of the material search task 1 is occupied.

[0449] S1603, the task queue module records the occupancy status of task group number 1 according to the detection result.

[0450] S1604 : When the task queue module determines that the occupied state of task group number 1 is unoccupied, the task queue module sends an instruction to time the material search task 1 to the timeout detector.

[0451] It should be understood that after receiving the instruction to time the material search task 1 , the timeout detector starts to time the material search task 1 .

[0452] S1605: The task queue module adds the material search task 1 to the end of the task queue.

[0453] S1606, the task queue module sends the task sequence number 1 of the material search task 1 to the creation module.

[0454] S1607, the creation module records the current task number as task number 1.

[0455] S1608, the creation module sends the execution result 1 of the material search task 1 corresponding to the task number 1 to the YOYO assistant based on the task number 1 (the execution result 1 indicates that the material search task 1 is being executed in the task queue).

[0456] S1609, based on the execution result 1, the YOYO assistant displays the interface 1 including “Searching for materials”.

[0457] S1610 , the task queue module schedules the execution of the first task (ie, material search task 1 ) in the task queue.

[0458] S1611, the task queue module sends an execution instruction of the material search task 1 to the task module.

[0459] S1612: The task module receives the execution instruction of the material search task 1 and performs pre-processing on the material search task 1.

[0460] S1613, the task module sends a snapshot processing instruction to the snapshot cache module.

[0461] S1614: The snapshot cache module obtains the material collection from the session state module.

[0462] S1615 , when the snapshot cache module obtains the material set, it performs snapshot processing on the material set to generate snapshot information of the material set.

[0463] S1616: The task module detects whether the material search task 1 is interrupted.

[0464] S1617 , when the task module detects that the material search task 1 is not interrupted, the task module executes the material search task 1 according to the task input 1 object in the material search task 1 .

[0465] S1618: The timeout detection module confirms that the execution time of the material search task 1 is greater than or equal to the preset timeout time.

[0466] For example, assuming that the preset timeout period is 30ms, after the timeout detection module receives the instruction to time the material search task 1 and starts timing the material search task 1 (for example, starting from the 0th ms), the timeout detection module sends a timeout notification to the task module at the 30th ms or the 31st ms.

[0467] S1619, the timeout detection module sends a timeout notification to the task module.

[0468] S1620, the task module generates a create task output instruction (carrying the timed-out task status) according to the task status when executing the task.

[0469] S1621, the task module sends a create task output instruction (carrying a timed-out task status) to the task output module.

[0470] S1622: The task output module creates a task output object (including the task status of the material search task 1 having timed out).

[0471] S1623, the task output module returns the task output object (carrying the timed-out task status) to the task module.

[0472] S1624, the task module generates an execution result 6 (including a timeout status) based on the task output object.

[0473] S1625, the task module detects whether the material search task 1 is stopped.

[0474] S1626 , when the task module detects that the material search task 1 has not been stopped, it confirms that the execution result 6 is effective.

[0475] S1627, the task module sends the execution result 6 (carrying the timeout status) to the result notifier based on the result notifier 1 object.

[0476] S1628, the result notifier records the execution result 6 in the result notifier 1 object.

[0477] S1629, the result notifier sends the execution result 6 (with a timeout status) to the YOYO assistant.

[0478] S1630, YOYO Assistant displays interface 7 according to execution result 6.

[0479] For example, interface 7 can refer to Figure 8 The interface 801 shown has a text 8013 displayed therein that reads “The search for material has timed out”.

[0480] S1631, the task module sends a task output object to the task queue module (the task output object includes the task status of the material search task 1 having timed out).

[0481] S1632, the task queue module cleans up the tasks based on the task output objects.

[0482] It should be understood that the task queue module clears the search material task 1 in the task queue that is in a timed-out state.

[0483] S1633: The timeout detector generates an exception log.

[0484] It should be understood that the exception log includes the reason for task timeout, faulty program node, etc.

[0485] It should be understood that Figure 16 This is the implementation process of the timeout of the search material task 1. The implementation process of other video generation related tasks when they timeout can refer to Figure 16 .

[0486] Next, the execution process of the electronic device involved in the above scenario five is introduced.

[0487] Scenario 5 involves a situation where an electronic device is interrupted while performing a task related to video generation. This embodiment of the application is described by taking the situation where an electronic device is interrupted while performing the video generation task 2 as an example. Figure 17 This is a timing diagram of another electronic device performing a video generation task provided by an embodiment of the present application.

[0488] S1701, the creation module sends the generated video task 2 to the task queue module.

[0489] S1702: After receiving the generate video task 2, the task queue module detects whether the task group number 2 of the generate video task 2 is occupied.

[0490] S1703, the task queue module records the occupancy status of task group number 2 according to the detection result.

[0491] S1704: When the task queue module determines that the occupied state of task group number 2 is unoccupied, the task queue module sends an instruction to the timeout detector to time the generation video task 2.

[0492] S1705, the task queue module adds the generated video task 2 to the end of the task queue.

[0493] S1706, the task queue module sends the task sequence number 2 of generating video task 2 to the creation module.

[0494] S1707, the creation module records the current task number as task number 2.

[0495] S1708, based on task number 2, the creation module sends the execution result 3 of the generate video task 2 corresponding to task number 2 to the YOYO assistant (execution result 3 indicates that the generate video task 2 is being executed in the task queue).

[0496] S1709, based on the execution result 3, the YOYO assistant displays the interface 3 including “Searching for materials”.

[0497] S1710: The task queue module schedules the execution of the first task in the task queue (ie, generating video task 2).

[0498] S1711, the task queue module sends an execution instruction for generating video task 2 to the task module.

[0499] S1712, the task module receives the execution instruction of generating video task 2, and performs pre-processing on generating video task 2.

[0500] S1713, the task module sends a snapshot processing instruction to the snapshot cache module.

[0501] S1714: The snapshot cache module obtains the material collection from the session state module.

[0502] S1715 , when the snapshot cache module obtains the material set, it performs snapshot processing on the material set to generate snapshot information of the material set.

[0503] S1716, the task module detects whether the video generation task 2 is interrupted.

[0504] It should be understood that when the task module executes tasks corresponding to some task group numbers, it needs to call the editing module in the media platform framework layer to execute the task. For example, referring to Table 1, when the task module executes the task corresponding to task group number 6, it needs to call the editing module to execute the task corresponding to task group number 6. When the task module executes task groups 1 to 5 and 7 to 8, it does not need to call the editing module, and the task module executes the task itself.

[0505] In implementation, the task module can detect whether the task is interrupted by detecting whether the editing module can be called. For example, when the editing module has been uninstalled or cannot be called due to an exception, when the task module executes the generate video task 6 corresponding to task group number 6 shown in Table 1, the task execution module originally needs to call the editing module to execute the generate video task 6, but because the editing module has been uninstalled or cannot be called due to an exception, the detection result can be that the generate video task 1 is interrupted.

[0506] S1717, when the task module detects that the video generation task 2 has been interrupted, it generates a create task output instruction (carrying the interrupted task status) according to the task status when executing the task.

[0507] S1718, the task module sends a create task output instruction (carrying the interrupted task status) to the task output module.

[0508] S1719, the task output module creates a task output object (including generating a task status indicating that the video task 2 has been interrupted).

[0509] S1720: The task output module returns a task output object (carrying the interrupted task status) to the task module.

[0510] S1721, the task module generates an execution result 7 (including an interrupt status) based on the task output object.

[0511] S1722, the task module detects whether the video generation task 2 is stopped.

[0512] S1723: When the task module detects that the video generation task 2 has not stopped, it confirms that the execution result 7 is effective.

[0513] S1724, the task module sends the execution result 7 (carrying the interrupt status) to the result notifier based on the result notifier 2 object.

[0514] S1725, the result notifier records the execution result 7 in the result notifier 2 object.

[0515] S1726, the result notifier sends the execution result 7 (with interrupt status) to the YOYO assistant.

[0516] S1727, YOYO Assistant displays interface 8 including pop-up window 1 according to execution result 7.

[0517] For example, interface 8 can refer to Figure 9 In the shown interface 901 , pop-up window 1 may refer to pop-up window 9011 included in the interface 901 .

[0518] S1728, the task module sends a task output object to the task queue module (the task output object includes the task status of generating video task 2 being interrupted).

[0519] S1729, the task queue module cleans up the tasks based on the task output objects.

[0520] S1730 , the task queue module sends an instruction to the timeout detector to stop timing the video generation task 2 .

[0521] S1731, the task queue module records the task sequence number of the currently interrupted task.

[0522] It should be understood that the task sequence number in the embodiment of the present application is the task sequence number of generating video task 2.

[0523] S1732, YOYO Assistant receives the user's operation on pop-up window 1.

[0524] For example, the user can operate the pop-up window 1 by Figure 9 The operation of clicking control 9012 in interface 901 is shown.

[0525] S1733, YOYO Assistant generates a continuation instruction for the current task in response to the operation on pop-up window 1.

[0526] It should be understood that the current task refers to generating video task 2.

[0527] S1734, YOYO Assistant sends a continuation instruction to the task queue module.

[0528] In some embodiments, the YOYO assistant can send a continuation instruction to the task module, and the task module will reissue the generate video task 2 to the task queue.

[0529] S1735, the task queue module reissues the interrupted task according to the recorded task sequence number of the currently interrupted task.

[0530] It should be understood that the task queue module can re-issue the interrupted generate video task 2, and then the electronic device can execute Figure 13 Steps S1226 to S1268 in .

[0531] It should be understood that Figure 17 This is the implementation process of the interruption when the electronic device performs the task 2 of generating video. When the electronic device performs other tasks related to video generation (such as the task corresponding to the task group number 6 shown in Table 1), the implementation process of the interruption can refer to Figure 17 .

[0532] The following describes an implementation process of a video generation method provided in an embodiment of the present application.

[0533] Please refer to Figure 18 , Figure 18 This is a schematic flow chart of a video generation method provided in an embodiment of the present application. This method can be executed by an electronic device or by a processor or chip within the electronic device, and is not limited in any way by the present embodiment. For ease of description, this method is described in detail using an electronic device as an example.

[0534] S1801: The electronic device generates a first task in response to a first user instruction, where the first task is used to instruct a search for a first material.

[0535] It should be understood that the first user instruction can be a user's voice or text instruction. Figure 4 In the illustrated user interface 403, a text 4032 is entered into the YOYO assistant dialog box 4031, which reads "Help me generate a video of my second daughter dancing." Alternatively, the user may enter other instructions for generating a video, such as "Help me generate a video of my trip to Beijing." The user may enter the first user instruction via voice or text, which is not limited in this embodiment of the present application.

[0536] The first task may be a material search task as shown in Table 1. The first material may be a picture or video. For example, if the first user instruction is “help me generate a video of my second daughter dancing”, the first material may be a picture or video of my second daughter dancing. The first material may refer to Figure 5 The material of the second daughter is searched in the interface 502 shown.

[0537] In the embodiment of the present application, the first user instruction can be input into the voice model, and the first task can be generated through the language model.

[0538] S1802: The electronic device saves the first task into a task queue. The tasks in the task queue are executed in the order of the received user instructions associated with the tasks.

[0539] For example, the user can enter the Figure 4 After entering the text "Help me generate a video of my second daughter dancing" in dialog box 4031 of user interface 403, the electronic device can generate a material search task based on the user instruction and then save the material search task to a task queue. The electronic device then executes the tasks sequentially according to the order in which the tasks are arranged in the task queue. For example, if Task 1 and Task 2 are arranged in order in the task queue, the electronic device will execute Task 1 first and then Task 2 after Task 1 is completed.

[0540] It should be understood that the order of arrangement of tasks in the task queue is based on user instructions. For example, the user first enters the instruction "Help me generate a video of my second daughter dancing" and then enters the instruction "Generate video". Then, the material search task generated based on the instruction "Help me generate a video of my second daughter dancing" is ranked before the video generation task generated based on the instruction "Generate video".

[0541] S1803: When the electronic device detects that the first task group number of the first task does not exist in the task queue, the electronic device executes the first task.

[0542] It should be understood that the first task group number is used to distinguish the first task. Exemplarily, the first task group number can refer to the task group numbers shown in Table 1. Assuming that the first task is a search material task and its task group number is 1, after the electronic device saves the first task in the task queue, it detects whether task group number 1 exists in the task queue. If it detects that task group number 1 does not exist, it executes the search material task.

[0543] S1804: When the first task is completed, the electronic device generates a second task in response to a second user instruction, where the second task is used to instruct generation of a first video based on the first material.

[0544] For example, when the electronic device completes the task of searching for materials, it can display Figure 5 The user interface 502 includes a generate video control 50232, and the user can click the video control 50232. The above operation of the user can be regarded as the user inputting a second user instruction to the electronic device, and the electronic device generates a generate video task in response to the second user instruction.

[0545] The first video can be referenced Figure 5 5043 shown in interface 504.

[0546] S1805: The electronic device saves the second task into the task queue.

[0547] It should be understood that in the task queue, the second task is arranged after the first task.

[0548] S1806 , when the electronic device detects that the second task group number of the second task does not exist in the task queue, the electronic device executes the second task to obtain the first video.

[0549] It should be understood that the second task group number is used to distinguish the second task. Exemplarily, the second task group number can refer to the task group numbers shown in Table 1. Assume that the second task is a generate video task and its task group number is 6. After saving the second task in the task queue, the electronic device detects whether task group number 6 exists in the task queue. If it detects that task group number 6 does not exist, it executes the generate video task to obtain the first video.

[0550] In the above technical solution, after the electronic device receives a user instruction (for example, a first user instruction), it saves the task (for example, the first task) generated based on the user instruction into a task queue. Thereafter, by executing the task in the task queue, a video (for example, the first video) that matches the user instruction is generated. That is, the present application provides a new method for generating a video. In addition, the execution order of the tasks in the above-mentioned task queue is executed in the order of the received user instructions associated with the task. In this way, it can be ensured that the execution order of the multiple tasks corresponding to the multiple user instructions in the task queue by the electronic device is executed in the order of the user instructions input by the user, and it can be ensured that after the electronic device completes the execution of the task corresponding to the first received user instruction in the task queue, it executes the task corresponding to the user instruction received after the first received user instruction in the task queue, thereby generating a video that meets the user's needs and improves the user experience.

[0551] Moreover, before executing a task in a task queue, the electronic device needs to detect whether the task group number of the task is occupied (for example, detecting whether there is a first task group number of the first task in the task queue, and detecting whether there is a second task group number of the second task in the task queue). If it is detected that the task group number of the task is not occupied (that is, when it is detected that there is no first task group number of the first task in the task queue, or when it is detected that there is no second task group number of the second task in the task queue), the task in the task queue is executed, which can ensure that the tasks in the task queue are executed normally.

[0552] In some embodiments, the electronic device generates a first task in response to a first user instruction, including:

[0553] The electronic device generates a first command in response to a first user instruction, wherein the first command carries a first session identifier and first semantic information corresponding to the first user instruction; and generates a first task based on the first command.

[0554] It should be understood that the session identifier is used to identify the session page when the user enters the session input content into the YOYO Assistant when creating a video. Different videos created by users correspond to different session identifiers. For an explanation of the session identifier, please refer to the above embodiment and will not be repeated here.

[0555] Semantic information includes information such as time, place, person, and event corresponding to the user's voice. For a detailed explanation of semantic information, please refer to the above embodiment and will not be repeated here.

[0556] In the embodiment of the present application, the first user instruction can be Figure 10 In the user voice shown in 1, the first command can be Figure 10The material search command 1 shown, the first session identifier can be Figure 10 The session identifier 1 shown in FIG, the first semantic information can be Figure 10 The semantic information 1 is shown.

[0557] It should also be understood that the electronic device can input the first user instruction into the language model to generate a first command, and input the first command into the creation module in the media platform framework layer to generate a first task.

[0558] In other embodiments, the electronic device may generate other tasks related to video generation, such as the video generation task and video storage task shown in Table 1, based on the implementation method of the embodiments of the present application.

[0559] In the embodiment of the present application, the electronic device can generate a first command according to a first user instruction, and generate a first task based on the first command, so that the user instruction can be quickly converted into a task.

[0560] In some embodiments, the electronic device generates a first task based on the first command, including:

[0561] The electronic device creates, based on the first command, first task input data of the first task, first task configuration data of the first task, a first result notifier of the first task and a first task sequence number of the first task, wherein the first task input data includes a first session identifier, first semantic information and a first task identifier, the first task identifier is obtained based on the first command, the first configuration data includes a first task name, a first task group number, a first timeout threshold and an address of a first snapshot cache, the first task name, the first task group number, the first timeout threshold and the address of the first snapshot cache are obtained based on the first command, the first result notifier is used to record the execution result of the first task, and the first task sequence number is obtained based on the first command; and generates the first task according to the first task input data, the first task configuration data, the first result notifier and the first task sequence number.

[0562] It should be understood that the implementation process of the electronic device creating the first task input data of the first task, the first task configuration data of the first task, the first result notifier of the first task and the first task serial number of the first task, as well as the implementation process of generating the first task based on the first task input data, the first task configuration data, the first result notifier and the first task serial number can refer to the implementation process of S1009 to S1024, which will not be repeated here.

[0563] In other embodiments, the electronic device may generate other tasks related to video generation, such as the video generation task and video storage task shown in Table 1, based on the implementation method of the embodiments of the present application.

[0564] In an embodiment of the present application, the electronic device can create first task input data of the first task, first task configuration data of the first task, a first result notifier of the first task and a first task serial number of the first task, and then generate the first task based on the first task input data, the first task configuration data, the first result notifier and the first task serial number. Moreover, when generating other tasks (such as the second task), they can also be generated in the above manner, and different tasks can be flexibly configured.

[0565] In some embodiments, after the electronic device detects that the second task group number of the second task does not exist in the task queue and executes the second task to obtain the first video, the electronic device further includes:

[0566] When the electronic device detects that the second task is stopped, it determines that the first execution result of the second task is not effective, the first execution result is used to indicate the result of the completion of the second task, and the first execution result includes the first video; the first video is rolled back to obtain the second execution result of the second task, and the rollback process is used to indicate that the first execution result is restored to the state before the second task is executed.

[0567] It should be understood that the first execution result of the second task refers to the result after the second task is executed normally, and the first execution result includes the generated video. The second execution result refers to the result when the second task is stopped, and the second execution result does not include the generated video.

[0568] In implementation, for example, please refer to Figure 14 The electronic device can detect whether the task group number of material search task 1 is occupied through S14002. When it is detected that the task group number of material search task 1 is not occupied (that is, when it is detected that the task group number of material search task 1 does not exist in the task queue and is not occupied), the electronic device also needs to detect whether material search task 1 is stopped through S1432. When it is detected through S1432 that material search task 1 is stopped, it is confirmed through 1433 that the execution result of material search task 1 is invalid (the execution result is that the required material is searched, such as "a picture of the second daughter dancing"). Then, through S1437 and S1438, the material in the execution result of material search task 1 is rolled back, and the electronic device can restore the data before the execution of material search task 1.

[0569] The above describes the implementation of the embodiment of the present application using the material search task as an example. When the task is the second task (i.e., the video generation task), the electronic device detects that the task has been stopped, and its implementation is the same as the above implementation, which will not be repeated. It should also be understood that when the task is the task in Table 1 of the above embodiment, the electronic device detects that the task has been stopped, and its implementation is the same as the above implementation, which will not be repeated.

[0570] In an embodiment of the present application, when it is detected that the second task is stopped, the electronic device can execute the second task normally, and then roll back the first video to obtain a second execution result of the second task. In this way, when a situation arises during the execution of the task that requires stopping the task, the electronic device can follow the original task execution process without affecting the original task execution process.

[0571] In some embodiments, the method further comprises:

[0572] A first interface is displayed, wherein the first interface includes a stop control; in response to an operation on the stop control, a stop instruction for the second task is generated; and based on the stop instruction for the second task, it is detected that the second task is stopped.

[0573] It should be understood that the first interface can refer to Figure 5 For the interface 503 in FIG. 5 , the stop control may refer to the control 5034 in the interface 503 .

[0574] In implementation, the user may click on the control 5034, and the electronic device generates a stop instruction for stopping the second task in response to the user's click operation on the control 5034. After receiving the stop instruction, the electronic device may detect that the second task is stopped.

[0575] In some embodiments, after the electronic device saves the second task to the task queue, the method further includes:

[0576] While executing the second task, the electronic device receives a third user instruction and generates a third task based on the third user instruction, where the third task is used to instruct the editing of the first video; saves the third task to a task queue; and clears the third task in the task queue when a third task group number of the third task is detected in the task queue; and displays a second interface, where the second interface is used to indicate that the third task is occupied.

[0577] The third task may be the tasks corresponding to task identifiers 6 to 10 in Table 1, and the third task may mark the first video.

[0578] In the implementation, for example, the electronic device receives a third user instruction (for example, user voice input generates a video) during the process of executing the generate video task (second task). Based on the third user instruction, the electronic device generates a save video task (third task) and saves the third task to the task queue. At this time, in the task queue, there are a generate video task and a save video task. The task group number of the generate video task is 6, and the task group number of the save video task is 6. Therefore, the task group number of the save video task already exists in the task queue. Therefore, when the electronic device detects whether the task group number of the save video task is occupied, the detection result obtained is occupied. The electronic device can clear the save video task whose task group number is occupied and display it. Figure 7 The interface 7021 (second interface) shown has a text 7021 displayed on it that reads "The video has not been completed and cannot be saved." The text 7021 is used to indicate that the third task is occupied.

[0579] In the embodiment of the present application, when it is detected that the third task group number of the third task exists in the task queue, the third task in the task queue is cleared, which can simplify the number of tasks in the task queue.

[0580] When determining that the third task group number is the same as the second task group number, the electronic device detects that the third task group number of the third task exists in the task queue.

[0581] For example, the second task group number is 6 as shown in Table 1, and the third task group number is also 6 as shown in Table 1, then the electronic device can determine that the third task group number is the same as the second task group number, then the electronic device can detect that there is a third task group number of the third task in the task queue.

[0582] In some embodiments, after the electronic device saves the second task to the task queue, the method further includes:

[0583] When the electronic device detects that the second task is interrupted, it clears the second task in the task queue; displays a third interface, which includes a first pop-up window; generates a second command in response to an operation on the first pop-up window, and the second command is used to instruct the interrupted second task to be re-published; and saves the second task to the task queue.

[0584] It should be understood that the third interface can refer to Figure 9 In the shown interface 901 , the first pop-up window can refer to 9011 in the interface 901 .

[0585] It should also be understood that when an electronic device performs some tasks, such as the task corresponding to task group number 6 in Table 1, the electronic device needs to call the internal editing application to perform the task. If the editing module has been uninstalled or an exception occurs and it cannot be called, the task will be interrupted.

[0586] In progress, please refer to Figure 17 , assuming that the editing module has been uninstalled or an abnormality occurs, the electronic device passes Figure 17 The electronic device can detect that the second task is interrupted in step S1716, and then clear the second task in the task queue in step S1729. Figure 9 The interface 901 shown, after the user clicks 9011 in the first pop-up window 901 shown in the interface 901, the electronic device can also display Figure 9 The pop-up window 9021 shown in the middle interface 902, the user can click the control 9022 in the pop-up window 9021, and in response to the user's operation on the control 9022, a second command is generated. The second command can be Figure 17 The second command of the continuation command mentioned in S1733 is used to instruct the interrupted second task to be reissued.

[0587] In some embodiments, the method is applied to an electronic device including a task queue module and a clipping module, wherein the clipping module is configured to execute the second task and save the second task to the task queue, including:

[0588] The electronic device saves the second task to the task queue in the task queue module; after the second task is saved to the task queue in the task queue module, the method further includes: if the task queue module fails to call the clipping module, detecting that the second task is interrupted.

[0589] It should be understood that when the editing module has been uninstalled or an exception occurs, the editing module cannot be called, and the task queue module will fail when calling the editing module.

[0590] In some embodiments, after the electronic device detects that the second task group number of the second task does not exist in the task queue and executes the second task to obtain the first video, the method further includes:

[0591] The electronic device receives a third user instruction and generates a third task based on the third user instruction, where the third task is used to instruct editing of the first video; saves the third task to a task queue; and when it is detected that the execution time of the third task is greater than or equal to a second timeout threshold, clears the third task in the task queue and generates an exception log.

[0592] In an embodiment of the present application, the electronic device can perform timeout detection on a task. When it is detected that a task has timed out, the electronic device clears the timed-out task and generates an exception log.

[0593] In the implementation, it is assumed that when the electronic device detects whether the third task (the third task may be a video saving task) has timed out, it can detect whether the execution time of the third task is greater than or equal to the timeout threshold. If the execution time of the third task is greater than or equal to the timeout threshold, it is detected that the third task has timed out.

[0594] In some embodiments, the method is applied to an electronic device including a task queue module and a clipping module, and executing the second task includes:

[0595] The electronic device task queue module sends an instruction to execute the second task to the editing module; the editing module executes the second task based on the instruction of the second task.

[0596] For example, please refer to Figure 13 The electronic device may send an instruction to execute the video generation task 2 to the editing module through S1241, and the editing module may execute the video generation task after receiving the instruction.

[0597] In some embodiments, the method is applied to an electronic device including a session state module and a snapshot cache module, the session state module is used to save the first material and the first video, the snapshot cache module is used to perform snapshot processing on the first material to obtain snapshot information of the first material, and to perform snapshot processing on the first video to obtain snapshot information of the first video.

[0598] For example, please refer to Figure 13 After the electronic device completes the task, it can save the pictures or materials generated by executing the task in the session state module through S1258, and the snapshot cache module can perform snapshot processing on the pictures or videos saved in the session state module to obtain snapshot information of the pictures or videos.

[0599] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0600] The present application provides a computer program product that, when executed on an electronic device, enables the electronic device to execute the technical solution in the above embodiment. The implementation principle and technical effects are similar to those of the above method-related embodiments and will not be described in detail here.

[0601] The embodiment of the present application provides a readable storage medium, which contains instructions. When the instructions are executed on an electronic device, the electronic device executes the technical solution of the above embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0602] The present application provides a chip for executing instructions. When the chip is running, the technical solution of the above embodiment is executed. The implementation principle and technical effect are similar and will not be described here.

[0603] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0604] It should be understood that the “embodiment” mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the various embodiments in the entire specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in the various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0605] In this application, elements expressed in the singular are intended to mean "one or more" rather than "one and only one" unless otherwise specified. In this application, unless otherwise specified, "at least one" is intended to mean "one or more" and "a plurality" is intended to mean "two or more."

[0606] In this document, the term "at least one of..." or "at least one of..." means all or any combination of the listed items. For example, "at least one of A, B and C" may mean: A exists alone, B exists alone, C exists alone, A and B exist at the same time, B and C exist at the same time, and A, B and C exist at the same time. A may be singular or plural, B may be singular or plural, and C may be singular or plural.

[0607] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0608] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0609] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0610] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0611] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0612] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0613] The same or similar parts between the various embodiments in this application can refer to each other. In the various embodiments in this application, and the various implementation methods / implementation methods / implementation methods in each embodiment, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment are consistent and can be referenced to each other. The technical features in different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment can be combined to form new embodiments, implementation methods, implementation methods, or implementation methods according to their inherent logical relationships. The above-described implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0614] The above is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims. In short, the above is only a preferred embodiment of the technical solution of the present application, and is not used to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A video generation method, characterized in that: include: In response to a first user instruction, generating a first task, wherein the first task is used to instruct a search for a first material; Saving the first task to a task queue, wherein the tasks in the task queue are executed in the order of the received user instructions associated with the tasks; executing the first task if it is detected that the first task group number of the first task does not exist in the task queue, where the first task group number is used to distinguish the first task; When the first task is completed, generating a second task in response to a second user instruction, wherein the second task is used to instruct generation of a first video based on the first material; Saving the second task to the task queue; When it is detected that the second task group number of the second task does not exist in the task queue, the second task is executed to obtain the first video, and the second task group number is used to distinguish the second task.

2. The method according to claim 1, characterized in that The generating of the first task in response to the first user instruction includes: In response to a first user instruction, generating a first command, wherein the first command carries a first session identifier and first semantic information corresponding to the first user instruction; Based on the first command, the first task is generated.

3. The method according to claim 2, characterized in that The generating the first task based on the first command includes: Based on the first command, first task input data of the first task, first task configuration data of the first task, a first result notifier of the first task, and a first task sequence number of the first task are created, wherein the first task input data includes the first session identifier, the first semantic information, and a first task identifier, the first task identifier is obtained based on the first command, the first configuration data includes a first task name, a first task group number, a first timeout threshold, and an address of a first snapshot cache, the first task name, the first task group number, the first timeout threshold, and the address of the first snapshot cache are obtained based on the first command, the first result notifier is used to record an execution result of the first task, and the first task sequence number is obtained based on the first command; The first task is generated according to the first task input data, the first task configuration data, the first result notifier and the first task sequence number.

4. The method according to any one of claims 1 to 3, characterized in that In the case where it is detected that the second task group number of the second task does not exist in the task queue, after executing the second task to obtain the first video, the method further includes: In a case where it is detected that the second task is stopped, determining that a first execution result of the second task is invalid, the first execution result being used to indicate a result of completion of execution of the second task, the first execution result including the first video; The first video is rolled back to obtain a second execution result of the second task, wherein the rollback process is used to indicate that the first execution result is restored to a state before the second task is executed.

5. The method according to claim 4, characterized in that The method further comprises: Displaying a first interface, wherein the first interface includes a stop control; generating a stop instruction for the second task in response to an operation on the stop control; Based on the stop instruction of the second task, it is detected that the second task is stopped.

6. The method according to any one of claims 1 to 5, characterized in that After saving the second task to the task queue, the method further includes: During the execution of the second task, a third user instruction is received, and based on the third user instruction, a third task is generated, wherein the third task is used to instruct editing of the first video; Saving the third task to the task queue; When detecting that the third task group number of the third task exists in the task queue, clearing the third task in the task queue; A second interface is displayed, where the second interface is used to indicate that the third task is occupied.

7. The method according to claim 6, characterized in that The method further comprises: When it is determined that the third task group number is the same as the second task group number, it is detected that the third task group number of the third task exists in the task queue.

8. The method according to any one of claims 1 to 7, characterized in that After saving the second task to the task queue, the method further includes: When detecting that the second task is interrupted, clearing the second task in the task queue; Displaying a third interface, wherein the third interface includes the first pop-up window; In response to an operation on the first pop-up window, generating a second command, wherein the second command is used to instruct the interrupted second task to be reissued; The second task is saved in the task queue.

9. The method according to claim 8, characterized in that The method is applied to an electronic device including a task queue module and a clipping module, wherein the clipping module is used to execute the second task, and the step of saving the second task to the task queue includes: Saving the second task to the task queue in the task queue module; After the second task is saved in the task queue in the task queue module, the method further includes: In the case that the task queue module fails to call the clipping module, it is detected that the second task is interrupted.

10. The method according to any one of claims 1 to 9, characterized in that In the case where it is detected that the second task group number of the second task does not exist in the task queue, after executing the second task to obtain the first video, the method further includes: receiving a third user instruction and generating a third task based on the third user instruction, wherein the third task is used to instruct editing of the first video; Saving the third task to the task queue; When it is detected that the execution time of the third task is greater than or equal to the second timeout threshold, the third task in the task queue is cleared and an exception log is generated.

11. The method according to any one of claims 1 to 10, characterized in that The method is applied to an electronic device including a task queue module and a clipping module, wherein the executing the second task includes: The task queue module sends an instruction to execute the second task to the editing module; The editing module executes the second task based on the instruction of the second task.

12. The method according to any one of claims 1 to 11, characterized in that The method is applied to an electronic device including a session state module and a snapshot cache module, wherein the session state module is used to store the first material and the first video, and the snapshot cache module is used to perform snapshot processing on the first material to obtain snapshot information of the first material, and to perform snapshot processing on the first video to obtain snapshot information of the first video.

13. An electronic device, characterized in that: The electronic device includes: one or more processors, and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 12.

14. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium comprises instructions, which, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 12.

Images