Video generation method, electronic equipment and computer readable storage medium

Abstract

The invention relates to the field of terminals, in particular to a video generation method, electronic equipment and a computer readable storage medium. The method comprises the following steps: in response to a first voice instruction of a user, obtaining a first material set matched with the first voice instruction; in response to a second voice instruction of the user, determining a modification type of the material according to the second voice instruction; modifying the materials in the first material set according to the modification type to obtain a modified material set; and generating a video according to the materials in the modification material set. In the embodiment of the invention, the electronic equipment can automatically modify the material according to the voice instruction of the user, so that the tedious steps of material modification in the video generation process are effectively simplified, the video generation efficiency is improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of terminals, and in particular to a video generation method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Currently, many electronic devices are equipped with cameras, allowing users to take photos and videos. Electronic devices also have communication functions, allowing users to download images and videos from the Internet or retrieve images and videos sent by other electronic devices. Users can edit images and / or videos stored on their electronic devices to create new videos from selected images and / or videos.

[0003] As an example scenario, a user inputs a user instruction to the electronic device. In response to the user instruction, the electronic device obtains corresponding materials from a gallery application according to the user instruction and generates a new video using the obtained materials.

[0004] Based on the existing footage, users may need to modify it. For example, they may want to add specific footage, delete specific footage from the current footage, or adjust the order of the footage. As the number of footage increases, manual modification becomes increasingly tedious, resulting in a more complex and inefficient way to obtain videos, which in turn affects the user experience. Summary of the Invention

[0005] The present application provides a video generation method, an electronic device, and a computer-readable storage medium, which solve the problems in the prior art of tedious manual modification of materials and low video generation efficiency.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] In a first aspect, a video generation method is provided, comprising:

[0008] In response to a first voice instruction of a user, obtaining a first material set matching the first voice instruction;

[0009] In response to a second voice instruction of the user, determining a modification type of the material according to the second voice instruction;

[0010] Modifying the materials in the first material set according to the modification type to obtain a modified material set;

[0011] A video is generated based on the materials in the modified material set.

[0012] In the embodiment of the present application, the electronic device can automatically modify the material according to the user's voice instructions, effectively simplifying the tedious steps of material modification during the video generation process, which is conducive to improving video generation efficiency and thus enhancing user experience.

[0013] In an implementation of the first aspect, modifying the material in the first material set according to the modification type to obtain a modified material set includes:

[0014] If the modification type is material addition, searching for a second material set according to the second voice instruction;

[0015] The first material set is processed with material addition according to the second material set to obtain the modified material set.

[0016] In an implementation of the first aspect, performing material addition processing on the first material set according to the second material set to obtain the modified material set includes:

[0017] performing a first screening on the second material set to obtain a first screened set;

[0018] The first material set is processed with material addition according to the first screening set to obtain the modified material set.

[0019] One implementation of the rough selection of materials may include: performing a deduplication process on the second material set, in which duplicate materials in the second material set may be deleted.

[0020] Another implementation of the rough selection of materials may include: performing deduplication processing on the second material set based on the first material set. Specifically, the materials in the second material set that are duplicated with the first material set are deleted. This method can effectively avoid duplication of new and old materials.

[0021] In an implementation of the first aspect, performing material addition processing on the first material set according to the first screening set to obtain the modified material set includes:

[0022] If the second voice instruction does not include a specified addition quantity, performing a second screening on the first filter set according to the first preset quantity to obtain a second filter set;

[0023] If the second voice instruction includes a specified additional quantity, performing a second screening on the first filter set according to the specified additional quantity to obtain a third filter set;

[0024] The first material set is subjected to material addition processing according to a target screening set to obtain the modified material set, wherein the target screening set is the second screening set or the third screening set.

[0025] In the embodiment of the present application, the material can be flexibly modified according to user instructions, which helps to improve the user experience.

[0026] In an implementation of the first aspect, performing material addition processing on the first material set according to the target screening set to obtain the modified material set includes:

[0027] If the sum of the number of materials in the target screening set and the first material set exceeds a second preset number, merging the first material set and the target screening set to obtain a merged set;

[0028] The modified material set is determined according to the merged set.

[0029] In one implementation, determining the modified material set according to the merged set includes:

[0030] Determine whether the material order has been manually adjusted; if the material order has not been manually adjusted, sort the new and old materials in the merged set according to time to obtain a modified material set; if the material order has been manually adjusted, sort the new and old materials in the merged set according to the manually adjusted material order to obtain a modified material set.

[0031] In an embodiment of the present application, the electronic device can automatically add materials according to user addition instructions and automatically adjust the quantity of materials according to different situations, so that the obtained added material set meets user expectations and helps to improve user experience.

[0032] In an implementation of the first aspect, merging the first material set and the target screening set to obtain a merged set includes:

[0033] The first material set and the target screening set are merged according to a ratio of the material quantities in the first material set and the target screening set, or the first material set and the target screening set are merged according to the second preset quantity and the specified addition quantity to obtain a merged set.

[0034] In the embodiment of the present application, the merging is performed in proportion so that the difference in the quantity of new and old materials is not too large, which is conducive to improving the effect of generating the video.

[0035] In an implementation of the first aspect, modifying the material in the first material set according to the modification type to obtain a modified material set includes:

[0036] If the modification type is material deletion, determining a deletion strategy and deletion method according to the second voice instruction;

[0037] The materials in the first material set are modified according to the deletion strategy and the deletion method to obtain the modified material set.

[0038] In an implementation of the first aspect, modifying the material in the first material set according to the deletion policy and the deletion method to obtain the modified material set includes:

[0039] If the deletion strategy is material deletion and the deletion method is semantic deletion, searching for a third material set according to the second voice instruction;

[0040] Deduplication processing is performed on the first material set and the third material set to obtain the modified material set.

[0041] In an implementation of the first aspect, modifying the material in the first material set according to the deletion policy and the deletion method to obtain the modified material set includes:

[0042] If the deletion strategy is material deletion and the deletion method is sequence number deletion, identifying the sequence number to be deleted according to the second voice instruction;

[0043] The first material set is deleted according to the sequence number to be deleted to obtain the modified material set.

[0044] In an implementation of the first aspect, modifying the material in the first material set according to the deletion policy and the deletion method to obtain the modified material set includes:

[0045] If the deletion strategy is material retention and the deletion method is semantic retention, searching for a third material set according to the second voice instruction;

[0046] The modified material set is obtained according to the intersection of the first material set and the third material set.

[0047] In an implementation of the first aspect, modifying the material in the first material set according to the deletion policy and the deletion method to obtain the modified material set includes:

[0048] If the deletion policy is material retention and the deletion method is sequence number retention, identifying the sequence number to be deleted according to the second voice instruction;

[0049] The modified material set is obtained from the first material set according to the sequence number to be retained.

[0050] In an embodiment of the present application, the electronic device can identify the corresponding deletion strategy and deletion method based on the semantic information of the user instruction, and delete the material according to the identified deletion strategy and deletion method, which greatly improves the intelligence level of material deletion in the video generation process, effectively simplifies the tediousness of user operations in the material deletion process, and is conducive to improving the user experience.

[0051] In an implementation of the first aspect, modifying the material in the first material set according to the modification type to obtain a modified material set includes:

[0052] If the modification type is to adjust the order, converting the second voice instruction to obtain adjustment information;

[0053] The order of the materials in the first material set is adjusted according to the adjustment information to obtain the modified material set.

[0054] In the embodiment of the present application, the electronic device can automatically adjust the order of the materials according to the user's instructions, which effectively simplifies the complexity of the user's operations during the video generation process and helps to improve the user experience.

[0055] In an implementation of the first aspect, adjusting the order of materials in the first material set according to the adjustment information to obtain the modified material set includes:

[0056] detecting, according to the adjustment information, whether the first field is a first key value, wherein when the first field is the first key value, it indicates that a process of adjusting the sequence of the materials is executed;

[0057] If the first field is the first key value, then determining whether the second field is empty; wherein, when the second field is not empty, it indicates that the material is inserted;

[0058] If the second field is not empty, determine whether the third field and the fourth field are valid; wherein the third field indicates the position number of the material to be adjusted, and the fourth field indicates the target position of the adjustment;

[0059] If both the third field and the fourth field are valid, the order of corresponding materials in the first material set is adjusted according to the third field and the fourth field to obtain the modified material set.

[0060] In the embodiment of the present application, the electronic device can automatically adjust the order of the materials according to the user's instructions, which effectively simplifies the complexity of the user's operations during the video generation process and helps to improve the user experience.

[0061] In an implementation of the first aspect, after determining whether the second field is empty, the method further includes:

[0062] If the second field is empty, then determine whether the fifth field is empty; wherein, when the fifth field is not empty, it indicates that the material is exchanged;

[0063] If the fifth field is not empty, determining whether the sixth field and the seventh field are valid; wherein the sixth field and the seventh field represent two lists with the same number of elements to be exchanged;

[0064] If both the sixth field and the seventh field are valid, the elements in the sixth field and the seventh field are swapped to obtain the modified material set.

[0065] In a second aspect, a chip system is provided, which is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method as described in any one of the first aspects.

[0066] According to a third aspect, an electronic device is provided, comprising one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code comprising computer instructions, and the one or more processors calling the computer instructions to enable the electronic device to execute a method as described in any one of the first aspects.

[0067] According to a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes instructions. When the instructions are executed on an electronic device, the electronic device executes the method as described in any one of the first aspects.

[0068] In a fifth aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device can implement the method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0070] Figure 2 A software structure block diagram of the electronic device 100 provided in an embodiment of the present application;

[0071] Figure 3 A schematic diagram of an interface for generating a video through a voice assistant provided in an embodiment of the present application;

[0072] Figure 4 An interactive flow chart for adding materials provided in an embodiment of the present application;

[0073] Figure 5 A schematic diagram of the method for adding materials provided in an embodiment of the present application;

[0074] Figure 6 : is an interactive flow chart of deleting material provided in an embodiment of the present application;

[0075] Figure 7 1 is a flow chart of a method for deleting material provided in an embodiment of the present application;

[0076] Figure 8 1 is an interactive flow chart of adjusting the order of materials provided in an embodiment of the present application;

[0077] Figure 9 1 is a flow chart of a method for adjusting the order of materials provided in an embodiment of the present application;

[0078] Figure 10 is a schematic diagram of a material sequence adjustment process provided in an embodiment of the present application;

[0079] Figure 11 A flowchart of the video generation method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] In the following description, specific details such as specific system structures and technologies are provided for illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details.

[0081] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0082] It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more than two; "and / or" describes the relationship between associated objects, indicating that three relationships can exist; for example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0083] In addition, in the description of this application specification and the appended claims, the terms "first", "second", "third", "fourth", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0084] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0085] The present invention provides a method for generating a video. The method can be applied to an electronic device, such as a tablet computer, a mobile phone, a wearable device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or the like. The present invention does not limit the specific type of electronic device.

[0086] Figure 1 The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0087] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0088] The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices or integrated into one or more processors. For example, the processor 110 is used to execute the video generation method in the embodiment of the present application.

[0089] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0090] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and an application required for at least one function (such as an image playback function, etc.). The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.

[0091] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0092] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. For example, in the embodiments of the present application, Figure 3 The interfaces shown are all displayed by the monitor.

[0093] The above is a specific description of the embodiments of the present application using the electronic device 100 as an example. It should be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. The electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different component configuration. The various components shown in the figure may be implemented in hardware, including one or more signal processing and / or application-specific integrated circuits, software, or a combination of hardware and software.

[0094] In addition, an operating system runs on top of the above components. The operating system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes the Android system with a layered architecture as an example to illustrate the software structure of the electronic device 100.

[0095] See also Figure 2 , is a software structure block diagram of the electronic device 100 provided in an embodiment of the present application. The layered architecture of the electronic device 100 divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime (Android Runtime) and system library, and the kernel layer.

[0096] The application layer can include a series of application packages, such as Figure 2 As shown, the application package can include applications such as a gallery, a voice assistant, a quick application engine, and a light editing service. For example, the voice assistant can parse the conversation content entered by the user and, based on the conversation parsing results, call other applications or middleware to generate videos. The quick application engine can provide a JS (JavaScript) card download service. The light editing service can generate videos based on searched static image materials and / or video materials.

[0097] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer, including various components and services to support developers' Android development. The application framework layer includes some predefined functions. Figure 2 As shown, the application framework layer may include a window manager, a content provider, a notification manager, a resource manager, a media processing middleware, a search engine, and a visual image module, etc.

[0098] The search engine can be used to search for image materials that match the conversation input content. The visual image module can determine the video theme for video generation based on the content characteristics of the image materials. The media processing middleware is used to add, delete, or sort the materials.

[0099] Image materials can include static image materials and dynamic image materials. Static image materials can be images that remain unchanged within a preset time period. Such image materials are usually captured or created in an instant and do not exhibit dynamic characteristics that change over time. Static image materials can be, for example, photographs, paintings, charts, icons, etc. Dynamic image materials (such as videos or animations) contain information about changes in the time dimension and can present movement or changes through continuous image frames.

[0100] It should be noted that the image material involved in the embodiments of the present application can be photographed by an electronic device, downloaded by the electronic device from a server, or received by the electronic device from other electronic devices. The embodiments of the present application do not limit this.

[0101] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0102] Content providers are used to store and retrieve data and make it accessible to applications. Data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0103] The resource manager can provide various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0104] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0105] The system runtime layer includes the Android Runtime and system libraries. The Android Runtime includes the core library and virtual machine. The Android Runtime is responsible for scheduling and management of the Android system.

[0106] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0107] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0108] The system library can include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0109] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0110] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0111] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0112] A 2D graphics engine is a drawing engine for 2D drawings.

[0113] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0114] It is understandable that Figure 2 The components included in the illustrated system framework layer, system library, and runtime layer do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or split certain components, or arrange the components differently.

[0115] The embodiments of the present application do not particularly limit the specific structure of the execution subject of a video generation method; as long as the code recording the video generation method of the embodiments of the present application can be executed to communicate according to the video generation method provided by the embodiments of the present application. For example, the execution subject of the video generation method provided by the embodiments of the present application can be a functional module in an electronic device that can call and execute a program, or a communication device used in an electronic device, such as a chip.

[0116] Currently, many electronic devices are equipped with cameras, allowing users to take photos and videos. Electronic devices also have communication functions, allowing users to download images and videos from the Internet or retrieve images and videos sent by other electronic devices. Users can edit images and / or videos stored on their electronic devices to create new videos from selected images and / or videos.

[0117] The following first describes an application scenario of the video generation method provided by an embodiment of the present application, which is introduced by taking the example of a voice assistant of an electronic device calling a video editing application. For the sake of convenience, the function of the electronic device generating a video according to the user's instructions in the embodiment of the present application is referred to as "smart filming".

[0118] See also Figure 3 , is a schematic diagram of the interface for generating a video through a voice assistant provided in an embodiment of the present application.

[0119] like Figure 3 As shown in (a) of the figure, the user speaks the voice "yoyo" within the voice detection range of the electronic device to wake up the voice assistant of the electronic device. After the electronic device detects the voice "yoyo", it responds with a voice "I am listening, please speak" and displays Figure 3 The interface shown in (a) in the figure. Figure 3 In the interface shown in (a), a voice assistant window 31 is displayed, which includes the text message "I'm listening, please speak." Of course, the purpose of this voice response and text message is to remind the user that the voice assistant has been awakened and to ask the user to speak the voice command. In actual applications, other voice response content and text message content can also be set according to the situation to serve the same reminder function.

[0120] The user responds to the voice "I'm listening, please speak" and / or the text message "I'm listening, please speak", and speaks the voice "Help me make a video of my daughter" within the voice monitoring range of the electronic device. After the electronic device detects the voice "Help me make a video of my daughter" (the first voice instruction), it recognizes the user's instruction to identify the character information in the user's instruction, searches for relevant materials from the gallery based on the character information, and displays the searched materials in the voice assistant window 31. Figure 3 As shown in (b) in FIG. 3 , the voice assistant window 31 displays thumbnails of multiple materials.

[0121] If the user clicks the "Generate Video" control 32, in response to the user operation, the electronic device generates a video based on the currently searched material, and after successfully generating the video, it issues a voice message "A video of my daughter has been generated for you" and displays the following information: Figure 3 The interface shown in (c) is shown in the figure. Figure 3As shown in (c) in the figure, the text "A video of my daughter has been generated for you" and the cover of the generated video are displayed in the voice assistant window. The cover displays a play control 34 and a video duration of "00:30". The video duration of "00:30" indicates that the generated video is 30 seconds long. In actual application, the user can click the play control 34 to trigger the electronic device to play the video. This application will not use illustrations as examples.

[0122] If the user clicks the "View All" control 33, in response to the user operation, the electronic device jumps to the interface of more materials in the gallery. Figure 3 As shown in (d) in FIG, a plurality of thumbnails of materials are displayed in the display window 35, and a selection control 36 is provided on each thumbnail. Figure 3 As shown in (d) in FIG, the selection control 36 of the searched material is in the selected state by default. The user can change the selected state of the material by clicking the selection control 36, thereby realizing the selection of the material.

[0123] In the application scenario described above, users may need to modify the currently searched footage. For example, they may want to add specific footage, delete specific footage from the current footage, or adjust the order of the footage. As the number of footage increases, manual modification becomes increasingly tedious, resulting in a more complex and inefficient video acquisition process, which in turn affects the user experience.

[0124] Based on this, an embodiment of the present application provides a video generation method. In this embodiment of the present application, an electronic device can automatically modify the material according to the user's voice command, effectively simplifying the tedious steps of material modification during the video generation process, which is conducive to improving video generation efficiency and thus enhancing user experience.

[0125] It should be noted that, in the embodiment of the present application, the material adjustment is performed based on the material already acquired according to the user's generate video instruction. For ease of explanation, in the embodiment of the present application, the material first searched by the electronic device according to the user's generate video instruction is referred to as the original material or the first material set.

[0126] In the embodiment of the present application, the adjustment methods for the materials include adding materials, deleting materials, and adjusting the order of materials. The above adjustment methods are described below.

[0127] First, let's introduce how to add materials. Figure 4 , is an interactive flow chart of adding materials provided in the embodiment of the present application. As an example and not a limitation, Figure 4 As shown, the interactive process of adding materials may include the following steps:

[0128] S401, in response to a user adding instruction, the voice assistant sends a material adding request to the media processing center.

[0129] A request to add content can include both voice and text commands. It carries a session ID, semantic information, and slot information. The session ID identifies the session to which the request belongs. The semantic information and slot information are obtained by the voice assistant through semantic recognition of the user's request. Slot information can include time, location, person, and event.

[0130] In some implementations, the voice assistant sends a material addition request to the media processing center through the service management framework.

[0131] S402: After receiving the material adding request, the media processing center sends a material search request to the search engine.

[0132] The material search request carries semantic information and slot information.

[0133] S403, the search engine searches for materials that match the semantic information in the gallery, and returns the searched material set (second material set) to the media processing center.

[0134] S404: The media processing center performs a rough selection based on the material set returned by the search engine to obtain a rough selected material set.

[0135] S405, the media processing center performs material selection based on the roughly selected material set to obtain a selected material set.

[0136] S406: The media processing center removes duplicates and merges the selected material set with the original material to obtain a new material. The card URL, session ID, and image data of the new material are sent to the voice assistant.

[0137] S407: The voice assistant downloads the JS card of the new material from the quick application engine.

[0138] S408, the voice assistant displays a card of the new material.

[0139] In the embodiment of the present application, the electronic device can automatically add materials according to the user's instructions, which effectively simplifies the complexity of user operations during the material addition process and helps to improve the user experience.

[0140] In some implementations, the voice assistant may choose to display at least one of the top-ranked materials in the voice assistant window.

[0141] See also Figure 5 , is a flow chart of the method for adding materials provided in the embodiment of the present application. As an example and not a limitation, Figure 5As shown, the process of executing material addition by the media processing station (such as steps S404-S406 above) may include:

[0142] S501: Search for materials according to semantic information to obtain a second material set.

[0143] like Figure 4 As described in the embodiment, the voice assistant recognizes the user-added instructions, identifies the semantic information, and sends the semantic information to the media processing center, which then searches for materials from the gallery based on the semantic information.

[0144] Semantic information can include command types and material tags. The command type indicates the type of user command, while the material tag indicates the type of material. For example, if a user adds the command "Add photos of delicious food," the semantic information is recognized to include "add" and "food." Based on "add," the user's command is determined to be for adding a material, and photos tagged "food" are searched in the gallery.

[0145] Semantic information can also include the number of materials. For example, if a user adds the instruction "add 5 photos of delicious food", the recognized semantic information includes "add", "delicious food", and "5 photos". Based on "5 photos", the number of materials to be added can be determined.

[0146] S502: Perform rough material selection (first screening) on ​​the second material set to obtain a first screened set.

[0147] One implementation of the rough selection of materials may include: performing a deduplication process on the second material set, in which duplicate materials in the second material set may be deleted.

[0148] Another implementation of the rough selection of materials may include: performing deduplication processing on the second material set based on the first material set. Specifically, the materials in the second material set that are duplicated with the first material set are deleted. This method can effectively avoid duplication of new and old materials.

[0149] It should be noted that the above two implementations can be executed one by one, in parallel, or in series, which is not specifically limited in the present embodiment.

[0150] S503: Determine whether the user's adding instruction includes a specified adding quantity.

[0151] As described in S501, after the voice assistant performs semantic recognition on the user's added instruction, it can identify the semantic information contained in the user's added instruction, such as the instruction type, material label, and material quantity. Based on the semantic information, it is determined whether the user instruction includes a specified number to be added. For example, if the user's added instruction is "add 5 photos of delicious food", the recognized semantic information includes "add", "delicious food", and "5 photos", where "5" is the specified number to be added.

[0152] S504: If the user adding instruction does not include a designated adding quantity, then the first filter set is selected based on the first preset quantity (secondary filtering) to obtain a second filter set.

[0153] In the embodiment of the present application, the first preset number may be a number of materials preset by the system. For example, the first preset number is set to 30.

[0154] One implementation method of material selection in step S504 is: if the number of materials in the current first filtering set is greater than the first preset number, the first filtering set is selected until the number of materials in the first filtering set is less than or equal to the first preset number; if the number of materials in the current first filtering set is less than or equal to the first preset number, the first filtering set is not selected.

[0155] S505: If the user adding instruction includes a specified adding quantity, materials are selected from the first filtering set according to the specified adding quantity to obtain a third filtering set.

[0156] One implementation method of material selection in step S505 is: if the number of materials in the current first filtering set is greater than the specified addition number, the first filtering set is selected until the number of materials in the first filtering set is less than or equal to the specified addition number; if the number of materials in the current first filtering set is less than or equal to the specified addition number, the first filtering set is not selected.

[0157] In steps S504 and S505, one implementation of material selection includes filtering the materials in the first screening set based on material scores. Specifically, material scores can be calculated based on indicators such as image quality, image clarity, and image resolution, and the materials in the first screening set can be filtered in descending order of material scores. In the embodiments of the present application, the method for calculating indicators such as image quality, image clarity, and image resolution is not specifically limited.

[0158] S506: After the materials are selected, it is determined whether the sum of the number of materials in the current target screening set (the second screening set or the third screening set) and the first material set exceeds a second preset number.

[0159] Wherein, the second preset number is greater than the first preset number. In the embodiment of the present application, the second preset number represents the maximum number of videos that can be used to generate a video. The second preset number can be pre-set by the system.

[0160] If the number does not exceed the second preset number, execute S508.

[0161] S507: If the number exceeds the second preset number, the first material set (original material) and the target screening set are merged to obtain a merged set.

[0162] Step S507 includes the following two situations:

[0163] Case 1: The user's add instruction does not include the specified add quantity

[0164] In this case, the merging process may include: merging based on the ratio of the number of materials in the first material set to the number of materials in the target screening set. Specifically, calculating a first number of materials that can be retained in the first material set based on the second preset number and the third preset number; calculating a second number of materials that can be retained in the target screening set based on the second preset number and the first number; selecting materials from the first material set based on the first number to obtain a third screening set; selecting materials from the target screening set based on the second number to obtain a fourth screening set; and merging the third screening set and the fourth screening set to obtain a merged set.

[0165] For example, the first material set includes M materials, the second screening set includes N materials, and the second preset number is 50. If M+N>50, then according to the formula Calculate the first quantity, and calculate the second quantity according to the formula N′=50-M′, where M′ is the first quantity, N′ is the second quantity, and C is the third preset quantity.

[0166] The third preset number can be set according to actual needs. In one example, C=25.

[0167] Case 2: The user's instruction to add a specific quantity

[0168] For this case, the merging process may include: calculating the third quantity of retainable materials in the first material set based on the second preset quantity and the specified addition quantity; selecting materials from the first material set based on the third quantity to obtain a fifth filtered set; and merging the fifth filtered set and the target filtered set to obtain a merged set.

[0169] For example, the first material set includes M materials, the second screening set includes N materials, and the second preset number is 50. If M+N>50, the third number is calculated according to the formula M′=50-N, and all materials in the second screening set are retained.

[0170] S508: Determine whether the material sequence has been manually adjusted.

[0171] In one implementation, if the user has manually adjusted the order of the materials, the electronic device will store a corresponding adjustment log, and the adjustment log can be queried to determine whether the order of the materials has been adjusted by any means.

[0172] S509: If the order of the materials has not been manually adjusted, the new and old materials in the merged set are sorted according to time to obtain an eighth material set.

[0173] For example, the order can be from earliest to latest, or from latest to earliest.

[0174] S510: If the material sequence is manually adjusted, the new and old materials in the merged set are sorted according to the manually adjusted material sequence to obtain an eighth material set.

[0175] The eighth material set is the material set after the media processing center adds the material according to the user's addition instruction. The media processing center sends the third material set to the voice assistant so that the voice assistant displays the material in the third material set to the user.

[0176] In an embodiment of the present application, the electronic device can automatically add materials according to user addition instructions and automatically adjust the quantity of materials according to different situations, so that the obtained added material set meets user expectations and helps to improve user experience.

[0177] The following describes how to delete materials. Figure 6 , is an interactive flow chart of deleting materials provided in the embodiment of the present application. As an example and not a limitation, Figure 6 As shown, the interactive process of deleting materials may include the following steps:

[0178] S601, in response to a user deletion instruction, the voice assistant sends a material deletion request to the media processing center.

[0179] A material deletion request can include semantic information and a session ID. The session ID is used to identify the session to which the current request belongs, and the semantic information is the information obtained by the voice assistant through semantic recognition of the user's added instructions.

[0180] The media processing center determines whether the material deletion request is semantic deletion or serial number deletion.

[0181] In some implementations, it is determined whether the semantic information in the material deletion request includes a material serial number; if so, it is determined to be a serial number deletion; if not, it is determined to be a semantic deletion.

[0182] S603: If it is a semantic deletion, the media processing center sends a material deletion request to the search engine.

[0183] S604: The search engine returns the third material set to be deleted to the media processing center according to the material deletion request.

[0184] In some implementations, the search engine obtains matching materials from the image library based on semantic information in the material deletion request to obtain a third material set.

[0185] S605: The media processing center deletes the material according to the third material set to obtain a fourth material set after deletion.

[0186] S606: If it is a sequence number deletion, the media processing center deletes the material according to the sequence number to be deleted in the material deletion request, and obtains the fifth material set after deletion.

[0187] S607: The media processing center sends the deleted fourth material set / fifth material set and related information to the voice assistant.

[0188] S608: The voice assistant downloads the JS card of the new material from the quick application engine.

[0189] S609, the voice assistant displays a card of the new material.

[0190] In the embodiment of the present application, the electronic device can automatically delete the material according to the user's deletion instruction, which effectively simplifies the tediousness of the user's operation during the material deletion process and helps to improve the user experience.

[0191] In some implementations, the voice assistant may choose to display at least one of the top-ranked materials in the voice assistant window.

[0192] In the embodiment of the present application, the electronic device can also retain the corresponding content in the original material according to the user's retention instruction, thereby realizing the function of automatically deleting the material. It should be noted that the interactive process of deleting the material according to the user's retention instruction in the embodiment of the present application is similar to the interactive process of deleting the material according to the user's deletion instruction. For details, please refer to Figure 6 The description in the embodiments will not be repeated here.

[0193] See also Figure 7 , is a flow chart of the method for deleting material provided in the embodiment of the present application. As an example and not a limitation, Figure 7 As shown, the process of deleting the material (such as steps S604-S606 above) may include:

[0194] S701 : In response to a user deletion instruction, determine a deletion strategy according to semantic information of the user deletion instruction.

[0195] If the deletion policy is material deletion, execute S702-706.

[0196] If the deletion policy is to retain the material, then S707 to S711 are executed.

[0197] In some implementations, semantic recognition is performed on user deletion commands to identify keywords within the command, and a deletion policy is determined based on the keywords. For example, keywords can be verbs. Accordingly, if the user deletion command is "delete daughter's image" and the keyword is identified as "delete," the deletion policy is determined to be material deletion. If the user deletion command is "retain daughter's image" and the keyword is identified as "retain," the deletion policy is determined to be material retention.

[0198] S702: If the deletion policy is material deletion, determine the deletion method according to the semantic information of the user's deletion instruction.

[0199] In some implementations, semantic recognition is performed on the user deletion instruction to identify the words contained in the user deletion instruction; it is detected whether the identified words contain numbers; if they contain numbers, the deletion method is determined to be serial number deletion; if no numbers are detected, the deletion method is determined to be semantic deletion.

[0200] In some application scenarios, such as when a user says "delete 5 pictures," the recognized word may contain a number, but the number is not a sequence number, but rather indicates quantity. To reduce the chance of false positives in the deletion method, some implementations detect whether the recognized word contains a number. If so, the number is determined to be a sequence number based on the words before and after it. If so, the deletion method is determined to be sequence number deletion; otherwise, the deletion method is determined to be semantic deletion.

[0201] S703: If the deletion method is semantic deletion, search for materials from the gallery according to the semantic information to obtain a third material set.

[0202] Semantic information can include material tags. Material tags are used to indicate the type of material. For example, if a user adds the instruction "delete photos of delicious food," the semantic information is recognized to include the material tag "delicious food," and photos tagged "delicious food" are searched in the gallery.

[0203] In some implementations, the gallery application can cluster the images in the gallery to obtain cluster information. This cluster information includes the image type and image ID of each photographed object. Accordingly, searching for images in the gallery based on semantic information involves searching the gallery for images that match the image tags in the semantic information based on the cluster information generated by the gallery application.

[0204] It is understood that users can edit the tags of the materials in the gallery. The electronic device receives the edit information input by the user, modifies / adds the tags of the corresponding materials in the gallery based on the edit information, and synchronizes the results with the clustering information of the gallery application. In this way, the accuracy of semantic search can be greatly improved.

[0205] S704: Deduplication is performed on the first material set (original material) and the third material set to obtain a fourth material set after deletion.

[0206] In some implementations, after the media processing center obtains the fourth material set, it sends the image information, card ID and other information of the materials in the fourth material set to the voice assistant for display.

[0207] In some implementations, each material in the first material set carries a material tag, and each material in the third material set carries a material tag. Accordingly, deduplication processing may include: if a first image in the first material set and a second image in the third material set have the same material tag, deleting the first image in the first material set; and using the remaining images in the first material set as the fourth material set.

[0208] In other implementations, the similarity between the first image in the first material set and the second image in the third material set can be identified; if the similarity is greater than a preset threshold, the first image in the first material set is deleted; and the remaining images in the first material set are used as the fourth material set.

[0209] S705: If the deletion method is sequence number deletion, identify the sequence number to be deleted based on the semantic information.

[0210] S706: Delete the first material set according to the sequence number to be deleted to obtain a fifth material set after deletion.

[0211] In some implementations, after the media processing center obtains the fifth material set, it sends the image information, card ID and other information of the material in the fifth material set to the voice assistant for display.

[0212] In the embodiment of the present application, during the process of obtaining the first material set, the electronic device numbers each material in the first material set. In other words, each material in the first material set corresponds to a material number. Accordingly, S706 includes: based on the material number corresponding to each material in the first material set, deleting the material in the first material set that matches the sequence number to be deleted, and determining the remaining materials in the first material set as the fifth material set.

[0213] The material matching the sequence number to be deleted can refer to a material whose corresponding material number is equal to the sequence number to be deleted, or a material whose corresponding material number is within the sequence number to be deleted. For example, if the user command is "delete images 1-8," where the sequence number to be deleted is 1-8, then the materials in the first material set whose material numbers are 1-8 can be deleted.

[0214] S707: If the deletion policy is to retain the material, determine the retention method according to the semantic information of the user's deletion instruction.

[0215] S708: If the retention mode is semantic retention, search for materials from the gallery according to the semantic information to obtain a third material set.

[0216] The implementation method of steps S707-S708 is the same as that of S702-S703. For details, please refer to the description of S702-S703 and will not be repeated here.

[0217] S709 , obtaining a sixth material set according to the intersection of the first material set and the third material set.

[0218] In some implementations, after the media processing center obtains the sixth material set, it sends the image information, card ID and other information of the materials in the sixth material set to the voice assistant for display.

[0219] In some implementations, each material in the first material set carries a material tag, and each material in the third material set carries a material tag. Accordingly, the intersection processing may include: if a first image in the first material set and a second image in the third material set have the same material tag, retaining the first image; and using the retained image in the first material set as the sixth material set.

[0220] In other implementations, the similarity between the first image in the first material set and the second image in the third material set can be identified; if the similarity is greater than a preset threshold, the first image in the first material set is retained; and the image retained in the first material set is used as the sixth material set.

[0221] S710: If the reservation mode is sequence number reservation, identify the sequence number to be reserved according to the semantic information.

[0222] S711 , obtaining a seventh material set from the first material set according to the sequence number to be retained.

[0223] In some implementations, after the media processing center obtains the seventh material set, it sends the image information, card ID and other information of the material in the seventh material set to the voice assistant for display.

[0224] In the embodiment of the present application, during the process of obtaining the first material set, the electronic device numbers each material in the first material set. In other words, each material in the first material set corresponds to a material number. Accordingly, S711 includes: obtaining, from the first material set, materials that match the sequence number to be retained, based on the material number corresponding to each material in the first material set, to obtain a seventh material set.

[0225] The material matching the sequence number to be retained can mean that the material number corresponding to the material is equal to the sequence number to be retained, or it can also mean that the material number corresponding to the material belongs to the sequence number to be retained. For example, if the user instruction is "Retain images 1-8", where the sequence number to be retained is 1-8, then the materials in the first material set with material numbers 1-8 will be retained.

[0226] In an embodiment of the present application, the electronic device can identify the corresponding deletion strategy and deletion method based on the semantic information of the user instruction, and delete the material according to the identified deletion strategy and deletion method, which greatly improves the intelligence level of material deletion in the video generation process, effectively simplifies the tediousness of user operations in the material deletion process, and is conducive to improving the user experience.

[0227] It should be noted that Figure 7 The fourth material set, the fifth material set, the sixth material set and the seventh material set in the embodiment may all be referred to as modified material sets.

[0228] In some application scenarios, during the "smart filming" process, the electronic device can adjust the order of the searched materials (original materials) according to user instructions. For example, "put the nth photo at the front", "swap the nth photo with the mth photo", "put the last 10 photos at the front", etc.

[0229] The media processing center adjusts the order according to the request issued by the voice assistant, and then sends the adjusted material order to the voice assistant for display.

[0230] See also Figure 8 , is an interactive flow chart of adjusting the material sequence provided by the embodiment of this application. Figure 8 As shown, the interaction process may include:

[0231] S801, in response to the user's sequence update instruction, the voice assistant sends a material adjustment request to the media processing center.

[0232] In some implementations, the format of the material adjustment request sent by the voice assistant is as follows:

[0233]

[0234] Among them, "command" is the name of the command issued by the voice assistant. For the material adjustment function, it is a fixed value. "sub-action": For the material adjustment order, this value is also a fixed value. "parameter" represents the adjustment program parameter parsed according to the user's semantics. "insert" represents the parameter content for the semantic insertion scenario. If the user input is not an insert command, the parameter content is empty. "materials_list" represents the position sequence number set of the materials whose order needs to be adjusted. "position" means adjusting to the front of a certain element. If the user says: "Insert at the end", y is the number of all elements + 1. "change" is the parameter content for the semantic exchange scenario. If the content input is not an exchange command, the parameter content is empty. "materials_0" represents the element list 0 to be exchanged. "materials_1" represents the element list 1 to be exchanged / / The number of elements in these two material lists is the same.

[0235] S802: The media processing center updates the material sequence according to the material adjustment request.

[0236] S803: The media processing center sends the updated material and related data to the voice assistant.

[0237] In some implementations, the command format returned by the media center is as follows:

[0238]

[0239]

[0240] Among them, "isManualAdjust" indicates whether the material has been manually adjusted. "materials" indicates the material list returned to the YOYO Assistant. "file_id" uniquely identifies a material. "media_type" indicates whether the material type is a photo or video. "duration" is a field that identifies the video duration for video materials. "coverData" indicates the cover data of the material. "isSelected" indicates whether the material is selected on the card.

[0241] S804: The voice assistant downloads the JS card of the new material from the quick application engine.

[0242] S805, the voice assistant displays a card of the new material.

[0243] In the embodiment of the present application, the electronic device can automatically adjust the order of the materials according to the user's instructions, which effectively simplifies the complexity of the user's operations during the video generation process and helps to improve the user experience.

[0244] See also Figure 9 , is a flow chart of the method for adjusting the material sequence provided in the embodiment of the present application. As an example and not a limitation, Figure 9 As shown, the material sequence adjustment method may include:

[0245] S901 , in response to a user's sequence update instruction, convert the sequence update instruction into adjustment information.

[0246] In one example, the supported voice scripts for material adjustment are:

[0247] 1. Insert: Place the xth element at the yth position.

[0248] 2. Insert: Place the xth element after the yth element.

[0249] 3. Insert: Place the xth element before the yth element.

[0250] 4. Insert: put the xth element at the beginning / end.

[0251] 5. Exchange: Swap the positions of the x-th element and the y-th element.

[0252] 6. Insert: Put the elements x1, x2, ... at the end.

[0253] 7. Insert: Put the elements x1, x2, ... at the front.

[0254] 8. Insert: Place the x1, x2, … elements before / after the y element.

[0255] 9. Exchange: swap the positions of x1 and y1, x2 and y2, ...

[0256] Among them, "insert" and "swap" before ":" represent the semantic information after conversion, and the voice words after ":" are the sequential update instructions before conversion.

[0257] S902: Adjust the order of the materials in the first material set according to the converted adjustment information.

[0258] As an example, see Figure 10 , is a schematic diagram of the material sequence adjustment process provided in the embodiment of the present application. As an example and not a limitation, Figure 10 As shown, S902 may include:

[0259] S1001, check whether the subaction field (first field) is sequence (first key value).

[0260] As mentioned above, when the subaction field is sequence, it indicates the material sequence adjustment logic; when the subaction field is not sequence, it indicates other material adjustment logic.

[0261] S1002: If the subaction field is not sequence, it means that the material sequence adjustment logic is not used, and other material modification logic is called.

[0262] S1003: If the subaction field is sequence, indicating that the material sequence adjustment logic is adopted, it is determined whether the insert field (the second field) is empty.

[0263] When the insert field is not empty, it means that the user input is an insert instruction; when the insert field is empty, it means that the user input is not an insert instruction.

[0264] S1004: If the insert field is not empty, determine whether the materials_list field (third field) and the position field (fourth field) are valid.

[0265] The materials_list field indicates the position number of the material whose order is to be adjusted, and the position field indicates the target position of the adjustment.

[0266] S1005: If both the materials_list field and the position field are valid, place the elements in materials_list before the element specified by position.

[0267] If either the materials_list field or the position field is invalid, execute S1009.

[0268] S1006: If the insert field is empty, determine whether the change field (fifth field) is empty.

[0269] When the change field is not empty, it means that the user input is a swap instruction; when the change field is empty, it means that the user input is not a swap instruction.

[0270] S1007: If the change field is not empty, determine whether the materials_0 field (sixth field) and the materials_1 field (seventh field) are valid.

[0271] If the change field is empty, execute S1009.

[0272] The materials_0 field indicates the first material list to be exchanged, and the materials_1 field indicates the second material list to be exchanged. The number of elements in the two lists is the same.

[0273] S1008: If both the materials_0 field and the materials_1 field are valid, swap the positions of the elements in the materials_0 and materials_1 lists.

[0274] If either the materials_0 field or the materials_1 field is invalid, execute S1009.

[0275] S1009, returns an error code.

[0276] It should be noted that the voice assistant generates a material adjustment request based on the user's sequential update instruction, and the material adjustment request includes the aforementioned fields 1 to 7. In other words, the first to seventh fields can be obtained based on the user's sequential update instruction. Of course, the adjustment information can also include fields 1 to 7.

[0277] In the embodiment of the present application, the electronic device can automatically adjust the order of the materials according to the user's instructions, which effectively simplifies the complexity of the user's operations during the video generation process and helps to improve the user experience.

[0278] In one embodiment, see Figure 11 , is a flow chart of the video generation method provided in the embodiment of the present application. As an example and not a limitation, Figure 11 As shown, the video generation method may include:

[0279] S1101: In response to a first voice instruction of a user, obtain a first material set matching the first voice instruction.

[0280] S1102: In response to a second voice instruction of the user, determine a modification type of the material according to the second voice instruction.

[0281] S1103: Modify the materials in the first material set according to the modification type to obtain a modified material set.

[0282] S1104: Generate a video based on the materials in the modified material set.

[0283] In this embodiment of the present application, the second voice command can be Figure 4 In the embodiment, the user adds an instruction, Figure 6 In the embodiment, the user deletes the instruction or Figure 9The sequential update instruction in the embodiment. Accordingly, when the second voice instruction is a user add instruction, the modified material set may be the third material set; when the second voice instruction is a user delete instruction, the modified material set may be the fourth material set, the fifth material set, the sixth material set, or the seventh material set; when the second voice instruction is a sequential update instruction, the modified material set may be the material set obtained in S1005 or S1008.

[0284] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0285] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is run on an electronic device, it can implement the steps in the above-mentioned various method embodiments.

[0286] The embodiments of the present application further provide a computer program product. When the computer program product is run on an electronic device or a wireless router, the electronic device can implement the steps in the above-mentioned various method embodiments.

[0287] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include at least: any entity or device capable of carrying the computer program code to the first device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0288] The present application also provides a chip comprising a processor coupled to a memory, wherein the processor invokes a computer program stored in the memory to implement the steps of any method embodiment of the present application. The chip may be a single chip or a chip module composed of multiple chips.

[0289] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0290] Those skilled in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0291] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A video generation method, characterized in that: include: In response to a first voice instruction of a user, obtaining a first material set matching the first voice instruction; In response to a second voice instruction of the user, determining a modification type of the material according to the second voice instruction; Modifying the materials in the first material set according to the modification type to obtain a modified material set; A video is generated based on the materials in the modified material set.

2. The method according to claim 1, characterized in that The step of modifying the material in the first material set according to the modification type to obtain a modified material set includes: If the modification type is material addition, searching for a second material set according to the second voice instruction; The first material set is processed with material addition according to the second material set to obtain the modified material set.

3. The method according to claim 2, characterized in that The performing material addition processing on the first material set according to the second material set to obtain the modified material set includes: performing a first screening on the second material set to obtain a first screened set; The first material set is processed with material addition according to the first screening set to obtain the modified material set.

4. The method according to claim 3, characterized in that The performing material addition processing on the first material set according to the first screening set to obtain the modified material set includes: If the second voice instruction does not include a specified addition quantity, performing a second screening on the first filter set according to the first preset quantity to obtain a second filter set; If the second voice instruction includes a specified additional quantity, performing a second screening on the first filter set according to the specified additional quantity to obtain a third filter set; The first material set is subjected to material addition processing according to a target screening set to obtain the modified material set, wherein the target screening set is the second screening set or the third screening set.

5. The method according to claim 4, characterized in that The performing material addition processing on the first material set according to the target screening set to obtain the modified material set includes: If the sum of the number of materials in the target screening set and the first material set exceeds a second preset number, merging the first material set and the target screening set to obtain a merged set; The modified material set is determined according to the merged set.

6. The method according to claim 5, characterized in that The merging of the first material set and the target screening set to obtain a merged set includes: The first material set and the target screening set are merged according to a ratio of the material quantities in the first material set and the target screening set, or the first material set and the target screening set are merged according to the second preset quantity and the specified addition quantity to obtain a merged set.

7. The method according to claim 1, characterized in that The step of modifying the material in the first material set according to the modification type to obtain a modified material set includes: If the modification type is material deletion, determining a deletion strategy and deletion method according to the second voice instruction; The materials in the first material set are modified according to the deletion strategy and the deletion method to obtain the modified material set.

8. The method according to claim 7, characterized in that The step of modifying the material in the first material set according to the deletion strategy and the deletion method to obtain the modified material set includes: If the deletion strategy is material deletion and the deletion method is semantic deletion, searching for a third material set according to the second voice instruction; Deduplication processing is performed on the first material set and the third material set to obtain the modified material set.

9. The method according to claim 7, characterized in that The step of modifying the material in the first material set according to the deletion strategy and the deletion method to obtain the modified material set includes: If the deletion strategy is material deletion and the deletion method is sequence number deletion, identifying the sequence number to be deleted according to the second voice instruction; The first material set is deleted according to the sequence number to be deleted to obtain the modified material set.

10. The method according to claim 7, characterized in that The step of modifying the material in the first material set according to the deletion strategy and the deletion method to obtain the modified material set includes: If the deletion strategy is material retention and the deletion method is semantic retention, searching for a third material set according to the second voice instruction; The modified material set is obtained according to the intersection of the first material set and the third material set.

11. The method according to claim 7, characterized in that The step of modifying the material in the first material set according to the deletion strategy and the deletion method to obtain the modified material set includes: If the deletion policy is material retention and the deletion method is sequence number retention, identifying the sequence number to be deleted according to the second voice instruction; The modified material set is obtained from the first material set according to the sequence number to be retained.

12. The method according to claim 1, characterized in that The step of modifying the material in the first material set according to the modification type to obtain a modified material set includes: If the modification type is to adjust the order, converting the second voice instruction to obtain adjustment information; The order of the materials in the first material set is adjusted according to the adjustment information to obtain the modified material set.

13. The method according to claim 12, characterized in that The adjusting the order of the materials in the first material set according to the adjustment information to obtain the modified material set includes: detecting, according to the adjustment information, whether the first field is a first key value, wherein when the first field is the first key value, it indicates that a process of adjusting the sequence of the materials is executed; If the first field is the first key value, then determining whether the second field is empty; wherein, when the second field is not empty, it indicates that the material is inserted; If the second field is not empty, determine whether the third field and the fourth field are valid; wherein the third field indicates the position number of the material to be adjusted, and the fourth field indicates the target position of the adjustment; If both the third field and the fourth field are valid, the order of corresponding materials in the first material set is adjusted according to the third field and the fourth field to obtain the modified material set.

14. The method according to claim 13, characterized in that After determining whether the second field is empty, the method further includes: If the second field is empty, then determine whether the fifth field is empty; wherein, when the fifth field is not empty, it indicates that the material is exchanged; If the fifth field is not empty, determining whether the sixth field and the seventh field are valid; wherein the sixth field and the seventh field represent two lists with the same number of elements to be exchanged; If both the sixth field and the seventh field are valid, the elements in the sixth field and the seventh field are swapped to obtain the modified material set.

15. An electronic device, characterized in that: The electronic device includes one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program code, where the computer program code includes computer instructions. The one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 14.

16. A computer-readable storage medium, characterized in that The computer-readable storage medium includes instructions, and when the instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 14.

17. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions to enable the electronic device to execute the method as described in any one of claims 1 to 14.

Images