Video generation method and related equipment

Abstract

The invention provides a video generation method and related equipment. The method comprises the steps of obtaining a user instruction; generating a prompt text based on semantic information of the user instruction, wherein the prompt text comprises at least one target object; obtaining a target material based on the prompt text, wherein the target material comprises an image material and / or a video material; and generating a target video based on the target material.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a video generation method and related equipment. Background Art

[0002] At present, when a vehicle is moving, users can only use mobile devices, such as mobile phones or driving recorders, to capture images or videos along the way. However, these mobile devices are located inside the vehicle or in a fixed position, and the images or videos captured are often not good. The shooting time needs to be manually selected, and it is impossible to automatically capture the images or videos that the user expects. This not only increases the user's workload, but also distracts the user's attention and reduces driving safety. Moreover, when users want to synthesize images or videos along the way, they can only edit them manually, which brings a lot of work to the user and reduces the efficiency of video generation. Summary of the invention

[0003] The present disclosure proposes a video generation method, device, equipment, storage medium and program product to solve the technical problem that it is impossible to automatically capture the image or video desired by the user during driving to a certain extent.

[0004] In a first aspect, the present disclosure provides a video generation method, comprising:

[0005] Get user instructions;

[0006] generating a prompt text based on the user instruction, wherein the prompt text includes at least one target object;

[0007] Acquire target material based on the prompt text, the target material including image material and / or video material;

[0008] A target video is generated based on the target material.

[0009] In a second aspect of the present disclosure, a video generating device is provided, comprising:

[0010] The acquisition module is used to obtain user instructions;

[0011] A prompt text module, used to generate a prompt text based on the user instruction, wherein the prompt text includes at least one target object;

[0012] An acquisition module, used for acquiring target material based on the prompt text, wherein the target material includes image material and / or video material;

[0013] A video generation module is used to generate a target video based on the target material.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, characterized in that it includes one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the program includes instructions for executing the method described in the first aspect or the second aspect.

[0015] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium containing a computer program is provided. When the computer program is executed by one or more processors, the processors execute the method described in the first aspect or the second aspect.

[0016] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method described in the first aspect.

[0017] From the above, it can be seen that the video generation method and related devices provided by the present disclosure generate prompt text by semantic analysis of user instructions, and obtain target materials based on the prompt text, thereby generating a target video. It is possible to automatically obtain the image or video desired by the user, help the user free his hands, and automatically generate the target video, thereby improving the effect and efficiency of video generation and reducing the workload of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of a video generation architecture according to an embodiment of the present disclosure.

[0020] Figure 2 The figure is a schematic diagram of the hardware structure of an exemplary electronic device according to an embodiment of the present disclosure.

[0021] Figure 3 The present invention is a schematic flowchart of a video generation method according to an embodiment of the present invention.

[0022] Figure 4 It is a schematic diagram of an interactive interface of an embodiment of the present disclosure.

[0023] Figure 5 Schematic diagram of a video generating device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0025] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0026] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0027] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0028] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0030] Figure 1 FIG. 1 is a schematic diagram showing a video generation architecture of an embodiment of the present disclosure. Figure 1The video generation architecture 100 may include a server 110, a terminal 120, and a network 130 providing a communication link. The server 110 and the terminal 120 may be connected via a wired or wireless network 130. The server 110 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, security services, and CDN.

[0031] The terminal 120 may be implemented in hardware or software. For example, when the terminal 120 is implemented in hardware, it may be various electronic devices having a display screen and supporting page display, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, and desktop computers, etc. When the terminal 120 device is implemented in software, it may be installed in the electronic devices listed above; it may be implemented as multiple software or software modules (such as software or software modules used to provide distributed services), or it may be implemented as a single software or software module, which is not specifically limited here.

[0032] It should be noted that the video generation method provided in the embodiment of the present application can be executed by the terminal 120 or by the server 110. It should be understood that Figure 1 The number of terminals, networks and servers in the embodiment is only for illustration and is not intended to limit the number of terminals, networks and servers. Any number of terminals, networks and servers may be provided as required.

[0033] Figure 2 FIG. 2 shows a schematic diagram of the hardware structure of an exemplary electronic device 200 provided in an embodiment of the present disclosure. Figure 2 As shown, the electronic device 200 may include: a processor 202, a memory 204, a network module 206, a peripheral interface 208 and a bus 210. The processor 202, the memory 204, the network module 206 and the peripheral interface 208 are connected to each other in communication within the electronic device 200 through the bus 210.

[0034] Processor 202 may be a central processing unit (CPU), a video generator, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. Processor 202 may be used to perform functions related to the technology described in the present disclosure. In some embodiments, processor 202 may also include multiple processors integrated into a single logical component. For example, Figure 2 As shown, processor 202 may include a plurality of processors 202a, 202b, and 202c.

[0035] The memory 204 may be configured to store data (eg, instructions, computer code, etc.). Figure 2 As shown, the data stored in the memory 204 may include program instructions (e.g., program instructions for implementing the video generation method of the embodiment of the present disclosure) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). The processor 202 may also access the program instructions and data stored in the memory 204, and execute the program instructions to operate on the data to be processed. The memory 204 may include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 204 may include a random access memory (RAM), a read-only memory (ROM), an optical disk, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0036] The network module 206 can be configured to provide communication with other external devices to the electronic device 200 via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC) etc.), a cellular network, the Internet or a combination thereof. It is understood that the type of network is not limited to the above specific examples. In some embodiments, the network module 306 can include any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, etc., in any combination.

[0037] The peripheral interface 208 can be configured to connect the electronic device 200 to one or more peripheral devices to achieve information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touch pad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.

[0038] The bus 210 can be configured to transmit information between various components of the electronic device 200 (e.g., the processor 202, the memory 204, the network module 206, and the peripheral interface 208), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0039] It should be noted that, although the architecture of the electronic device 200 only shows the processor 202, the memory 204, the network module 206, the peripheral interface 208 and the bus 210, in the specific implementation process, the architecture of the electronic device 200 may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the architecture of the electronic device 200 may also only include the components necessary for implementing the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0040] During the driving process of the vehicle, when the user wants to shoot images or videos along the way, he can only do so based on a mobile device, such as a mobile phone or a driving recorder. When the user sees the desired picture and wants to record it, he shoots the image or video through the mobile phone, or controls the driving recorder by voice to shoot. However, these mobile devices are located in the car or in a fixed position, and the effect of the images or videos taken is often not good; it is necessary to manually select the shooting time, for example, when the user finds the current scenery and wants to record it, he shoots it with a mobile phone, and it is impossible to automatically shoot the images or videos that the user expects. This not only increases the workload of the user, but also distracts the user's attention and reduces the safety of driving. Moreover, when the user expects to synthesize the images or videos along the way, he can only manually edit them, which brings a lot of work to the user and reduces the efficiency of video generation. This results in the inability to automatically shoot the images or videos that the user expects during driving, and the manual shooting and manual editing have a large workload, poor effect, and low efficiency. Therefore, how to automatically shoot the images or videos that the user expects, and improve the effect and efficiency of video synthesis have become technical problems that need to be solved urgently.

[0041] In view of this, the embodiments of the present disclosure provide a video generation method and related devices. A prompt text is generated by semantic analysis of user instructions, and a target material is obtained based on the prompt text, thereby generating a target video. The method can automatically obtain the image or video desired by the user, help the user free his hands, and automatically generate the target video, thereby improving the effect and efficiency of video generation and reducing the workload of the user.

[0042] See also Figure 3 , Figure 3 A schematic flowchart of a video generation method according to an embodiment of the present disclosure is shown. Figure 3 In the embodiment, the video generation method 300 can be deployed in a vehicle-mounted terminal and can further include the following steps.

[0043] In step S310, a user instruction is obtained.

[0044] The user instruction may refer to an instruction sent by the user through an input device, which may be a text input, a voice input, etc. The user generates a corresponding user voice instruction based on voice input, and the user voice instruction may have semantics. Specifically, the user may input the user instruction based on the interactive interface. Figure 4 , Figure 4 A schematic diagram of an interactive interface according to an embodiment of the present disclosure is shown. Figure 4 In the embodiment, the interactive interface 400 can realize the interaction between the vehicle terminal (such as the central control device of the vehicle) and the user. Specifically, the vehicle terminal may include an interactive system for realizing functions such as awakening, interaction, voice timbre recognition, voice-to-text, display and save chat records, etc. For example, the user can input a voice awakening instruction through an input device (such as a microphone) in the vehicle terminal, and display the awakening instruction 411 in the interactive interface 400. The awakening instruction 411 may include an awakening logo for awakening the interactive system to work. The interactive system in the vehicle terminal can display the awakening reply 421 in the interactive interface 400 in response to the awakening instruction 411, and output the awakening reply 421. At this time, the interactive system starts to work, and the user can continue to input the user instruction 412, and display the user instruction 412 in the interactive interface 400. The user instruction 412 can be used to instruct the vehicle terminal to generate a video. For example, the user instruction 412 can be "record my journey and edit it into a good-looking video." Accordingly, the interactive system can display the instruction response 422 in the interactive interface 400 in response to the user instruction 412 and output the instruction response 422 .

[0045] In step S320, a prompt text is generated based on the user instruction, and the prompt text includes at least one target object.

[0046] Among them, the prompt text may refer to a text that is semantically related to the user instruction and includes a specific target object. The target object may refer to various objects, such as people, animals, buildings, significant objects, etc. Since the user instruction given by the user cannot exhaustively list all the things that are desired to be photographed, it is often relatively abstract. For example, "record the journey", it is not expected that all journeys are expected to be recorded. Therefore, the semantics of the user instruction can be understood based on a text generation model, such as an AIGC (Artificial Intelligence Generated Content, artificial intelligence generated text) language model, to generate an accurate, complete and relatively specific prompt text. For example, the text generation model can understand the user instruction 412 "record the journey for me and edit it into a good-looking video", and believe that the beautiful scenery outside the car and the happy time inside the car during the journey are what the user expects to record. It can generate a prompt text "Pictures and videos worth recording during the journey will include snow-capped mountains, sunsets, rainbows outside the car, and happy moments inside the car. These will be captured for you", and the prompt text includes specific target objects, such as snow-capped mountains, sunsets, rainbows, and happy scenes inside the car. The prompt text can be displayed in the interactive interface and can also be played in voice form. It should be understood that the above prompt text is only an example and is not intended to limit it. The prompt text may also include more or fewer target objects, which is not limited here.

[0047] In some embodiments, generating a prompt text based on the user instruction may include:

[0048] The user instruction is input into a text generation model to obtain the prompt text; the prompt text also includes at least one of the following: shooting time, shooting duration, shooting angle, and target condition.

[0049] Among them, the prompt text can be one or more, and each prompt text can include one or more target objects. Further, each prompt text can include at least one of the following: shooting time, shooting duration, shooting angle, target condition. The target object or target condition of each prompt text can be partially the same or all the same or all different, and the shooting time, shooting duration, and shooting angle of each prompt text can also be the same or different. Specifically, the prompt text can include a first prompt text, and the first prompt text can include a first target object, and can also include at least one of a first shooting time, a first shooting duration, a first shooting angle, and a first target condition; the second prompt text can include a second target object, and can also include at least one of a second shooting time, a second shooting duration, a second shooting angle, and a second target condition. For example, a user can issue a user instruction "Record today's street corner" when going out at 8:00 in the morning, and generate n prompt texts text_1, ..., text_i, ... text_n, where i and n are positive integers. Among them, the prompt text text_1 may include: 10 seconds (shooting duration) of video material including a person (target object 2) at a road corner (target object 1) shot by the front camera and the right camera (shooting angle) at 8:00-10:00 in the morning (shooting time). The prompt text text_i may include: 20 seconds (shooting duration) of video material including a vehicle (target object 2) or a plant (target object 3) at a crossroad (target object 1) shot by the front camera (shooting angle) at 12:00-13:00 in the afternoon (shooting time).

[0050] In some embodiments, method 300 may further include:

[0051] Outputting the prompt text;

[0052] In response to receiving a regeneration instruction for the prompt text, a new prompt text is regenerated based on the user instruction.

[0053] Among them, when the user is not satisfied with the generated prompt text, a regeneration instruction can be issued to regenerate the prompt text. When the vehicle-mounted terminal receives a regeneration instruction from the user for one or several prompt texts via the interactive system, a new prompt text can be regenerated based on the user's instruction. Furthermore, the user can also update one or several prompt texts based on the corresponding instructions, such as adding, deleting or replacing some text in the prompt text (such as target object, shooting time, shooting duration, shooting angle, target conditions, etc.) to obtain a new prompt text. Furthermore, the vehicle-mounted terminal can regenerate new prompt texts multiple times until the user is satisfied, for example, the user issues an instruction to start video generation via the interactive system.

[0054] In step S330, target material is acquired based on the prompt text, and the target material includes image material and / or video material.

[0055] The prompt text includes one or more target objects, and the vehicle terminal can obtain target materials based on the target objects. For example, when the prompt text includes snow-capped mountains, sunsets, rainbows, and happy scenes inside the car, the vehicle terminal will record the scenes outside the car containing snow-capped mountains, sunsets, rainbows, and the scenes inside the car containing laughter as target materials.

[0056] In some embodiments, obtaining the target material based on the prompt text includes:

[0057] Based on the shooting time in the prompt text, the target material including the target object is acquired in real time, and / or based on the shooting time in the prompt text, the target material including the target object is acquired from historical materials.

[0058] Among them, the shooting time may refer to the time when the material is shot. The user instruction may include time-related information related to the shooting time, and the text generation model generates the shooting time based on the time-related information. For example, the text generation model may generate a prompt text with the shooting time of 19:00-05:00 based on the user instruction "Record the starry sky". The user instruction may also not include information related to the shooting time, and the text generation model may generate a specified time as the shooting time. For example, the text generation model may generate a prompt text with the shooting time of 00:00-24:00 (i.e., 24 hours a day) or 07:00-24:00 based on the user instruction "Record the fireworks in the city". The target material can be collected in real time based on the shooting time in the prompt text. For example, if the shooting time in the prompt text is 19:00-05:00, the image material or video material including the starry sky can be collected at 19:00-05:00. It can also be obtained from historical materials collected historically. For example, the user instruction "Edit a good-looking journey video" can be to obtain the target material containing the target object from historical journey images and historical journey videos.

[0059] In some embodiments, obtaining the target material based on the prompt text includes:

[0060] Acquire the target material having the shooting duration and including the target object in real time;

[0061] and / or,

[0062] The target material having the shooting duration and including the target object is acquired from historical materials.

[0063] Among them, the shooting duration can refer to the duration of shooting material. The text generation model can generate the corresponding shooting duration based on the target object. Specifically, when the target object is an object such as the sun, the moon, the starry sky, etc. that usually changes less within a certain period of time, its shooting duration can be determined to be shorter, for example, t1 (t1 is a positive integer) seconds. When the target object is an object such as a street or the scenery beside the road that changes faster within a certain period of time, its shooting duration can be determined to be t2 (t2 is a positive integer) minutes or 60*t2 seconds. It should be understood that the above-mentioned shooting duration t1 seconds or t2 minutes are examples and are not intended to limit the shooting duration. The shooting duration can also be longer or shorter, which is not limited here. Video material including the target object can be shot in real time based on the shooting duration, or video material with the shooting duration and including the target object can be selected from historical materials. Furthermore, the prompt text may include the shooting time and shooting duration, and the video material including the target object can be shot in real time at the shooting time based on the shooting duration, or the video material with the shooting duration and including the target object can be obtained from the historical material at the shooting time.

[0064] In some embodiments, obtaining the target material based on the prompt text includes:

[0065] Determine a target image acquisition device based on the shooting angle;

[0066] The target material including the target object acquired from the target image acquisition device is acquired.

[0067] Among them, the shooting angle may refer to the angle of shooting the material. The text generation model may determine the presentation angle of the target object, thereby obtaining the desired material. Specifically, one or more image acquisition devices, such as cameras, may be deployed inside and outside the vehicle. A target recognition algorithm may be deployed in each image acquisition device to identify the target object. The vehicle terminal sends the prompt text to each image acquisition device, and the image acquisition device recognizes its own real-time picture. When the real-time picture includes the target object in the prompt text, the image acquisition device triggers itself to shoot the current real-time picture as a target image acquisition device to obtain the target material. The target material is stored in the storage area corresponding to the target image acquisition device. The position of the image acquisition device is pre-set, and the shooting angle can be reflected based on the position of the image acquisition device. For example, the vehicle can set up image acquisition devices in four directions: front, back, left, and right. The text generation model can determine that the target object is an intersection. If you want to capture the panoramic view of the intersection, you need to use the image acquisition device located in front of or behind the vehicle. Then, you can determine that the shooting angle is from the front side of the vehicle or from the rear side of the vehicle. Furthermore, the image acquisition device located in front of or behind the vehicle can be determined as the target image acquisition device. When the target image acquisition device recognizes an intersection, it starts to acquire video material and sends the acquired video material including the intersection to the vehicle terminal.

[0068] In step S340, a target video is generated based on the target material.

[0069] The generated target video can be displayed via an interactive interface, such as Figure 4 The target video 423 in the video. The user can click on the target video 423 to play it.

[0070] Furthermore, in some embodiments, generating a target video based on the target material includes:

[0071] Filter the target material based on the target condition to obtain candidate materials;

[0072] Matching a target template based on the attributes of the candidate material;

[0073] The target video is obtained by synthesizing the target template and the candidate material.

[0074] Among them, the target condition may refer to the conditions that the target material used to generate the target video needs to meet, such as at least one of the target video duration, picture quality, target material quantity or content matching. The picture quality score can be evaluated based on clarity, color saturation, etc., and the content matching degree can be based on the semantic similarity with the user instruction. The target material can be further screened based on the target condition to obtain the candidate material that is ultimately used to generate the target video. The candidate material can be marked, for example, by adding a specified tag, etc., to indicate the candidate material to be used for the voyage target video. The attributes of the candidate material may include at least one of the following: the duration of the candidate material, the picture style. Based on the attribute, a suitable splicing template can be matched for the candidate material, for example, the style of the splicing template is consistent with the style of the candidate material, etc.

[0075] In some embodiments, method 300 may further include: outputting the target video;

[0076] In response to receiving an editing instruction for the target video, the target video is edited based on the editing instruction to obtain a new target video.

[0077] When the user wants to edit the target video, he can issue an editing instruction based on the input device, such as using text input or voice input. The vehicle terminal obtains the editing instruction through the interactive system, and edits the target video based on the editing instruction to obtain a new target video.

[0078] In some embodiments, editing the target video based on the editing instruction to obtain a new target video includes:

[0079] Acquire a to-be-edited position of the target video based on the editing instruction;

[0080] Editing the candidate material at the position to be edited to obtain the new target video;

[0081] The step of editing the candidate material at the position to be edited further includes:

[0082] Performing designated processing on the candidate material at the position to be edited;

[0083] Or, randomly replacing the candidate material at the position to be edited with a new candidate material that meets the target condition;

[0084] Or, the candidate material at the to-be-edited position is replaced with a designated candidate material, where the designated candidate material is determined based on the editing instruction.

[0085] Specifically, the in-vehicle terminal can semantically understand the editing instructions to obtain the position to be edited and the editing method in the target video. For example: if the editing instruction is "Change the video at the X second", it can be determined that the position to be edited is the X second in the video, and the editing method is to randomly replace the material; if the editing instruction is "Change the Yth material", it can be determined that the position to be edited is the Yth material (slot) in the video, and the editing method is to randomly replace the material; if the editing instruction is "The image at the M second is processed in a comic style as the cover", it can be determined that the position to be edited is the M second in the video, and the editing method is to process it in a comic style; if the editing instruction is "The image at the N second is processed in a comic style as the cover", it can be determined that the position to be edited is the cover, and the editing method is to replace the cover with the image of the N second processed in a comic style.

[0086] It can be seen that according to the video generation method of the embodiment of the present disclosure, a prompt text is generated by semantic analysis of the user's instructions, and the target material is obtained based on the prompt text, thereby generating a target video. It is possible to automatically obtain the user's desired image or video during driving, help the user free their hands, and automatically generate the target video, thereby improving the effect and efficiency of video generation and reducing the user's workload.

[0087] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0088] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0089] Based on the same technical concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides a video generation device, such as Figure 5 As shown, the video generating device includes:

[0090] The acquisition module is used to obtain user instructions;

[0091] A prompt text module, used to generate a prompt text based on the user instruction, wherein the prompt text includes at least one target object;

[0092] An acquisition module, used for acquiring target material based on the prompt text, wherein the target material includes image material and / or video material;

[0093] A video generation module is used to generate a target video based on the target material.

[0094] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0095] The device of the above embodiment is used to implement the corresponding video generation method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0096] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the video generation method described in any of the above embodiments.

[0097] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0098] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the video generation method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0099] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0100] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the known power / ground connections to the integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it is apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0101] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0102] The embodiments of the present disclosure are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A video generation method, the method comprising: Get user instructions; Generate a prompt text based on the user instruction, wherein the prompt text includes at least one target object; Acquire target material based on the prompt text, where the target material includes image material and / or video material; A target video is generated based on the target material.

2. The method according to claim 1, wherein The prompt text also includes the shooting time; Then, the target material is obtained based on the prompt text, including: Based on the shooting time in the prompt text, the target material including the target object is acquired in real time, and / or based on the shooting time in the prompt text, the target material including the target object is acquired from historical materials.

3. The method according to claim 1, wherein The prompt text also includes the shooting time; Then, the target material is obtained based on the prompt text, including: Acquire the target material having the shooting duration and including the target object in real time; and / or, The target material having the shooting duration and including the target object is acquired from historical materials.

4. The method according to any one of claims 1 to 3, wherein: The prompt text also includes a shooting angle; Then, the target material is obtained based on the prompt text, including: Determine a target image acquisition device based on the shooting angle; The target material including the target object acquired from the target image acquisition device is acquired.

5. The method according to claim 1, wherein: The prompt text also includes target conditions; Then generating a target video based on the target material includes: Filter the target material based on the target condition to obtain candidate materials; Matching a target template based on the attributes of the candidate material; The target video is obtained by synthesizing the target template and the candidate material.

6. The method according to claim 3, further comprising: Outputting the target video; In response to receiving an editing instruction for the target video, the target video is edited based on the editing instruction to obtain a new target video.

7. The method according to claim 6, wherein: Editing the target video based on the editing instruction to obtain a new target video includes: Acquire a to-be-edited position of the target video based on the editing instruction; Editing the candidate material at the position to be edited to obtain the new target video; The step of editing the candidate material at the position to be edited further includes: Performing designated processing on the candidate material at the position to be edited; Or, randomly replacing the candidate material at the position to be edited with a new candidate material that meets the target condition; Or, the candidate material at the to-be-edited position is replaced with a designated candidate material, where the designated candidate material is determined based on the editing instruction.

8. A video generating device, comprising: The acquisition module is used to obtain user instructions; A prompt text module, used to generate a prompt text based on the user instruction, wherein the prompt text includes at least one target object; An acquisition module, used for acquiring target material based on the prompt text, wherein the target material includes image material and / or video material; A video generation module is used to generate a target video based on the target material.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program. 10 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to claim 1 .

Images