Video generation methods and related equipment

By generating prompt text and acquiring target materials through semantic analysis of user commands, this system solves the problem of not being able to automatically capture the user's desired images or videos while the vehicle is in motion. It achieves automatic video generation, reduces user workload, and improves video generation quality and efficiency.

CN119854595BActive Publication Date: 2026-11-17BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202311354576.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-18
Publication Date
2026-11-17
Estimated Expiration
2043-10-18

AI Technical Summary

Technical Problem

While the vehicle is in motion, users cannot automatically capture the desired images or videos and must manually select the shooting time, which increases workload, distracts driving attention, and reduces video generation efficiency.

Method used

The system generates prompt text by semantically analyzing user commands, and then obtains target materials based on the prompt text to generate the target video.

Benefits of technology

It enables the automatic acquisition of images or videos desired by users, reducing user workload and improving video generation quality and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119854595B_ABST
    Figure CN119854595B_ABST
Patent Text Reader

Abstract

This disclosure provides a video generation method and related equipment. The method includes: acquiring a user instruction; generating a prompt text based on semantic information of the user instruction, the prompt text including at least one target object; acquiring target material based on the prompt text, the target material including image material and / or video material; and generating a target video based on the target material.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a video generation method and related equipment. Background Technology

[0002] Currently, when users want to capture images or videos along their route while driving, they can only do so using mobile devices such as smartphones or dashcams. However, these mobile devices are located inside the vehicle or in a fixed position, often resulting in poor image or video quality. Furthermore, the timing of recording requires manual selection, making it impossible to automatically capture the desired images or videos. This not only increases the user's workload but also distracts them, reducing driving safety. Moreover, when users want to composite images or videos along their route, manual editing is required, adding significant work and reducing video generation efficiency. Summary of the Invention

[0003] This disclosure proposes a video generation method, apparatus, device, storage medium, and program product to address, to some extent, the technical problem of not being able to automatically capture images or videos desired by users during driving.

[0004] In a first aspect, this disclosure provides a video generation method, comprising:

[0005] Obtain user instructions;

[0006] A prompt text is generated based on the user instruction, and the prompt text includes at least one target object;

[0007] The target material is obtained based on the prompt text, and the target material includes image material and / or video material;

[0008] Generate a target video based on the target material.

[0009] A second aspect of this disclosure provides a video generation apparatus, comprising:

[0010] The acquisition module is used to acquire user commands;

[0011] The prompt text module is used to generate prompt text based on the user instruction, wherein the prompt text includes at least one target object;

[0012] The acquisition module is used to acquire target materials based on the prompt text, wherein the target materials include image materials and / or video materials;

[0013] The video generation module is used to generate a target video based on the target material.

[0014] A third aspect of this disclosure provides an electronic device, characterized in that it includes one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs including instructions for performing the method according to the first or second aspect.

[0015] A fourth aspect of this disclosure provides a non-volatile computer-readable storage medium comprising a computer program that, when executed by one or more processors, causes the processors to perform the method described in the first or second aspect.

[0016] A fifth aspect of this disclosure provides a computer program product including computer program instructions that, when executed on a computer, cause the computer to perform the method described in the first aspect.

[0017] As can be seen from the above, the video generation method and related equipment provided in this disclosure generate prompt text through semantic analysis of user commands, and obtain target materials based on the prompt text, thereby generating the target video. This enables the automatic acquisition of images or videos desired by the user, freeing up the user's hands, and automatically generating the target video, thereby improving the effect and efficiency of video generation and reducing the user's workload. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the video generation architecture according to an embodiment of the present disclosure.

[0020] Figure 2 This is a schematic diagram of the hardware structure of an exemplary electronic device according to an embodiment of the present disclosure.

[0021] Figure 3 This is a schematic flowchart illustrating a video generation method according to an embodiment of the present disclosure.

[0022] Figure 4 This is a schematic diagram of the interactive interface of an embodiment of this disclosure.

[0023] Figure 5 This is a schematic diagram of a video generation apparatus according to an embodiment of the present disclosure. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0025] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0027] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0030] Figure 1 A schematic diagram of a video generation architecture according to an embodiment of this disclosure is shown. (See reference...) Figure 1The video generation architecture 100 may include a server 110, a terminal 120, and a network 130 providing a communication link. The server 110 and the terminal 120 can be connected via a wired or wireless network 130. The server 110 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, security services, and CDN.

[0031] Terminal 120 can be implemented in hardware or software. For example, when terminal 120 is implemented in hardware, it can be various electronic devices with a display screen and support page display, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers. When terminal 120 is implemented in software, it can be installed in the electronic devices listed above; it can be implemented as multiple software programs or software modules (e.g., software programs or software modules used to provide distributed services) or as a single software program or software module, without specific limitations.

[0032] It should be noted that the video generation method provided in this application embodiment can be executed by the terminal 120 or by the server 110. It should be understood that... Figure 1 The number of terminals, networks, and servers shown is for illustrative purposes only and is not intended to be a limitation. Any number of terminals, networks, and servers can be used depending on implementation needs.

[0033] Figure 2 A schematic diagram of the hardware structure of an exemplary electronic device 200 provided in an embodiment of this disclosure is shown. For example... Figure 2 As shown, the electronic device 200 may include: a processor 202, a memory 204, a network module 206, a peripheral interface 208, and a bus 210. The processor 202, memory 204, network module 206, and peripheral interface 208 are interconnected within the electronic device 200 via the bus 210.

[0034] Processor 202 may be a central processing unit (CPU), a video generator, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or one or more integrated circuits. Processor 202 can be used to perform functions related to the techniques described in this disclosure. In some embodiments, processor 202 may also include multiple processors integrated as a single logic component. For example, such as... Figure 2 As shown, processor 202 may include multiple processors 202a, 202b and 202c.

[0035] Memory 204 can be configured to store data (e.g., instructions, computer code, etc.). Figure 2 As shown, the data stored in memory 204 may include program instructions (e.g., program instructions for implementing the video generation method of embodiments of this disclosure) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). Processor 202 may also access the program instructions and data stored in memory 204 and execute the program instructions to operate on the data to be processed. Memory 204 may include volatile or non-volatile storage devices. In some embodiments, memory 204 may include random access memory (RAM), read-only memory (ROM), optical disk, magnetic disk, hard disk, solid-state drive (SSD), flash memory, memory stick, etc.

[0036] Network module 206 can be configured to provide communication with other external devices to electronic device 200 via a network. This network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, Near Field Communication (NFC), etc.), a cellular network, the Internet, or a combination thereof. It is understood that the type of network is not limited to the specific examples described above. In some embodiments, network module 306 may include any combination of any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, etc.

[0037] The peripheral interface 208 can be configured to connect the electronic device 200 to one or more peripheral devices to enable information input and output. For example, peripheral devices may include input devices such as keyboards, mice, touchpads, touch screens, microphones, and various sensors, as well as output devices such as displays, speakers, vibrators, and indicator lights.

[0038] Bus 210 can be configured to transfer information between various components of electronic device 200 (e.g., processor 202, memory 204, network module 206, and peripheral interface 208), such as internal buses (e.g., processor-memory bus), external buses (USB port, PCI-E bus), etc.

[0039] It should be noted that although the architecture of the above-described electronic device 200 only shows the processor 202, memory 204, network module 206, peripheral interface 208, and bus 210, in specific implementations, the architecture of the electronic device 200 may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the architecture of the above-described electronic device 200 may only include the components necessary for implementing the embodiments of this disclosure, and does not necessarily include all the components shown in the figures.

[0040] While driving, users can only capture images or videos of their journey using mobile devices such as smartphones or dashcams. Users can record images or videos using their smartphones or by voice control of the dashcam when they see a desired scene. However, these mobile devices are either located inside the vehicle or in a fixed position, often resulting in poor image or video quality. Manual selection of recording times is required; for example, a user might use their smartphone to record a scene they want to capture, making automatic capture of desired images or videos impossible. This not only increases the user's workload but also distracts them, reducing driving safety. Furthermore, when users want to composite images or videos along the way, manual editing is necessary, adding significant workload and reducing video generation efficiency. This results in the inability to automatically capture desired images or videos while driving, coupled with the high workload, poor quality, and low efficiency of manual shooting and editing. Therefore, how to automatically capture desired images or videos and improve the quality and efficiency of video compositing have become urgent technical problems to be solved.

[0041] In view of this, embodiments of this disclosure provide a video generation method and related apparatus. By semantic analysis of user commands, prompt text is generated, and target materials are obtained based on the prompt text, thereby generating the target video. This enables the automatic acquisition of images or videos desired by the user, freeing up the user's hands, and automatically generating the target video, thereby improving the quality and efficiency of video generation and reducing the user's workload.

[0042] See Figure 3 , Figure 3 A schematic flowchart of a video generation method according to an embodiment of the present disclosure is shown. Figure 3 In this context, the video generation method 300 can be deployed on an in-vehicle terminal and may further include the following steps.

[0043] In step S310, user instructions are obtained.

[0044] User commands refer to instructions sent by the user through an input device, which can be text input, voice input, etc. When a user inputs a command via voice, a corresponding voice command is generated, and this voice command can have semantic meaning. Specifically, users can input user commands through an interactive interface. See also Figure 4 , Figure 4 A schematic diagram of an interactive interface according to an embodiment of the present disclosure is shown. Figure 4 In this system, the interactive interface 400 enables interaction between the in-vehicle terminal (e.g., the vehicle's central control unit) and the user. Specifically, the in-vehicle terminal may include an interactive system for functions such as activation, interaction, voice and tone recognition, speech-to-text conversion, and displaying and saving chat history. For example, the user can input a voice activation command through an input device (e.g., a microphone) in the in-vehicle terminal, and the activation command 411 will be displayed on the interactive interface 400. The activation command 411 may include an activation indicator to activate the interactive system. The interactive system in the in-vehicle terminal can respond to the activation command 411 by displaying and outputting an activation response 421 on the interactive interface 400. At this point, the interactive system begins to work, and the user can continue to input a user command 412, which will be displayed on the interactive interface 400. This user command 412 can be used to instruct the in-vehicle terminal to generate video. For example, the user command 412 could be "Record my journey and edit it into a nice video." Accordingly, the interactive system can respond to user instruction 412 by displaying instruction response 422 in the interactive interface 400 and outputting instruction response 422.

[0045] In step S320, a prompt text is generated based on the user instruction, the prompt text including at least one target object.

[0046] The prompt text refers to text that is semantically related to the user's instruction and includes a specific target object. The target object can refer to various objects, such as people, animals, buildings, and prominent objects. Since user instructions are unlikely to exhaustively list all things the user wants to photograph, they are often quite abstract, such as "record the journey," which doesn't necessarily mean the user expects to record every journey. Therefore, a text generation model, such as the AIGC (Artificial Intelligence Generated Content) language model, can be used to understand the semantics of the user's instructions and generate accurate, complete, and relatively specific prompt text. For example, this text generation model can semantically understand user instruction 412, "Record my journey and edit it into a nice video," assuming that the beautiful scenery outside the car and the joyful moments inside the car are what the user wants to record. It can generate the prompt text, "There will be pictures and videos worth recording on the journey, such as snow-capped mountains, sunsets, and rainbows outside the car, and joyful moments inside the car; these will all be captured for you." This prompt text includes specific target objects, such as snow-capped mountains, sunsets, rainbows, and joyful scenes inside the car. This prompt text can be displayed in the interactive interface or played via voice. It should be understood that the above prompt text is for illustrative purposes only and is not intended to limit its scope. Prompt text may include more or fewer target objects, which is not limited here.

[0047] In some embodiments, generating prompt text based on the user instruction may include:

[0048] The user command is input into the text generation model to obtain the prompt text; the prompt text also includes at least one of the following: shooting time, shooting duration, shooting angle, and target conditions.

[0049] The prompt text can be one or more, and each prompt text can include one or more target objects. Further, each prompt text can include at least one of the following: shooting time, shooting duration, shooting angle, and target conditions. The target objects or target conditions of each prompt text can be partially the same, all the same, or all different, and the shooting time, shooting duration, and shooting angle of each prompt text can also be the same or different. Specifically, the prompt text can include a first prompt text, which can include a first target object and at least one of a first shooting time, a first shooting duration, a first shooting angle, and a first target condition; a second prompt text can include a second target object and at least one of a second shooting time, a second shooting duration, a second shooting angle, and a second target condition. For example, a user can issue the user command "Record today's street corner" when leaving home at 8:00 AM, generating n prompt texts text_1, ..., text_i, ..., text_n, where i and n are positive integers. The prompt text_1 can include: 10 seconds (shooting duration) of video footage captured between 8:00-10:00 AM (shooting time) by the front camera and the right-side camera (shooting angle), including a person (target object 1) at a road corner (target object 1). The prompt text_i can include: 20 seconds (shooting duration) of video footage captured between 12:00-13:00 PM (shooting time) by the front camera (shooting angle), including a vehicle (target object 2) or vegetation (target object 3) at an intersection (target object 1).

[0050] In some embodiments, method 300 may further include:

[0051] Output the aforementioned prompt text;

[0052] In response to receiving a regeneration instruction for the prompt text, a new prompt text is regenerated based on the user instruction.

[0053] If a user is dissatisfied with the generated prompt text, they can issue a regeneration command to regenerate the prompt text. When the in-vehicle terminal receives a regeneration command from the user for one or more prompt texts via the interactive system, it can regenerate new prompt text based on the user's command. Furthermore, the user can also update one or more prompt texts based on corresponding commands, such as adding, deleting, or replacing some text in the prompt text (e.g., target object, shooting time, shooting duration, shooting angle, target conditions, etc.) to obtain new prompt text. Even further, the in-vehicle terminal can regenerate new prompt text multiple times until the user is satisfied, for example, when the user issues a command to start video generation via the interactive system.

[0054] In step S330, target materials are obtained based on the prompt text, and the target materials include image materials and / or video materials.

[0055] The prompt text includes one or more target objects, and the in-vehicle terminal can obtain target materials based on these target objects. For example, if the prompt text includes images of snow-capped mountains, sunsets, rainbows, or joyful scenes inside the car, the in-vehicle terminal will record images of snow-capped mountains, sunsets, rainbows, etc. outside the car, and scenes of laughter inside the car as target materials.

[0056] In some embodiments, obtaining target material based on the prompt text includes:

[0057] Based on the shooting time in the prompt text, the target material including the target object is acquired in real time, and / or, based on the shooting time in the prompt text, the target material including the target object is acquired from historical materials.

[0058] The shooting time can refer to the time when the footage was captured. User instructions may include time-related information, and the text generation model generates the shooting time based on this information. For example, the text generation model can generate a prompt text with the shooting time set to 19:00-05:00 based on the user instruction "Record the starry sky". Alternatively, user instructions may not include information related to the shooting time; in this case, the text generation model can generate a specified time as the shooting time. For example, the text generation model can generate a prompt text with the shooting time set to 00:00-24:00 (i.e., 24 hours a day) or 07:00-24:00 based on the user instruction "Record the everyday life in the city". The target footage can be collected in real-time based on the shooting time in the prompt text. For example, if the prompt text states the shooting time is 19:00-05:00, then image or video footage including the starry sky can be collected between 19:00 and 05:00. It can also be obtained from historical footage. For example, the user instruction "Edit a beautiful travel video" can obtain target footage containing the target object from historical travel images and videos.

[0059] In some embodiments, obtaining target material based on the prompt text includes:

[0060] The target material, which has the shooting duration and includes the target object, is acquired in real time.

[0061] And / or,

[0062] The target footage, which includes the stated shooting duration and the stated target object, is obtained from historical footage.

[0063] The shooting duration refers to the length of the captured footage. The text generation model can generate a corresponding shooting duration based on the target object. Specifically, when the target object is an object that typically changes little over a certain period of time, such as the sun, moon, or starry sky, its shooting duration can be set to a shorter duration, such as t1 (t1 is a positive integer) seconds. When the target object is an object that changes rapidly over a certain period of time, such as streets or roadside scenery, its shooting duration can be set to t2 (t2 is a positive integer) minutes or 60*t2 seconds. It should be understood that the shooting durations t1 seconds or t2 minutes mentioned above are just examples and are not intended to limit the shooting duration. The shooting duration can also be longer or shorter, without limitation here. Video footage including the target object can be shot in real time based on this shooting duration, or video footage with this shooting duration and including the target object can be selected from historical footage. Furthermore, the prompt text may include the shooting time and shooting duration, so that video footage including the target object can be shot in real time based on the shooting duration, or video footage including the target object with the shooting duration can be obtained from historical footage at that shooting time.

[0064] In some embodiments, obtaining target material based on the prompt text includes:

[0065] The target image acquisition device is determined based on the shooting angle;

[0066] Acquire the target material, including the target object, acquired by the target image acquisition device.

[0067] The shooting angle refers to the angle of the captured material. The text generation model can determine the presentation angle of the target object to obtain the desired material. Specifically, one or more image acquisition devices, such as cameras, can be deployed inside and outside the vehicle. Each image acquisition device can be equipped with a target recognition algorithm to identify the target object. The vehicle terminal sends a prompt text to each image acquisition device. The image acquisition device recognizes its own real-time image. When it recognizes the target object in the prompt text within the real-time image, that image acquisition device, acting as the target image acquisition device, triggers itself to capture the current real-time image, obtaining the target material. This target material is then stored in the storage area corresponding to the target image acquisition device. The positions of the image acquisition devices are pre-set, so the shooting angle can be determined based on the position of the image acquisition device. For example, a vehicle can have image acquisition devices set up in the front, rear, left, and right directions. The text generation model can determine that the target object is an intersection. To capture a panoramic view of the intersection, it is necessary to use image acquisition devices located in front of or behind the vehicle. Therefore, the shooting angle can be determined accordingly, shooting from the front or rear of the vehicle. Furthermore, the image acquisition device located in front of or behind the vehicle can be identified as the target image acquisition device. When the target image acquisition device recognizes the intersection, it begins to acquire video footage and sends the acquired video footage, including the intersection, to the vehicle terminal.

[0068] In step S340, a target video is generated based on the target material.

[0069] The generated target video can be displayed via an interactive interface, such as... Figure 4 The target video 423 is shown in the image. Users can click on the target video 423 to play it.

[0070] Furthermore, in some embodiments, generating a target video based on the target material includes:

[0071] Based on the target conditions, the target materials are filtered to obtain candidate materials;

[0072] Match the target template based on the attributes of the candidate materials;

[0073] The target video is synthesized based on the target template and the candidate materials.

[0074] The target conditions refer to the conditions that the target materials used to generate the target video must meet, such as at least one of the following: target video duration, picture quality, number of target materials, or content matching degree. Picture quality scores can be evaluated based on clarity, color saturation, etc., and content matching degree can be based on semantic similarity to user commands. Target materials can be further filtered based on the target conditions to obtain candidate materials for generating the final target video. Candidate materials can be tagged, for example, by adding specified tags, to indicate that they will be used for the flight target video. The attributes of the candidate materials can include at least one of the following: the duration of the candidate materials and the picture style. A suitable splicing template can be matched to the candidate materials based on these attributes, for example, the style of the splicing template can be consistent with the style of the candidate materials.

[0075] In some embodiments, method 300 may further include: outputting the target video;

[0076] In response to receiving an editing instruction for the target video, the target video is edited based on the editing instruction to obtain a new target video.

[0077] When a user wishes to edit a target video, they can issue an editing command via an input device, such as text or voice input. The in-vehicle terminal receives the editing command through the interactive system and edits the target video accordingly to obtain a new target video.

[0078] In some embodiments, editing the target video based on the editing instructions to obtain a new target video includes:

[0079] The editing position of the target video is obtained based on the editing instructions;

[0080] The candidate materials at the location to be edited are edited to obtain the new target video;

[0081] Editing the candidate material at the location to be edited further includes:

[0082] The candidate materials at the location to be edited are specified and processed accordingly;

[0083] Alternatively, the candidate material at the location to be edited may be randomly replaced with a new candidate material that meets the target conditions;

[0084] Alternatively, the candidate material at the position to be edited can be replaced with a specified candidate material, which is determined based on the editing instruction.

[0085] Specifically, the vehicle-mounted terminal can semantically understand editing commands to obtain the editable position and editing method in the target video. For example, the editing command "Change the video at second X" determines that the editable position is second X in the video, and the editing method is to randomly replace the material; the editing command "Change the material at second Y" determines that the editable position is the material at second Y (slot) in the video, and the editing method is to randomly replace the material; the editing command "Process the image at second M with a comic style as the cover" determines that the editable position is second M in the video, and the editing method is to process it with a comic style; the editing command "Process the image at second N with a comic style as the cover" determines that the editable position is the cover, and the editing method is to replace the cover with the image at second N after processing it with a comic style.

[0086] As can be seen, the video generation method according to the embodiments of this disclosure generates prompt text through semantic analysis of user commands, acquires target materials based on the prompt text, and then generates a target video. This enables the automatic acquisition of images or videos desired by the user during driving, freeing the user's hands, and automatically generating target videos, thereby improving the effect and efficiency of video generation and reducing the user's workload.

[0087] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0088] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0089] Based on the same technical concept, corresponding to any of the methods in the above embodiments, this disclosure also provides a video generation apparatus, such as... Figure 5 As shown, the video generation device includes:

[0090] The acquisition module is used to acquire user commands;

[0091] The prompt text module is used to generate prompt text based on the user instruction, wherein the prompt text includes at least one target object;

[0092] The acquisition module is used to acquire target materials based on the prompt text, wherein the target materials include image materials and / or video materials;

[0093] The video generation module is used to generate a target video based on the target material.

[0094] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0095] The apparatus of the above embodiments is used to implement the corresponding video generation method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0096] Based on the same technical concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the video generation method as described in any of the above embodiments.

[0097] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0098] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the video generation method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0099] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0100] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuitry) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0101] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0102] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A video generation method, the method comprising: Obtain user instructions; Multiple different prompt texts are generated based on the semantic analysis of the user's instructions, and the prompt texts include at least one target object; Output the aforementioned prompt text; In response to receiving a regeneration instruction for the prompt text, the prompt text is regenerated based on the user instruction; The target material is obtained based on the prompt text, and the target material includes image material and / or video material; Generate a target video based on the target material.

2. The method according to claim 1, wherein, The notification text also includes the shooting time; The target material is then obtained based on the prompt text, including: Based on the shooting time in the prompt text, the target material including the target object is acquired in real time, and / or, based on the shooting time in the prompt text, the target material including the target object is acquired from historical materials.

3. The method according to claim 1, wherein, The notification text also includes the shooting duration; The target material is then obtained based on the prompt text, including: The target material, which has the shooting duration and includes the target object, is acquired in real time. And / or, The target footage, which includes the stated shooting duration and the stated target object, is obtained from historical footage.

4. The method according to any one of claims 1-3, wherein, The prompt text also includes the shooting angle; The target material is then obtained based on the prompt text, including: The target image acquisition device is determined based on the shooting angle; Acquire the target material, including the target object, acquired by the target image acquisition device.

5. The method according to claim 1, wherein, The prompt text also includes the target conditions; Then, based on the target material, a target video is generated, including: Based on the target conditions, the target materials are filtered to obtain candidate materials; Match the target template based on the attributes of the candidate materials; The target video is synthesized based on the target template and the candidate materials.

6. The method of claim 5, further comprising: Output the target video; In response to receiving an editing instruction for the target video, the target video is edited based on the editing instruction to obtain a new target video.

7. The method according to claim 6, wherein, Edit the target video based on the editing instructions to obtain a new target video, including: The editing position of the target video is obtained based on the editing instructions; The candidate materials at the location to be edited are edited to obtain the new target video; Editing the candidate material at the location to be edited further includes: The candidate materials at the location to be edited are specified and processed accordingly; Alternatively, the candidate material at the location to be edited may be randomly replaced with a new candidate material that meets the target conditions; Alternatively, the candidate material at the position to be edited can be replaced with a specified candidate material, which is determined based on the editing instruction.

8. A video generation apparatus, comprising: The acquisition module is used to acquire user commands; The prompt text module is used to generate multiple different prompt texts based on semantic analysis of the user command, wherein each prompt text includes at least one target object; and outputs the prompt texts. And in response to receiving a regeneration instruction for the prompt text, regenerate the prompt text based on the user instruction; The acquisition module is used to acquire target materials based on the prompt text, wherein the target materials include image materials and / or video materials; The video generation module is used to generate a target video based on the target material.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 7.

Citation Information

Patent Citations

  • Video generation method and video generation system

    CN112312189A

  • Delayed photography file generation method and device and electronic equipment

    CN113596339A

  • Shooting method and equipment for automatically calling vehicle-mounted image acquisition device

    CN115134533A