Video generation method and apparatus, and electronic device

Abstract

The present disclosure provides a video generation method and apparatus, and an electronic device. A specific embodiment of the method comprises: acquiring a first image and first guidance information; processing the first image on the basis of the first guidance information, to obtain a second image; using the second image as a first image frame to be processed for camera movement processing and stylization processing to obtain a plurality of third image frames; and generating a target video on the basis of the second image and the plurality of third image frames.

Description

Method, device and electronic device for generating video

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on and claims priority to an application filed in China with application number 202310952705.2 and filing date July 31, 2023. The disclosed content of the Chinese application is hereby introduced as a whole into this application. Technical Field

[0003] The present disclosure relates to the field of video processing technology, and in particular to a method, device, and electronic device for generating a video. Background Art

[0004] With the continuous development of network technology and streaming media technology, videos are increasingly used in people's lives, providing entertainment services for people's lives and bringing many conveniences.

[0005] Summary of the Invention

[0006] The present disclosure provides a method, apparatus, and electronic device for generating a video.

[0007] According to a first aspect, a method for generating a video is provided, the method comprising:

[0008] acquiring a first image and first guidance information;

[0009] processing the first image based on the first guidance information to obtain a second image;

[0010] Performing camera movement processing and stylization processing on the second image as the first frame image to be processed, to obtain multiple frames of third images;

[0011] A target video is generated based on the second image and the multiple frames of third images.

[0012] According to a second aspect, there is provided an apparatus for generating a video, the apparatus comprising:

[0013] An acquisition module, configured to acquire a first image and first guidance information;

[0014] a first processing module, configured to process the first image based on the first guidance information to obtain a second image;

[0015] a second processing module, configured to perform camera movement processing and stylization processing on the second image as a first frame image to be processed, to obtain multiple frames of third images;

[0016] A generation module is used to generate a target video based on the second image and the multiple frames of third images.

[0017] According to a third aspect, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of the above-mentioned first aspects is implemented.

[0018] According to a fourth aspect, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods described in the first aspect when executing the program.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] FIG1 is a schematic diagram of an exemplary system architecture for applying an embodiment of the present disclosure;

[0022] FIG2 is a schematic diagram of a link scenario for generating a video according to an exemplary embodiment of the present disclosure;

[0023] FIG3 is a flowchart of a method for generating a video according to an exemplary embodiment of the present disclosure;

[0024] FIG4 is a flowchart of another method for generating a video according to an exemplary embodiment of the present disclosure;

[0025] FIG5 is a block diagram of a device for generating a video according to an exemplary embodiment of the present disclosure;

[0026] FIG6 is a schematic block diagram of an electronic device provided by some embodiments of the present disclosure;

[0027] FIG7 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure;

[0028] FIG8 is a schematic diagram of a storage medium provided by some embodiments of the present disclosure. DETAILED DESCRIPTION

[0029] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments derived by those skilled in the art based on the embodiments in this specification without creative effort shall fall within the scope of protection of this specification.

[0030] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0031] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a," "the," and "the" used in this disclosure are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining."

[0033] With the continuous development of network technology and streaming media technology, videos are increasingly used in people's lives, providing entertainment services for people's lives and bringing many conveniences. At present, in order to increase the fun, a video can be made by using camera movement techniques. In related technologies, it is usually necessary to manually generate the image frames of the video frame by frame. Therefore, there is a need for an efficient method for generating camera movement videos. The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects:

[0034] Embodiments of the present disclosure provide a method and apparatus for generating a video. These methods obtain first guidance information, process a first image based on the first guidance information, obtain a second image, perform camera movement and stylization processing on the second image as the first frame to be processed, obtain multiple third images, and generate a target video based on the second image and the multiple third images. Because the first frame to be processed is generated based on the first image under the guidance of the first guidance information, the generated target video is prevented from deviating from the user's intent, thereby providing guidance for generating the target video toward the desired effect and improving the user experience.

[0035] See FIG1 , which is a schematic diagram of an exemplary system architecture applying an embodiment of the present disclosure.

[0036] As shown in FIG1 , system architecture 100 may include terminal devices 102, network 103, and server 104. It should be understood that the number or types of terminal devices, networks, and servers in FIG1 are merely illustrative. Any number or type of terminal devices, networks, and servers may be used as needed.

[0037] The network 103 is used to provide a medium for communication links between terminal devices and servers. The network 103 can include various connection types, such as wired or wireless communication links or optical fiber cables.

[0038] The terminal device 102 can interact with the server through the network 103 to receive or send requests or information, etc. The terminal device 102 can be various electronic devices, including but not limited to smart phones, tablet computers, smart wearable devices, and personal digital assistants, etc.

[0039] The server 104 can store, analyze, and process the received data, and can also send control commands or requests to terminal devices or other servers. The server can provide services in response to user service requests. It is understood that a server can provide one or more services, and the same service can be provided by multiple servers.

[0040] Based on the system architecture shown in Figure 1, in the embodiment of the present disclosure, user 101 can input a first image and first guidance information through terminal device 102, and terminal device 102 sends the first image and first guidance information to server 104 via network 103. After receiving the first image and the first guidance information, server 104 can process the first image based on the first guidance information to obtain a second image, and use the second image as the first frame image to perform camera movement processing and stylization processing to obtain multiple frames of third images. Then, based on the second image and the multiple frames of third images, generate a target video. Finally, server 104 can return the target video to terminal device 102 via network 103, so that user 101 can view the effect of the target video through terminal device 102.

[0041] Referring to FIG. 2 , it is a schematic diagram of a link scenario for generating a video according to an exemplary embodiment.

[0042] As shown in Figure 2, a user can first input image A and guidance information through a third-party application installed on a terminal device. The terminal device can then send a request Q to the video cloud editing platform, containing image A and guidance information, to upload them to the cloud editing platform. In response to request Q, the cloud editing platform transmits image A and guidance information to algorithm server 1, which processes image A based on the guidance information to produce image B. Algorithm server 1 returns image B to the cloud editing platform, which then transmits image B to algorithm server 2. After receiving image B, algorithm server 2 can use image B as the first frame and, through camera movement and stylization, generate multiple video frames frame by frame. Based on these multiple frames, it generates video C and returns video C to the cloud editing platform. The cloud editing platform can then further process video C based on a pre-set video template (for example, by adding special effects) to produce video D. Finally, the cloud editing platform can return video D to the terminal device, and the terminal device plays video D for the user to watch.

[0043] The present disclosure will be described in detail below with reference to specific embodiments.

[0044] FIG3 is a flow chart of a method for generating a video according to an exemplary embodiment. The execution subject of the method can be implemented as any device, platform, server or device cluster with computing and processing capabilities. The method includes the following steps:

[0045] As shown in FIG3 , in step 301 , a first image and first guidance information are acquired.

[0046] In this embodiment, the first image is used to generate the target video. The first image can be a frame of image directly input by the user, a frame of image directly taken by the user, or a frame of image selected by the user from an existing video. It can be understood that this embodiment does not limit the specific method of obtaining the first image.

[0047] In this embodiment, the first guidance information may be text information, image information, or a combination of text information and image information. The first guidance information is used to provide constraints for guiding the generation of the second image. For example, if the first guidance information includes text information, the text information may include keywords corresponding to one or more of the following constraints: theme constraints, environment constraints, atmosphere constraints, color constraints, style constraints, distant view constraints, and close view constraints. If the first guidance information includes image information, the image information may include, but is not limited to, image feature information that can reflect the following constraints: theme, environment, atmosphere, color, style, distant view, and close view.

[0048] In step 302, the first image is processed based on the first guidance information to obtain a second image.

[0049] In this embodiment, the first image can be processed based on the first guidance information. For example, the first guidance information and the first image can be input into a pre-trained model. The model then stylizes the first image based on the constraints provided by the first guidance information, thereby generating a second image. The second image can have a similar layout to the first image and satisfy the constraints provided by the first guidance information. It should be noted that the pre-trained model can be a diffusion model, and this embodiment does not limit the specific type of the model.

[0050] In step 303, the second image is used as the first frame image to be processed and subjected to camera movement processing and stylization processing to obtain multiple frames of third images.

[0051] In this embodiment, the second image can be used as the first frame to be processed, and camera movement processing and stylization processing can be performed to obtain multiple frames of third images. For example, if image 1 is the second image, image 1 can be processed first to generate image 2, then image 2 can be processed to generate image 3, then image 3 can be processed to generate image 4, then image 4 can be processed to generate image 5, and so on. Finally, some or all of the generated images can be used as the third image.

[0052] Specifically, in one implementation, camera movement parameters input by a user can be obtained. Based on the camera movement parameters, multiple frames of a third image are generated frame by frame based on the second image through camera movement processing and stylization. The camera movement parameters may include, but are not limited to, image translation coefficients, rotation coefficients, scaling coefficients, lens zoom coefficients, etc. The user can control the camera movement effect of the generated video using the input camera movement parameters. It should be noted that during the stylization process, different stylization processes can be randomly performed according to a variety of pre-trained stylization models.

[0053] In another implementation, in addition to inputting camera movement parameters, the user can also input second guidance information. Based on the camera movement parameters and the second guidance information input by the user, multiple third images can be generated frame by frame from the second image through camera movement processing and stylization. The second guidance information includes various constraints used to guide the generation of the third images. Because this embodiment references the second guidance information input by the user when generating the multiple third images, it is possible to generate multiple intermediate third images according to the user's wishes, thereby improving the user experience.

[0054] In step 304 , a target video is generated based on the second image and multiple frames of third images.

[0055] In this embodiment, after generating multiple frames of the third image, a target video can be generated based on the second image and the multiple frames of the third image. In one implementation, if the first image is a single frame uploaded, the first image can be used as the first frame, and the second image and the multiple frames of the third image can be combined to form the target video.

[0056] In another implementation, if the first image is a frame of image selected from an existing video, the existing video, the second image, and multiple frames of third images may be synthesized into a target video.

[0057] In another implementation, after the second image and multiple frames of the third image are synthesized into a video, special effects can be added to the synthesized video to obtain the target video.

[0058] The present disclosure provides a method for generating a video. The method obtains first guidance information, processes a first image based on the first guidance information to obtain a second image, uses the second image as the first frame image to be processed, performs camera movement processing and stylization processing to obtain multiple third images, and generates a target video based on the second image and the multiple third images. Because the first frame image to be processed is generated based on the first image under the guidance of the first guidance information, the generated target video is prevented from deviating from the user's intention, thereby providing guidance for generating the target video towards the desired effect and improving the user experience.

[0059] FIG4 is a flow chart showing another method for generating a video according to an exemplary embodiment. This embodiment describes a process of generating multiple frames of third images, including the following steps:

[0060] As shown in FIG4 , in step 401 , the camera movement parameters and the second guide information input by the user are obtained, and in step 402 , multiple frames of the third image are generated frame by frame based on the second image through camera movement processing and stylization processing according to the camera movement parameters and the second guide information.

[0061] In this embodiment, the user can pre-enter camera movement parameters and second guidance information. The second guidance information can include various constraints used to guide the generation of the third image. These constraints may include, but are not limited to, theme, environment, atmosphere, color tone, style, long shot, close shot, etc. The second guidance information can be text, image, or a combination of text and image.

[0062] In this embodiment, the target operation can be performed multiple times iteratively to generate multiple third images based on the second image. Each target operation can include: first, obtaining a current image to be processed. When the target operation is performed for the first time, the second image can be used as the current image to be processed. When the target operation is not performed for the first time, the third image generated by the previous target operation can be used as the current image to be processed.

[0063] Then, the current image to be processed is subjected to a mirror movement process to obtain a mirror movement image. If the preset condition is not currently satisfied, the mirror movement image can be determined as a third image. If the preset condition is currently satisfied, the mirror movement image can be stylized based on the second guidance information to obtain a third image. The preset condition can be a condition set for the number of executions of the current target operation. For example, the 3nth (n is greater than or equal to 1) execution of the target operation can be set as a preset condition in advance, and then the preset condition is satisfied when the target operation is executed for the 3rd, 6th, 9th... times. It can be understood that the preset condition can be any reasonable condition, and this embodiment does not limit the specific setting of the preset condition.

[0064] Taking image 1 as an example, image 1 is the second image generated based on the first image. First, the camera movement parameters and the second guidance information input by the user are obtained. Then, in the first target operation, image 1 is subjected to camera movement processing based on the camera movement parameters to obtain camera movement image 2 as the third image. In the second target operation, image 2 is subjected to camera movement processing based on the camera movement parameters to obtain camera movement image 3 as the third image. In the third target operation, image 3 is subjected to camera movement processing based on the camera movement parameters to obtain camera movement image 4. Then, image 4 is subjected to stylized camera movement processing based on the second guidance information to obtain image 5 as the third image. In the fourth target operation, image 5 is subjected to camera movement processing based on the camera movement parameters to obtain camera movement image 6 as the third image... And so on. The subsequent processes are not repeated here.

[0065] In this embodiment, when the preset conditions are currently met, the camera movement image can be stylized based on the second guidance information to obtain a third image in the following manner: partial constraints can be obtained from the second guidance information according to preset rules, and the camera movement image can be stylized based on the partial constraints to obtain the third image.

[0066] Specifically, because the second guidance information includes multiple constraints, only a portion of the constraints may be obtained during each stylization process. Preset rules for obtaining a portion of the constraints from the second guidance information can be set in advance for each stylization process. For example, the preset rules may indicate that a portion of the constraints are to be obtained from the second guidance information based on the constraint category and the number of stylization processes. Alternatively, a preset number of constraints may be selected each time based on the order of the constraints in the second guidance information. Alternatively, a preset number of constraints may be randomly obtained from the second guidance information, etc. It is understood that the preset rules may be any reasonable rules, and this embodiment does not limit the specific content of the preset rules.

[0067] Since this embodiment generates multiple frames of third images frame by frame through camera movement processing and stylization processing based on the camera movement parameters and second guidance information input by the user, the multiple frames of third images used to generate the target video can further avoid the generated target video from deviating from the user's intention, provide guidance direction for generating the target video towards the expected effect, and further enhance the user experience.

[0068] It should be noted that although the operations of the methods of the embodiments of the present disclosure are described in a specific order in the above embodiments, this does not require or imply that these operations must be performed in this specific order, or that all of the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flowcharts can be performed in a different order. Additionally or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0069] Corresponding to the aforementioned method embodiment for generating a video, the present disclosure also provides an embodiment of an apparatus for generating a video.

[0070] As shown in FIG5 , FIG5 is a block diagram of a device for generating a video according to an exemplary embodiment of the present disclosure. The device may include: an acquisition module 501 , a first processing module 502 , a second processing module 503 and a generation module 504 .

[0071] The acquisition module 501 is used to acquire a first image and first guidance information.

[0072] The first processing module 502 is configured to process the first image based on the first guidance information to obtain a second image.

[0073] The second processing module 503 is configured to perform camera movement processing and stylization processing on the second image as the first frame image to be processed, to obtain multiple frames of third images.

[0074] The generating module 504 is configured to generate a target video based on the second image and multiple frames of third images.

[0075] In some embodiments, the first guidance information includes text information and / or image information.

[0076] In other embodiments, the first guidance information is used to provide one or more of the following constraints to guide the generation of the second image: theme constraints, environment constraints, atmosphere constraints, color constraints, style constraints, distant view constraints, and close view constraints.

[0077] In some other implementations, the second processing module 503 may include: an acquisition submodule and a generation submodule (not shown in the figure).

[0078] The acquisition submodule is used to obtain the camera movement parameters and the second guidance information input by the user.

[0079] The generation submodule is used to generate multiple frames of third images frame by frame based on the second image through camera movement processing and stylization processing according to the camera movement parameters and the second guide information.

[0080] In other embodiments, the generating submodule generates multiple third images frame by frame based on the second image through camera movement processing and stylization processing according to the camera movement parameters and the second guidance information, which may include performing the following target operations multiple times:

[0081] Obtain the current image to be processed. If the target operation is being performed for the first time, the current image to be processed is the second image. If the target operation is not being performed for the first time, the current image to be processed is the third image generated by the previous target operation. Perform camera movement processing on the current image to be processed to obtain a camera movement image. If the preset conditions are not currently met, determine the camera movement image as the third image. If the preset conditions are currently met, perform stylized processing on the camera movement image based on the second guidance information to obtain the third image.

[0082] In other embodiments, the second guidance information may include a plurality of constraint conditions for guiding the generation of the third image.

[0083] In other embodiments, when the preset conditions are currently met, the generation submodule can stylize the camera movement image based on the second guidance information to obtain a third image in the following manner: obtain partial constraints from the second guidance information according to preset rules, and stylize the camera movement image based on the partial constraints to obtain the third image.

[0084] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiment scheme of the present disclosure. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0085] Figure 6 is a schematic block diagram of an electronic device provided by some embodiments of the present disclosure. As shown in Figure 6, the electronic device 910 includes a processor 911 and a memory 912, which can be used to implement a client or server. The memory 912 is used to store computer-executable instructions (e.g., one or more computer program modules) non-transiently. The processor 911 is used to run the computer-executable instructions, which can execute one or more steps in the method for generating a video described above when the computer-executable instructions are run by the processor 911, thereby implementing the method for generating a video described above. The memory 912 and the processor 911 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0086] For example, the processor 911 may be a central processing unit (CPU), a graphics processing unit (GPU), or other processing units with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) may be an X86 or ARM architecture. The processor 911 may be a general-purpose processor or a dedicated processor, and may control other components in the electronic device 910 to perform desired functions.

[0087] For example, the memory 912 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 911 may execute one or more computer program modules to implement various functions of the electronic device 910. Various applications and various data, as well as various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0088] It should be noted that, in the embodiment of the present disclosure, the specific functions and technical effects of the electronic device 910 can be referred to the description of the method for generating a video above, and will not be repeated here.

[0089] Figure 7 is a schematic block diagram of another electronic device provided by some embodiments of the present disclosure. The electronic device 920 is, for example, suitable for implementing the method for generating a video provided by an embodiment of the present disclosure. The electronic device 920 may be a terminal device, etc., which can be used to implement a client or a server. The electronic device 920 may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. It should be noted that the electronic device 920 shown in Figure 7 is only an example and does not impose any restrictions on the functions and scope of use of the embodiments of the present disclosure.

[0090] As shown in FIG7 , the electronic device 920 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 921, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 922 or a program loaded from a storage device 928 into a random access memory (RAM) 923. Various programs and data required for the operation of the electronic device 920 are also stored in the RAM 923. The processing device 921, the ROM 922, and the RAM 923 are connected to each other via a bus 924. An input / output (I / O) interface 925 is also connected to the bus 924.

[0091] Typically, the following devices may be connected to the I / O interface 925: an input device 926 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 927 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 928 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 929. The communication device 929 may allow the electronic device 920 to communicate with other electronic devices wirelessly or by wire to exchange data. Although FIG. 7 shows the electronic device 920 having various devices, it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 920 may alternatively implement or have more or fewer devices.

[0092] For example, according to an embodiment of the present disclosure, the above-mentioned method for generating a video can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the above-mentioned method for generating a video. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 929, or installed from the storage device 928, or installed from the ROM 922. When the computer program is executed by the processing device 921, the functions defined in the method for generating a video provided by the embodiment of the present disclosure can be implemented.

[0093] Figure 8 is a schematic diagram of a storage medium provided in some embodiments of the present disclosure. For example, as shown in Figure 8, storage medium 930 may be a non-transitory computer-readable storage medium for storing non-transitory computer-executable instructions 931. When non-transitory computer-executable instructions 931 are executed by a processor, the method for generating a video described in the embodiments of the present disclosure may be implemented. For example, when non-transitory computer-executable instructions 931 are executed by a processor, one or more steps of the method for generating a video described above may be performed.

[0094] For example, the storage medium 930 may be applied to the above-mentioned electronic device. For example, the storage medium 930 may include a memory in the electronic device.

[0095] For example, the storage medium may include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a flash memory, or any combination of the above storage media, or other applicable storage media.

[0096] For example, the description of the storage medium 930 can refer to the description of the memory in the embodiment of the electronic device, and the repeated parts are not repeated. The specific functions and technical effects of the storage medium 930 can refer to the description of the method for generating a video above, and are not repeated here.

[0097] It should be noted that in the context of the present disclosure, a computer-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0098] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

[0099] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for generating a video, the method comprising: Acquire a first image and first guidance information; Based on the first guidance information, processing the first image to obtain a second image; The second image is used as a first frame image to be processed to perform camera movement processing and stylization processing to obtain multiple frames of third images; A target video is generated based on the second image and the multiple frames of third images.

2. The method according to claim 1, wherein: The first guide information includes at least one of text information and image information.

3. The method according to claim 1 or 2, wherein: The first guidance information is used to provide one or more of the following constraints for guiding the generation of the second image: Theme constraints, environment constraints, atmosphere constraints, tone constraints, style constraints, distant view constraints, close view constraints.

4. The method according to any one of claims 1 to 3, wherein: The step of performing camera movement processing and stylization processing on the second image as the first frame image to be processed to obtain multiple frames of third images includes: Obtaining the camera movement parameters and second guidance information input by the user; According to the camera movement parameters and the second guide information, the multiple frames of third images are generated frame by frame based on the second image through camera movement processing and stylization processing.

5. The method according to claim 4, wherein: The step of generating the plurality of third images frame by frame based on the second image by means of camera movement processing and stylization processing according to the camera movement parameters and the second guide information includes performing the following target operations multiple times: Acquire a current image to be processed; wherein, when the target operation is performed for the first time, the current image to be processed is the second image; and when the target operation is not performed for the first time, the current image to be processed is the third image generated when the target operation was performed last time; Performing camera movement processing on the current image to be processed to obtain a camera movement image; If the preset condition is not met currently, the camera movement image is determined as a third image; if the preset condition is met currently, the camera movement image is stylized based on the second guide information to obtain a third image.

6. The method according to claim 5, wherein: The second guidance information includes a plurality of constraint conditions for guiding generation of the third image.

7. The method according to claim 6, wherein: When the preset condition is currently met, stylizing the camera movement image based on the second guide information to obtain a third image includes: Acquire some constraint conditions from the second guidance information according to preset rules; The camera movement image is stylized based on the partial constraint conditions to obtain a third image.

8. A device for generating a video, the device comprising: An acquisition module, used for acquiring a first image and first guidance information; A first processing module, configured to process the first image based on the first guidance information to obtain a second image; A second processing module is used to perform camera movement processing and stylization processing on the second image as a first frame image to be processed, so as to obtain multiple frames of third images; A generation module is used to generate a target video based on the second image and the multiple frames of third images.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 7.

10. An electronic device comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

12. A computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images