A video generation method and device based on AIGC

Abstract

The present application discloses a video generation method and device based on AIGC. After the transmitting end obtains the original video file to be transmitted, it no longer processes the original video file frame by frame according to the target special effect, but performs multiple noise addition on the frame feature map of each frame of the original video file, and transmits the noisy feature video file and the target special effect to the receiving end, so that the receiving end can perform frame-by-frame denoising processing locally according to the target special effect to obtain the target video file with the target special effect. The present application greatly reduces the computing burden of the terminal device at the initiating end, which is beneficial to low-end and mid-end devices with limited computing power; in addition, the present application only transmits feature video files instead of complete video files, effectively reducing the amount of transmitted data and improving video transmission efficiency.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a video generation method and device based on AIGC. Background Art

[0002] With the rapid development of information technology and the popularization of smart devices, video transmission has become an indispensable part of daily life. However, during the video transmission process, users often want to perform various editing processes on the video, such as style change, blurred background, character cartoonization, etc., to meet the needs of personalization, privacy protection or creative expression.

[0003] Traditional video processing methods usually perform complete video editing at the sending end and then send the processed video to the receiving end. Although this method can achieve the effect of video editing, video processing usually requires a large amount of calculation, especially when performing complex image processing and adding special effects, it will significantly increase the calculation burden of the sending end, reduce the overall transmission efficiency, and may even affect the real-time performance of video transmission. Summary of the invention

[0004] The purpose of this application is to provide a video generation method and device based on AIGC, which can improve the above problems.

[0005] The embodiment of the present application is implemented as follows:

[0006] On the first aspect, the present application provides a video generation method based on AIGC, including steps S1 to S4, wherein S1, S2, etc. are merely step identifiers, and the execution order of the method is not necessarily in ascending order of numbers, for example, step S2 may be executed first and then step S1, and the present application does not impose any restrictions thereon.

[0007] S1, obtaining an original video file, extracting features of frames in the original video file, and obtaining a frame feature map of each frame;

[0008] S2, performing noise addition processing on the frame feature map at least twice to obtain a frame target feature map;

[0009] S3, arranging the target feature maps of each frame in chronological order to obtain a feature video file;

[0010] S4, transmitting the feature video file and target special effects to other terminal devices.

[0011] It can be understood that the present application discloses a video generation method based on AIGC. After obtaining the original video file to be transmitted, the original video file is no longer processed frame by frame according to the target special effect, but the frame feature map of each frame of the original video file is denoised multiple times, and the noisy feature video file and the target special effect are transmitted to other target terminal devices, so that other target terminal devices can perform frame-by-frame denoising processing locally according to the target special effect to obtain the target video file with the target special effect. The present application greatly reduces the computing burden of the initiating terminal device, which is beneficial to low-end and mid-end devices with limited computing power; in addition, the present application only transmits feature video files instead of complete video files, which effectively reduces the amount of transmitted data and improves the efficiency of video transmission.

[0012] In an optional embodiment of the present application, at least one of the following items is further included before S2:

[0013] In response to the selection operation, determining the target special effect selected by the user;

[0014] According to the scene type of the frame picture, a target special effect is selected from a preset special effect library.

[0015] It can be understood that the target special effect can be a generative artificial intelligence (AIGC) effect for the background or foreground, such as blurred background and cartoonized face. In addition to the user's own selection through the selection operation, the scene type of the frame can also be identified through the scene recognition model based on the artificial neural network, and then the target special effect corresponding to the scene type can be selected from the pre-stored preset special effects library. For example, if the frame is identified as a selfie scene, the natural beauty special effect is selected as the target special effect in the preset special effects library.

[0016] In an optional embodiment of the present application, S1 includes steps S11 to S13.

[0017] S11, obtaining an original video file, and identifying a foreground area and a background area of ​​each frame in the original video file;

[0018] S12, extracting foreground features of the foreground area to obtain a corresponding foreground feature map of the frame;

[0019] S13, extracting background features of the background area to obtain a corresponding background feature map of the frame image, wherein the foreground feature map and the background features together constitute a frame feature map of the frame image.

[0020] In an optional embodiment of the present application, S2 includes: according to the foreground area targeted by the target special effect, after performing noise processing on the foreground feature map in the frame feature map at least twice, merging the background feature map to obtain a frame target feature map.

[0021] It can be understood that the target special effects may be character face cartoonization, natural beauty special effects, and other special effects for the foreground area of ​​the frame. In order to further reduce the amount of calculation, only the foreground feature map in the frame feature map can be denoised, and then the unprocessed background feature map can be merged. Figure 1 Together they form a frame target feature map for subsequent transmission.

[0022] In an optional embodiment of the present application, S2 includes: according to the background area targeted by the target special effect, after performing noise processing on the background feature map in the frame feature map at least twice, merging the foreground feature map to obtain a frame target feature map.

[0023] It can be understood that the target special effect may be a processing special effect for the background area of ​​the frame image, such as background scene replacement, weather replacement, etc. In order to further reduce the amount of calculation, only the background feature map in the frame feature map can be denoised, and then the unprocessed foreground feature map can be merged. Figure 1 Together they form a frame target feature map for subsequent transmission.

[0024] In an optional embodiment of the present application, the above method includes at least one of the following:

[0025] If the change value of the foreground area in a preset period is less than a first threshold, the foreground area of ​​the first frame in the current period is subjected to at least two noise addition processes, and the frame target feature map of the first frame is used in all frames in the current period;

[0026] If the change value of the background area within the preset period is less than the second threshold, the background area of ​​the first frame in this period is only subjected to at least two noise addition processes, and the frame target feature map of the first frame is used in all frames in this period.

[0027] It is understandable that for a video file, if the foreground or background area of ​​consecutive frames has little change, the foreground or background of the first frame can be used to replace the subsequent frames without being visually perceived by the user. This can further reduce the amount of computation for video processing and lower the performance requirements for the sender.

[0028] In an optional embodiment of the present application, the above method includes steps S5 to S7.

[0029] S5, receiving the feature video file and the target special effect transmitted by other terminal devices;

[0030] S6, performing denoising processing on each of the frame target feature maps at least twice based on the target special effect, so that a new feature map obtained by each denoising processing carries information corresponding to the target special effect;

[0031] S7, converting the new feature map to obtain a target frame picture, arranging each of the target frame pictures in chronological order, and obtaining a target video file.

[0032] It can be understood that since the sender does not send a complete video file, after the receiver receives the feature video file and the target special effects, it is necessary to denoise the frame target feature map according to the target special effects, so that the new feature map carries the information corresponding to the target special effects, and finally accurately restores the target video file desired by the sender.

[0033] In an optional embodiment of the present application, the S6 includes the following steps:

[0034] S61, based on the foreground area targeted by the target special effect, performing denoising processing on the foreground feature map in each of the frame target feature maps at least twice, so that a new foreground feature map obtained by each denoising processing carries information corresponding to the target special effect;

[0035] S62, merging the new foreground feature map and the background feature map to obtain a new feature map.

[0036] In an optional embodiment of the present application, the S6 includes the following steps:

[0037] S63, based on the background area targeted by the target special effect, performing denoising processing on the background feature map in each of the frame target feature maps at least twice, so that a new background feature map obtained by each denoising processing carries information corresponding to the target special effect;

[0038] S64, merging the new background feature map and the foreground feature map to obtain a new feature map.

[0039] In a second aspect, the present application discloses a video generation device based on AIGC, comprising a processor, an input device, an output device and a memory, wherein the processor, input device, output device and memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method as described in any one of the first aspects.

[0040] In a third aspect, the present application discloses a computer-readable storage medium, wherein the computer storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method as described in any one of the first aspects.

[0041] Beneficial effects: The present application discloses a video generation method and device based on AIGC. After the sending end obtains the original video file to be transmitted, it no longer processes the original video file frame by frame according to the target special effect, but performs multiple noise additions on the frame feature graphs of each frame of the original video file, and transmits the noisy feature video file and the target special effect to the receiving end, so that the receiving end can perform frame-by-frame denoising processing locally according to the target special effect to obtain the target video file with the target special effect. The present application greatly reduces the computing burden of the terminal device on the initiating end, which is beneficial to low-end and mid-end devices with limited computing power; in addition, the present application only transmits feature video files instead of complete video files, which effectively reduces the amount of data transmitted and improves the efficiency of video transmission. Since the present method has low computing requirements and efficient data transmission characteristics, it is also suitable for application scenarios with high real-time requirements such as real-time video communication and online live broadcast.

[0042] In order to make the above-mentioned objects, features and advantages of the present application more obvious and understandable, optional embodiments are specifically listed below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a flowchart of a video generation method based on AIGC provided by the present application;

[0045] Figure 2 It is a flow chart of the method for obtaining background feature map and foreground feature map provided by this application;

[0046] Figure 3 It is a flow chart of a method for obtaining a frame target feature map by adding a noise to a foreground feature map provided in the present application;

[0047] Figure 4 It is a flow chart of a method for obtaining a frame target feature map by adding a noise background feature map provided in the present application;

[0048] Figure 5 This is a flow chart of the method for obtaining a target frame image by denoising a background feature map provided in this application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] First, as Figure 1 As shown, the present application provides a video generation method based on AIGC, including steps S1 to S4, wherein S1, S2, etc. are only step identifiers, and the execution order of the method is not necessarily in the order of numbers from small to large, for example, step S2 may be executed first and then step S1, and the present application does not impose any restrictions.

[0051] S1, obtaining an original video file, extracting features of frames in the original video file, and obtaining a frame feature map of each frame.

[0052] In the embodiment of the present application, the original video file can be understood as the video file to which the target effect is to be added, and there are multiple ways to determine the original video file. As one way, the video currently shot by the terminal device can be used as the original video file; as another way, the terminal device can use the video transmitted by other devices as the original video file.

[0053] After the original video file is determined, the frame feature map of the frame in the original video file can be obtained. In the frame feature map, each element represents the color and texture information of the corresponding position in the frame. As a way, the frame can be processed by an image encoder to obtain the corresponding frame feature map. Optionally, the image encoder can be a VAE endoder in a stable diffusion model.

[0054] S2, after performing noise addition processing on the frame feature map at least twice, a frame target feature map is obtained.

[0055] After obtaining the frame feature graph corresponding to each frame, a forward diffusion process can be performed based on the frame feature graph. During the forward diffusion process, the frame feature graph can be subjected to multiple noise addition processes to obtain a frame target feature graph. In each noise addition process, the added noise is random noise. The random noise can be Gaussian noise.

[0056] S3, arranging the target feature maps of each frame in chronological order to obtain a feature video file.

[0057] S4, transmit the feature video file and target special effects to other terminal devices.

[0058] like Figure 1 As shown, when executing steps S1 to S4, the execution subject is the first terminal device, and at this time, the terminal devices including the second terminal device in the figure are all other terminal devices.

[0059] It can be understood that the present application discloses a video generation method based on AIGC. After obtaining the original video file to be transmitted, the original video file is no longer processed frame by frame according to the target special effect, but the frame feature map of each frame of the original video file is denoised multiple times, and the noisy feature video file and the target special effect are transmitted to other target terminal devices, so that other target terminal devices can perform frame-by-frame denoising processing locally according to the target special effect to obtain the target video file with the target special effect. The present application greatly reduces the computing burden of the initiating terminal device, which is beneficial to low-end and mid-end devices with limited computing power; in addition, the present application only transmits feature video files instead of complete video files, which effectively reduces the amount of transmitted data and improves the efficiency of video transmission.

[0060] In an optional embodiment of the present application, at least one of the following items is further included before S2:

[0061] In response to the selection operation, determining the target special effect selected by the user;

[0062] According to the scene type of the frame, select the target special effect from the preset special effects library.

[0063] It can be understood that the target special effect can be a generative artificial intelligence (AIGC) effect for the background or foreground, such as blurred background and cartoonized face. In addition to the user's own selection through the selection operation, the scene type of the frame can also be identified through the scene recognition model based on the artificial neural network, and then the target special effect corresponding to the scene type can be selected from the pre-stored preset special effects library. For example, if the frame is identified as a selfie scene, the natural beauty special effect is selected as the target special effect in the preset special effects library.

[0064] In an optional embodiment of the present application, S1 includes steps S11 to S13. S11, obtaining the original video file, identifying the foreground area and the background area of ​​each frame in the original video file. S12, extracting the foreground features of the foreground area, and obtaining the foreground feature map of the corresponding frame. S13, extracting the background features of the background area, and obtaining the background feature map of the corresponding frame. The foreground feature map and the background features together constitute the frame feature map of the frame.

[0065] For example, Figure 2Take the left frame as an example, the content of the foreground area of ​​the picture is the image of a little girl, and the content of the background area of ​​the picture is the seaside environment. According to steps S12 and S13, features can be extracted from the background area and the foreground area respectively to obtain a background feature map and a foreground feature map.

[0066] In an optional embodiment of the present application, Figure 3 As shown, S2 includes: according to the foreground area targeted by the target special effect, after performing noise addition processing on the foreground feature map in the frame feature map at least twice, merging the background feature map to obtain the frame target feature map.

[0067] It can be understood that the target special effects may be character face cartoonization, natural beauty special effects, and other special effects for the foreground area of ​​the frame. In order to further reduce the amount of calculation, only the foreground feature map in the frame feature map can be denoised, and then the unprocessed background feature map can be merged. Figure 1 Together they form a frame target feature map for subsequent transmission.

[0068] In an optional embodiment of the present application, Figure 4 As shown, S2 includes: according to the background area targeted by the target special effect, after performing noise addition processing on the background feature map in the frame feature map at least twice, merging the foreground feature map to obtain the frame target feature map.

[0069] It can be understood that the target special effect may be a processing special effect for the background area of ​​the frame image, such as background scene replacement, weather replacement, etc. In order to further reduce the amount of calculation, only the background feature map in the frame feature map can be denoised, and then the unprocessed foreground feature map can be merged. Figure 1 Together they form a frame target feature map for subsequent transmission.

[0070] In an optional embodiment of the present application, the above method includes at least one of the following:

[0071] If the change value of the foreground area in the preset period is less than the first threshold, the foreground area of ​​the first frame in the current period is subjected to at least two noise addition processes, and the frame target feature map of the first frame is used in all frames in the current period;

[0072] If the change value of the background area within the preset period is less than the second threshold, the background area of ​​the first frame in this period is only subjected to at least two noise addition processes, and the frame target feature map of the first frame is used in all frames in this period.

[0073] It is understandable that for a video file, if the foreground or background area of ​​consecutive frames has little change, the foreground or background of the first frame can be used to replace the subsequent frames without being visually perceived by the user. This can further reduce the amount of computation for video processing and lower the performance requirements for the sender.

[0074] In an optional embodiment of the present application, the above method includes steps S5 to S7.

[0075] S5, receiving feature video files and target special effects transmitted from other terminal devices;

[0076] S6, performing denoising processing on the target feature map of each frame at least twice based on the target special effect, so that a new feature map obtained by each denoising processing carries information corresponding to the target special effect;

[0077] S7, converting the new feature map to obtain a target frame image, arranging the target frame images in chronological order, and obtaining a target video file.

[0078] For example, in the case where the frame feature map is obtained by encoding the frame picture through the VAE endoder in the target diffusion model, the new feature map can be decoded by the VAE decoder in the target diffusion model to obtain the target frame picture.

[0079] like Figure 1 As shown, when executing steps S5 to S7, the execution subject is the second terminal device, and at this time, the terminal devices including the first terminal device in the figure are all other terminal devices.

[0080] It can be understood that since the sender does not send a complete video file, after the receiver receives the feature video file and the target special effects, it is necessary to denoise the frame target feature map according to the target special effects, so that the new feature map carries the information corresponding to the target special effects, and finally accurately restores the target video file desired by the sender.

[0081] In an optional embodiment of the present application, S6 includes the following steps:

[0082] S61, based on the foreground area targeted by the target special effect, perform denoising processing on the foreground feature map in the target feature map of each frame at least twice, so that a new foreground feature map obtained by each denoising processing carries information corresponding to the target special effect.

[0083] S62, merging the new foreground feature map and the background feature map to obtain a new feature map.

[0084] In an optional embodiment of the present application, S6 includes the following steps:

[0085] S63, based on the background area targeted by the target special effect, perform denoising processing on the background feature map in each frame target feature map at least twice, so that a new background feature map obtained by each denoising processing carries information corresponding to the target special effect.

[0086] S64, merging the new background feature map and the foreground feature map to obtain a new feature map.

[0087] For example, assuming that the target feature is the replacement of the garden background, then step S63 needs to perform denoising on the background feature map one by one, so that each new background feature map after denoising carries information corresponding to "garden" until the final new background feature map is obtained. Figure 5 As shown in the figure, the new background feature map obtained after multiple denoising is combined with the unprocessed foreground feature map to form a new feature map, and then the VAE decoder in the standard diffusion model is used to decode the new feature map, and finally restore the target frame with the background of the garden and the foreground of the little girl.

[0088] In a second aspect, the present application provides a video generation device based on AIGC. The video generation device based on AIGC includes one or more processors; one or more input devices, one or more output devices and a memory. The above-mentioned processor, input device, output device and memory are connected via a bus. The memory is used to store a computer program, which includes program instructions, and the processor is used to execute the program instructions stored in the memory. Among them, the processor is configured to call the program instructions to perform the operation of any method of the first aspect.

[0089] It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0090] Input devices may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and output devices may include a display (LCD, etc.), a speaker, etc.

[0091] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0092] In a specific implementation, the processor, input device, and output device described in the embodiments of the present invention may execute the implementation method described in any method of the first aspect, or may execute the implementation method of the terminal device described in the embodiments of the present invention, which will not be repeated here.

[0093] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the steps of any method of the first aspect are implemented.

[0094] The computer-readable storage medium may be an internal storage unit of the terminal device of any of the aforementioned embodiments, such as a hard disk or memory of the terminal device. The computer-readable storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the terminal device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0095] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0096] In the several embodiments provided in the present application, it should be understood that the disclosed terminal device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0097] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0098] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0099] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0100] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0101] The above description is only an optional embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other.

[0102] The above description is only an optional embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A video generation method based on AIGC, characterized in that: The following steps are involved: S1, obtaining an original video file, extracting features of frames in the original video file, and obtaining a frame feature map of each frame; S2, performing noise addition processing on the frame feature map at least twice to obtain a frame target feature map; S3, arranging the target feature maps of each frame in chronological order to obtain a feature video file; S4, transmitting the feature video file and target special effects to other terminal devices; Before S2, at least one of the following is also included: In response to the selection operation, determining the target special effect selected by the user; Selecting a target special effect from a preset special effect library according to the scene type of the frame; The S1 includes: S11, obtaining an original video file, and identifying a foreground area and a background area of ​​each frame in the original video file; S12, extracting foreground features of the foreground area to obtain a corresponding foreground feature map of the frame; S13, extracting background features of the background area to obtain a corresponding background feature map of the frame image, wherein the foreground feature map and the background features together constitute a frame feature map of the frame image.

2. The video generation method based on AIGC according to claim 1, characterized in that: The S2 includes: according to the foreground area targeted by the target special effect, after performing noise addition processing on the foreground feature map in the frame feature map at least twice, merging the background feature map to obtain a frame target feature map.

3. The video generation method based on AIGC according to claim 1, characterized in that: The S2 includes: according to the background area targeted by the target special effect, after performing noise addition processing at least twice on the background feature map in the frame feature map, merging the foreground feature map to obtain a frame target feature map.

4. The AIGC-based video generation method according to claim 2 or 3, characterized in that: Include at least one of the following: If the change value of the foreground area in a preset period is less than a first threshold, the foreground area of ​​the first frame in the current period is subjected to at least two noise addition processes, and the frame target feature map of the first frame is used in all frames in the current period; If the change value of the background area within the preset period is less than the second threshold, the background area of ​​the first frame in this period is only subjected to at least two noise addition processes, and the frame target feature map of the first frame is used in all frames in this period.

5. The AIGC-based video generation method according to any one of claims 1 to 3, characterized in that: The following steps are involved: S5, receiving the feature video file and the target special effect transmitted by other terminal devices; S6, performing denoising processing on each of the frame target feature maps at least twice based on the target special effect, so that a new feature map obtained by each denoising processing carries information corresponding to the target special effect; S7, converting the new feature map to obtain a target frame picture, arranging each of the target frame pictures in chronological order, and obtaining a target video file.

6. The AIGC-based video generation method according to claim 5, characterized in that: The S6 comprises the following steps: S61, based on the foreground area targeted by the target special effect, performing denoising processing on the foreground feature map in each of the frame target feature maps at least twice, so that a new foreground feature map obtained by each denoising processing carries information corresponding to the target special effect; S62, merging the new foreground feature map and the background feature map to obtain a new feature map.

7. The AIGC-based video generation method according to claim 5, characterized in that: The S6 comprises the following steps: S63, based on the background area targeted by the target special effect, performing denoising processing on the background feature map in each of the frame target feature maps at least twice, so that a new background feature map obtained by each denoising processing carries information corresponding to the target special effect; S64, merging the new background feature map and the foreground feature map to obtain a new feature map.

8. A video generation device based on AIGC, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

Images