Creative video generation method based on artificial intelligence

Abstract

The present invention discloses a creative video generation method based on artificial intelligence, and specifically relates to the technical field of video generation. By acquiring the load information of the creative video generation system based on artificial intelligence, the load condition of the creative video generation system based on artificial intelligence is analyzed, and the load state of the creative video generation system based on artificial intelligence is determined. By intelligently analyzing the upsampling processing requirements of video frames under a high load state, an upsampling processing requirement model is constructed based on the picture transition information, background frame ratio information and motion blur information of the video frames, different upsampling processing requirement levels are intelligently allocated, and the upsampling processing strategy is dynamically adjusted to optimize the use of computing resources, avoid unnecessary high-load processing, reduce the overall computing pressure of the system, and avoid the problems of freeze, delay and frame rate drop in the real-time generation and playback of creative videos, thereby improving the user's viewing experience.

Description

Technical Field

[0001] The present invention relates to the technical field of video generation, and more specifically, to a creative video generation method based on artificial intelligence. Background Art

[0002] The creative video generation method based on artificial intelligence is a way to automatically generate video content using artificial intelligence technology. It uses deep learning models to generate the video content required by users by collecting images, video clips, audio, text and other materials used for video generation. The initial resolution of the video output by the model is usually low, so it is necessary to upsample the initial output video. Upsampling refers to the process of increasing the video from a lower resolution to a higher resolution. This process is usually used to increase the resolution of a video when generating, editing or playing it, so that it remains clear when presented on a larger screen or higher-quality display device. However, for creative videos that need to be generated and played in real time, if each generated video frame is upsampled, especially when computing resources are insufficient, the creative video generated and played in real time may become stuck or delayed, and the frame rate may drop, affecting the viewing experience. The long upsampling processing time will also significantly increase the system load. On devices with limited resources (such as mobile devices or embedded systems), it may increase the energy consumption of the device, which in turn causes heating problems. Overheating of the device may cause the system to automatically reduce the frequency to reduce the temperature, further reducing the performance of video generation and playback. Summary of the invention

[0003] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a creative video generation method based on artificial intelligence to solve the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] The creative video generation method based on artificial intelligence includes the following steps:

[0006] Step S1, obtaining load information of the creative video generation system based on artificial intelligence, analyzing the load condition of the creative video generation system based on artificial intelligence, and determining the load state of the creative video generation system based on artificial intelligence;

[0007] Step S2, when the creative video generation system based on artificial intelligence is in a high-load state, obtaining picture transition information, background frame ratio information, and motion blur information of the generated initial video;

[0008] Step S3, constructing an upsampling processing requirement model, analyzing the upsampling processing requirement of each video frame in the initial video, and assigning different upsampling processing requirement levels to each video frame according to the analysis results;

[0009] Step S4: adopt different upsampling processing strategies according to different upsampling processing requirement levels.

[0010] In a preferred embodiment, in step S1, the load information of the creative video generation system based on artificial intelligence is obtained, the load condition of the creative video generation system based on artificial intelligence is analyzed, and the load state of the creative video generation system based on artificial intelligence is determined, as follows:

[0011] Obtaining load information of the artificial intelligence-based creative video generation system, wherein the load information includes CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization;

[0012] Normalize the CPU usage, video memory occupancy, memory read / write speed, display refresh rate, rendering time, and network bandwidth utilization, and build a load monitoring model based on the normalized CPU usage, video memory occupancy, memory read / write speed, display refresh rate, rendering time, and network bandwidth utilization to generate a load monitoring index. The model is based on the following formula , where They represent CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization. Respectively represent the preset proportional coefficients of CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization, and All are greater than 0;

[0013] The load monitoring index is compared with a preset load monitoring index threshold to determine the load status of the artificial intelligence-based creative video generation system. If the load monitoring index is greater than the load monitoring index threshold, a high load status signal is generated; if the load monitoring index is less than or equal to the load monitoring index threshold, a low load status signal is generated.

[0014] In a preferred embodiment, in step S2, the picture transition information includes a picture transition coefficient, the background frame proportion information includes a background frame proportion coefficient, and the motion blur information includes a motion blur coefficient.

[0015] In a preferred embodiment, the logic for obtaining the picture transition coefficient is as follows:

[0016] Get the average brightness difference between the target image frame and the next frame , the expression is as follows , where represents the average pixel value of the target image frame, Indicates the average pixel value of the image frame after the target image frame. is a constant factor;

[0017] Get the contrast difference between the target image frame and the next frame , the expression is as follows , where Represents the standard deviation of the pixel values ​​of the target image frame, Indicates the standard deviation of the pixel values ​​of the image frame after the target image frame. is a constant factor;

[0018] Get the structural difference value between the target image frame and its subsequent frame , the expression is as follows , where It represents the covariance of the pixel values ​​of the target image frame and its subsequent image frame. The expression is as follows , where Represents the pixel value of the target image frame at the nth pixel point, Indicates the pixel value of the nth pixel in the image frame after the target image frame. is a constant factor;

[0019] Calculate the screen transition coefficient , the expression is as follows .

[0020] In a preferred embodiment, the logic for obtaining the background frame ratio coefficient is as follows:

[0021] Calculate the pixel difference between the target image frame and its subsequent image frame at the same pixel position (x, y) , the expression is as follows , where Represents the pixel value of the target image frame at the pixel position (x, y), Represents the pixel value at the pixel position (x, y) of the image frame after the target image frame;

[0022] Compare the pixel difference value with the preset difference threshold, mark the pixels corresponding to the pixel difference value less than or equal to the difference threshold as background pixels, and count the number of background pixels in the target image frame to calculate the background frame ratio coefficient , the expression is as follows , where represents the number of background pixels in the target image frame, Indicates the total number of pixels in the target image frame.

[0023] In a preferred embodiment, the acquisition logic of the motion blur coefficient is as follows:

[0024] Taking the lower left corner of the target image frame as the origin, establish a two-dimensional coordinate system of the image frame, and take each pixel as a coordinate unit;

[0025] Get the horizontal motion component of the pixel in the target image frame , the expression is as follows , where represents the horizontal coordinate of the pixel in the target image frame, Indicates the horizontal coordinate of the pixel in the image frame after the target image frame;

[0026] Get the vertical motion component of the pixel in the target image frame , the expression is as follows , where represents the vertical coordinate of the pixel in the target image frame, Indicates the vertical coordinate of the pixel in the image frame after the target image frame;

[0027] Calculate the motion amplitude of each pixel , the expression is as follows ;

[0028] The motion amplitude is compared with the preset motion amplitude threshold. If the motion amplitude is greater than the motion amplitude threshold, the pixel is marked as a motion blurred pixel and the number of motion blurred pixels is counted. ;

[0029] Calculate motion blur coefficient , the expression is as follows , where represents the motion amplitude of the rth pixel in the target image frame, Is a positive integer.

[0030] In a preferred embodiment, in step S3, an upsampling processing requirement model is constructed to analyze the upsampling processing requirement of each video frame in the initial video, and different upsampling processing requirement levels are assigned to each video frame according to the analysis results, as follows:

[0031] Normalize the picture transition coefficient, background frame ratio coefficient, and motion blur coefficient, build an upsampling processing demand model based on the normalized picture transition coefficient, background frame ratio coefficient, and motion blur coefficient, and output the upsampling processing demand index The model is based on the following formula , where , , Respectively represent the preset proportional coefficients of the picture transition coefficient, the background frame ratio coefficient, and the motion blur coefficient, and , , Both are greater than 0.

[0032] In a preferred embodiment, the upsampling processing requirement index is compared with a preset upsampling processing requirement index stage threshold, and a different upsampling processing requirement level is assigned to each video frame, the upsampling processing requirement index stage threshold includes a first upsampling processing requirement index threshold and a second upsampling processing requirement index threshold, and the first upsampling processing requirement index threshold is less than the second upsampling processing requirement index threshold;

[0033] If the upsampling processing requirement index is greater than or equal to the second upsampling processing requirement index threshold, a first-level upsampling processing requirement level is generated;

[0034] If the upsampling process requirement index is less than the upsampling process requirement index second threshold value, and the upsampling process requirement index is greater than or equal to the upsampling process requirement index first threshold value, then generating a secondary upsampling process requirement level;

[0035] If the upsampling process requirement index is less than the first upsampling process requirement index threshold, a third upsampling process requirement level is generated.

[0036] In a preferred embodiment, when the secondary upsampling processing requirement level is generated, the upsampling processing requirement index corresponding to the generated secondary upsampling processing requirement level is obtained, and the upsampling processing time of the video frame is reduced according to the upsampling processing requirement index, the second threshold of the upsampling processing requirement index, and the first threshold of the upsampling processing requirement index. The adjustment formula of the upsampling processing time is as follows: , where represents the adjusted upsampling processing time, Indicates the maximum upsampling processing time in the video frame of the first-level upsampling processing requirement level, Indicates the first threshold of the upsampling processing requirement index, Indicates the second threshold of the upsampling processing requirement index.

[0037] Technical effects and advantages of the present invention:

[0038] 1. The present invention obtains the load information of the artificial intelligence-based creative video generation system, analyzes the load situation of the artificial intelligence-based creative video generation system, determines the load state of the artificial intelligence-based creative video generation system, and intelligently analyzes the upsampling processing requirements of the video frames under a high load state. Based on the picture transition information, background frame ratio information and motion blur information of the video frames, an upsampling processing requirement model is constructed, different upsampling processing requirement levels are intelligently allocated, the upsampling processing strategy is dynamically adjusted, the use of computing resources is optimized, unnecessary high-load processing is avoided, the overall computing pressure of the system is reduced, and the problems of freeze, delay and frame rate drop in the real-time generation and playback of creative videos are avoided, thereby improving the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings;

[0040] Figure 1 Schematic diagram of the structure of the method of the embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Example: Figure 1 The present invention provides a creative video generation method based on artificial intelligence, comprising the following steps:

[0043] Step S1, obtaining load information of the creative video generation system based on artificial intelligence, analyzing the load condition of the creative video generation system based on artificial intelligence, and determining the load state of the creative video generation system based on artificial intelligence;

[0044] Step S2, when the creative video generation system based on artificial intelligence is in a high-load state, obtaining picture transition information, background frame ratio information, and motion blur information of the generated initial video;

[0045] Step S3, constructing an upsampling processing requirement model, analyzing the upsampling processing requirement of each video frame in the initial video, and assigning different upsampling processing requirement levels to each video frame according to the analysis results;

[0046] Step S4, adopting different upsampling processing strategies according to different upsampling processing requirement levels;

[0047] After using a deep learning model to generate the video content required by the user, it is usually necessary to upsample the initially output video. However, for creative videos that need to be generated and played in real time, if each generated video frame is upsampled, especially when computing resources are insufficient, the creative video generated and played in real time may become stuck or delayed, and the frame rate may drop, affecting the viewing experience. The long upsampling processing time will also significantly increase the system load. On devices with limited resources (such as mobile devices or embedded systems), it may increase the energy consumption of the device, thereby causing heating problems. Overheating of the device may cause the system to automatically reduce the frequency to reduce the temperature, further reducing the performance of video generation and playback. Therefore, in step S1, the load information of the creative video generation system based on artificial intelligence is obtained, the load condition of the creative video generation system based on artificial intelligence is analyzed, and the load state of the creative video generation system based on artificial intelligence is determined, as follows:

[0048] Obtaining load information of the artificial intelligence-based creative video generation system, wherein the load information includes CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization;

[0049] It should be noted that the above load information can be monitored in real time and the corresponding load information can be obtained by using the system's own tools, third-party software, and programming interfaces, which will not be elaborated here;

[0050] CPU usage refers to the proportion of computing resources currently being used by the central processing unit, usually expressed as a percentage. It reflects how busy the CPU is when processing tasks and can indicate the load on the system at a given time;

[0051] Video memory usage refers to the proportion of graphics card memory in use, usually expressed as a percentage. Video memory is used to store data required for graphics processing tasks, such as textures, frame buffers, shader programs, and other information related to graphics rendering. Video memory usage reflects the usage of graphics card memory and shows the current degree of video memory usage. The expression is as follows: , where Indicates the memory usage. Indicates the used video memory capacity. Indicates the total capacity of video memory;

[0052] Memory read / write speed refers to the speed at which the system's memory performs data read and write operations. Memory read / write speed reflects the efficiency of the system in extracting and storing information from memory when running programs and processing data.

[0053] The refresh rate of a monitor refers to the number of times a monitor updates its display content per second. The refresh rate indicates how often a monitor can refresh an image within one second and is a key parameter that affects the smoothness and stability of the monitor image.

[0054] Rendering time refers to the time required to generate and process the video. It usually indicates the length of time from the start of rendering to the completion of rendering. Rendering time is an important indicator to measure the system's graphics processing performance and efficiency.

[0055] Network bandwidth utilization refers to the ratio of the actual data transmission rate of a network connection to its maximum bandwidth. The expression is as follows: , where Indicates the network bandwidth utilization. Indicates the actual data transmission rate, Indicates the expected maximum bandwidth;

[0056] Normalize the CPU usage, video memory occupancy, memory read / write speed, display refresh rate, rendering time, and network bandwidth utilization, and build a load monitoring model based on the normalized CPU usage, video memory occupancy, memory read / write speed, display refresh rate, rendering time, and network bandwidth utilization to generate a load monitoring index. The model is based on the following formula , where They represent CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization. Respectively represent the preset proportional coefficients of CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization, and All are greater than 0;

[0057] It should be noted that It is set according to the actual situation. For example, the expert empowerment method is adopted, that is, experts in related fields are invited to determine the preset proportion coefficients of various indicators through professional opinion surveys and comprehensive evaluations. In addition, various methods such as hierarchical analysis method and fuzzy comprehensive evaluation method can also be considered to determine the preset proportion coefficients to ensure the objectivity and scientificity of the preset proportion coefficients, which will not be elaborated here;

[0058] It can be seen from the above calculation expression that the greater the CPU usage rate, the greater the video memory occupancy rate, the lower the memory read and write speed, the lower the display refresh rate, the longer the rendering time, and the greater the network bandwidth utilization rate, the greater the load monitoring index, indicating that the load of the creative video generation system based on artificial intelligence is more serious. Conversely, the smaller the CPU usage rate, the smaller the video memory occupancy rate, the higher the memory read and write speed, the higher the display refresh rate, the shorter the rendering time, and the lower the network bandwidth utilization rate, the smaller the load monitoring index, indicating that the load of the creative video generation system based on artificial intelligence is lighter.

[0059] Compare the load monitoring index with a preset load monitoring index threshold to determine the load state of the artificial intelligence-based creative video generation system. If the load monitoring index is greater than the load monitoring index threshold, it indicates that the artificial intelligence-based creative video generation system is in a high-load state, and a high-load state signal is generated; if the load monitoring index is less than or equal to the load monitoring index threshold, it indicates that the artificial intelligence-based creative video generation system is not in a high-load state, and a low-load state signal is generated;

[0060] In step S2, when the creative video generation system based on artificial intelligence is in a high-load state, the picture transition information, background frame ratio information and motion blur information of the generated initial video are obtained;

[0061] The picture transition information includes a picture transition coefficient, the background frame proportion information includes a background frame proportion coefficient, and the motion blur information includes a motion blur coefficient;

[0062] The picture transition coefficient is used to quantify the transition smoothness or change degree between video frames. The picture transition coefficient is measured based on the visual difference or change range between video frames. A larger picture transition coefficient indicates that the video frames change dramatically, and the visual difference between the previous and next video frames is significant. Due to the significant change between video frames, the image quality problem caused by the low-resolution video frame is more significant, and the upsampling processing requirement level should be higher accordingly. A smaller picture transition coefficient indicates that the video frames change relatively smoothly, and the visual difference between the previous and next video frames is small. Due to the small change between video frames, the difference between the high-resolution video frame and the low-resolution video frame is small, and the upsampling processing priority of these video frames can be appropriately reduced to simplify the upsampling processing process, or even skip the upsampling processing;

[0063] The logic for obtaining the screen transition coefficient is as follows:

[0064] Get the average brightness difference between the target image frame and the next frame , the expression is as follows , where represents the average pixel value of the target image frame, Indicates the average pixel value of the image frame after the target image frame. It is a constant factor, which is used to avoid the denominator being 0 and can be set according to actual conditions;

[0065] It should be noted that a video is composed of many static images, each of which is called a frame. Therefore, the image frame in this embodiment refers to a video frame, and the target image frame is any frame of the initial creative video generated by the deep learning model. The average pixel value of the image frame is obtained by calculating the pixel value of each pixel in the image and then calculating the average pixel value of all pixels in the entire image. The formula for calculating the pixel value of each pixel in the image is as follows: , where Represents the pixel value of a pixel. Represents the intensity value of the red channel in the RGB (red, green, blue) color model. Represents the intensity value of the green channel in the RGB (red, green, blue) color model. Represents the intensity value of the blue channel in the RGB (red, green, blue) color model. Respectively The preset proportional coefficient can be set according to the actual situation;

[0066] Get the contrast difference between the target image frame and the next frame , the expression is as follows , where Represents the standard deviation of the pixel values ​​of the target image frame, Indicates the standard deviation of the pixel values ​​of the image frame after the target image frame. It is a constant factor, which is used to avoid the denominator being 0 and can be set according to actual conditions;

[0067] Get the structural difference value between the target image frame and its subsequent frame , the expression is as follows , where It represents the covariance of the pixel values ​​of the target image frame and its subsequent image frame. The expression is as follows , where Represents the pixel value of the target image frame at the nth pixel point, Indicates the pixel value of the nth pixel in the image frame after the target image frame. It is a constant factor, which is used to avoid the denominator being 0 and can be set according to actual conditions;

[0068] Calculate the screen transition coefficient , the expression is as follows ;

[0069] The background frame ratio coefficient is used to measure the proportion of the background in the video frame. The background frame ratio coefficient can help identify static scenes that occupy a large proportion of the video frame. These scenes usually do not require frequent or high-quality upsampling processing. Since it is difficult to visually perceive the difference in resolution in static scenes, the upsampling processing priority of these video frames with a large background ratio can be appropriately reduced to simplify the upsampling processing process, or even skip the upsampling processing. That is, the larger the background frame ratio coefficient, the lower the upsampling processing priority;

[0070] The logic for obtaining the background frame ratio coefficient is as follows:

[0071] Calculate the pixel difference between the target image frame and its subsequent image frame at the same pixel position (x, y) , the expression is as follows , where Represents the pixel value of the target image frame at the pixel position (x, y), Represents the pixel value at the pixel position (x, y) of the image frame after the target image frame;

[0072] Compare the pixel difference value with the preset difference threshold, mark the pixels corresponding to the pixel difference value less than or equal to the difference threshold as background pixels, and count the number of background pixels in the target image frame to calculate the background frame ratio coefficient , the expression is as follows , where represents the number of background pixels in the target image frame, Indicates the total number of pixels in the target image frame;

[0073] Motion blur information: including the motion blur coefficient, which is used to quantify the degree of motion blur in the video frame. The higher the motion blur coefficient, the faster the object in the video frame moves or the greater the camera shake, resulting in blurry images. Such frames may not require too high a processing priority during upsampling because the blur effect itself will obscure details.

[0074] The logic for obtaining the motion blur coefficient is as follows:

[0075] Taking the lower left corner of the target image frame as the origin, establish a two-dimensional coordinate system of the image frame, and take each pixel as a coordinate unit;

[0076] It should be noted that the two-dimensional coordinate system of the image frame is applicable to all image frames in the initially output video;

[0077] Get the horizontal motion component of the pixel in the target image frame , the expression is as follows , where represents the horizontal coordinate of the pixel in the target image frame, Indicates the horizontal coordinate of the pixel in the image frame after the target image frame;

[0078] Get the vertical motion component of the pixel in the target image frame , the expression is as follows , where represents the vertical coordinate of the pixel in the target image frame, Indicates the vertical coordinate of the pixel in the image frame after the target image frame;

[0079] Calculate the motion amplitude of each pixel , the expression is as follows ;

[0080] The motion amplitude is compared with the preset motion amplitude threshold. If the motion amplitude is greater than the motion amplitude threshold, it means that the pixel is in fast motion or there is significant motion blur. The pixel is marked as a motion blurred pixel and the number of motion blurred pixels is counted. ;

[0081] Calculate motion blur coefficient , the expression is as follows , where represents the motion amplitude of the rth pixel in the target image frame, is a positive integer;

[0082] Step S3, constructing an upsampling processing requirement model, analyzing the upsampling processing requirement of each video frame in the initial video, and assigning different upsampling processing requirement levels to each video frame according to the analysis results, as follows:

[0083] Normalize the picture transition coefficient, background frame ratio coefficient, and motion blur coefficient, build an upsampling processing demand model based on the normalized picture transition coefficient, background frame ratio coefficient, and motion blur coefficient, and output the upsampling processing demand index The model is based on the following formula , where , , Respectively represent the preset proportional coefficients of the picture transition coefficient, the background frame ratio coefficient, and the motion blur coefficient, and , , All are greater than 0;

[0084] It should be noted that , , It is set according to the actual situation. For example, the expert empowerment method is adopted, that is, experts in related fields are invited to determine the preset proportion coefficients of various indicators through professional opinion surveys and comprehensive evaluations. In addition, various methods such as hierarchical analysis method and fuzzy comprehensive evaluation method can also be considered to determine the preset proportion coefficients to ensure the objectivity and scientificity of the preset proportion coefficients, which will not be elaborated here;

[0085] It can be seen from the above calculation expression that the larger the picture transition coefficient, the smaller the background frame ratio coefficient, and the smaller the motion blur coefficient, the larger the upsampling processing requirement index is, indicating that the upsampling processing requirement of the video frame is higher. On the contrary, the smaller the picture transition coefficient, the larger the background frame ratio coefficient, and the larger the motion blur coefficient, the smaller the upsampling processing requirement index is, indicating that the upsampling processing requirement of the video frame is lower, and the upsampling processing process can be appropriately reduced and simplified, or even skipped.

[0086] Compare the upsampling processing requirement index with a preset upsampling processing requirement index stage threshold, and assign different upsampling processing requirement levels to each video frame, wherein the upsampling processing requirement index stage threshold includes a first upsampling processing requirement index threshold and a second upsampling processing requirement index threshold, and the first upsampling processing requirement index threshold is less than the second upsampling processing requirement index threshold;

[0087] If the upsampling processing requirement index is greater than or equal to the second upsampling processing requirement index threshold, it indicates that the upsampling processing requirement of the video frame is extremely high, and a first-level upsampling processing requirement level is generated;

[0088] If the upsampling processing requirement index is less than the second upsampling processing requirement index threshold, and the upsampling processing requirement index is greater than or equal to the first upsampling processing requirement index threshold, it indicates that the upsampling processing requirement of the video frame is moderate, and a secondary upsampling processing requirement level is generated;

[0089] If the upsampling processing requirement index is less than the first upsampling processing requirement index threshold, it indicates that the upsampling processing requirement of the video frame is extremely low, and a third level of upsampling processing requirement level is generated;

[0090] Step S4, adopting different upsampling processing strategies according to different upsampling processing requirement levels;

[0091] When a first-level upsampling processing requirement level is generated, it means that the upsampling processing requirement of the video frame is extremely high. A high-precision upsampling algorithm can be used to prioritize the allocation of computing resources to ensure that the resolution and clarity of the video frame are maximized;

[0092] When the secondary upsampling processing requirement level is generated, it indicates that the upsampling processing requirement of the video frame is moderate, and the upsampling processing requirement index corresponding to the generated secondary upsampling processing requirement level is obtained. The upsampling processing time of the video frame is reduced according to the upsampling processing requirement index, the second threshold of the upsampling processing requirement index, and the first threshold of the upsampling processing requirement index. The adjustment formula of the upsampling processing time is as follows: , where represents the adjusted upsampling processing time, Indicates the maximum upsampling processing time in the video frame of the first-level upsampling processing requirement level, Indicates the first threshold of the upsampling processing requirement index, represents the second threshold of the upsampling processing requirement index;

[0093] When the third level of upsampling processing requirement is generated, it means that the upsampling processing requirement of the video frame is extremely low, and the upsampling processing can be skipped to further save computing resources;

[0094] The present invention acquires the load information of the artificial intelligence-based creative video generation system, analyzes the load condition of the artificial intelligence-based creative video generation system, determines the load state of the artificial intelligence-based creative video generation system, and intelligently analyzes the upsampling processing requirements of video frames under a high load state. Based on the picture transition information, background frame ratio information and motion blur information of the video frames, an upsampling processing requirement model is constructed, different upsampling processing requirement levels are intelligently allocated, the upsampling processing strategy is dynamically adjusted, the use of computing resources is optimized, unnecessary high-load processing is avoided, the overall computing pressure of the system is reduced, and the problems of freeze, delay and frame rate drop in the real-time generation and playback of creative videos are avoided, thereby improving the user's viewing experience.

[0095] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0096] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A creative video generation method based on artificial intelligence, characterized by: The steps include: Step S1, obtaining load information of the creative video generation system based on artificial intelligence, analyzing the load condition of the creative video generation system based on artificial intelligence, and determining the load state of the creative video generation system based on artificial intelligence; Step S2, when the creative video generation system based on artificial intelligence is in a high-load state, obtaining picture transition information, background frame ratio information, and motion blur information of the generated initial video; The picture transition information includes a picture transition coefficient , the background frame ratio information includes the background frame ratio coefficient , the motion blur information includes the motion blur coefficient ; Step S3, constructing an upsampling processing requirement model, analyzing the upsampling processing requirement of each video frame in the initial video, and assigning different upsampling processing requirement levels to each video frame according to the analysis results; Normalize the picture transition coefficient, background frame ratio coefficient, and motion blur coefficient, build an upsampling processing demand model based on the normalized picture transition coefficient, background frame ratio coefficient, and motion blur coefficient, and output the upsampling processing demand index The model is based on the following formula , where , , Respectively represent the preset proportional coefficients of the picture transition coefficient, the background frame ratio coefficient, and the motion blur coefficient, and , , All are greater than 0; Step S4: adopt different upsampling processing strategies according to different upsampling processing requirement levels.

2. The creative video generation method based on artificial intelligence according to claim 1, characterized in that: In step S1, the load information of the creative video generation system based on artificial intelligence is obtained, the load condition of the creative video generation system based on artificial intelligence is analyzed, and the load state of the creative video generation system based on artificial intelligence is determined, as follows: Obtaining load information of the artificial intelligence-based creative video generation system, wherein the load information includes CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization; Normalize the CPU usage, video memory occupancy, memory read / write speed, display refresh rate, rendering time, and network bandwidth utilization, and build a load monitoring model based on the normalized CPU usage, video memory occupancy, memory read / write speed, display refresh rate, rendering time, and network bandwidth utilization to generate a load monitoring index. The model is based on the following formula , where They represent CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization. Respectively represent the preset proportional coefficients of CPU usage, video memory occupancy, memory read and write speed, display refresh rate, rendering time, and network bandwidth utilization, and All are greater than 0; The load monitoring index is compared with a preset load monitoring index threshold to determine the load status of the artificial intelligence-based creative video generation system. If the load monitoring index is greater than the load monitoring index threshold, a high load status signal is generated; if the load monitoring index is less than or equal to the load monitoring index threshold, a low load status signal is generated.

3. The creative video generation method based on artificial intelligence according to claim 1, characterized in that: The logic for obtaining the screen transition coefficient is as follows: Get the average brightness difference between the target image frame and the next frame , the expression is as follows , where represents the average pixel value of the target image frame, Indicates the average pixel value of the image frame after the target image frame. is a constant factor; Get the contrast difference between the target image frame and the next frame , the expression is as follows , where Represents the standard deviation of the pixel values ​​of the target image frame, Indicates the standard deviation of the pixel values ​​of the image frame after the target image frame. is a constant factor; Get the structural difference value between the target image frame and its subsequent frame , the expression is as follows , where It represents the covariance of the pixel values ​​of the target image frame and its subsequent image frame. The expression is as follows , where Represents the pixel value of the target image frame at the nth pixel point, Indicates the pixel value of the nth pixel in the image frame after the target image frame. is a constant factor; Calculate the screen transition coefficient , the expression is as follows .

4. The creative video generation method based on artificial intelligence according to claim 1, characterized in that: The logic for obtaining the background frame ratio coefficient is as follows: Calculate the pixel difference between the target image frame and its subsequent image frame at the same pixel position (x, y) , the expression is as follows , where Represents the pixel value of the target image frame at the pixel position (x, y), Represents the pixel value at the pixel position (x, y) of the image frame after the target image frame; Compare the pixel difference value with the preset difference threshold, mark the pixels corresponding to the pixel difference value less than or equal to the difference threshold as background pixels, and count the number of background pixels in the target image frame to calculate the background frame ratio coefficient. , the expression is as follows , where represents the number of background pixels in the target image frame, Indicates the total number of pixels in the target image frame.

5. The creative video generation method based on artificial intelligence according to claim 1, characterized in that: The logic for obtaining the motion blur coefficient is as follows: Taking the lower left corner of the target image frame as the origin, establish a two-dimensional coordinate system of the image frame, and take each pixel as a coordinate unit; Get the horizontal motion component of the pixel in the target image frame , the expression is as follows , where represents the horizontal coordinate of the pixel in the target image frame, Indicates the horizontal coordinate of the pixel in the image frame after the target image frame; Get the vertical motion component of the pixel in the target image frame , the expression is as follows , where represents the vertical coordinate of the pixel in the target image frame, Indicates the vertical coordinate of the pixel in the image frame after the target image frame; Calculate the motion amplitude of each pixel , the expression is as follows ; The motion amplitude is compared with the preset motion amplitude threshold. If the motion amplitude is greater than the motion amplitude threshold, the pixel is marked as a motion blurred pixel and the number of motion blurred pixels is counted. ; Calculate motion blur coefficient , the expression is as follows , where represents the motion amplitude of the rth pixel in the target image frame, Is a positive integer.

6. The method for generating creative videos based on artificial intelligence according to claim 1, characterized in that: Compare the upsampling processing requirement index with a preset upsampling processing requirement index stage threshold, and assign different upsampling processing requirement levels to each video frame, wherein the upsampling processing requirement index stage threshold includes a first upsampling processing requirement index threshold and a second upsampling processing requirement index threshold, and the first upsampling processing requirement index threshold is less than the second upsampling processing requirement index threshold; If the upsampling processing requirement index is greater than or equal to the second upsampling processing requirement index threshold, a first-level upsampling processing requirement level is generated; If the upsampling process requirement index is less than the upsampling process requirement index second threshold value, and the upsampling process requirement index is greater than or equal to the upsampling process requirement index first threshold value, then generating a secondary upsampling process requirement level; If the upsampling process requirement index is less than the first upsampling process requirement index threshold, a third upsampling process requirement level is generated.

7. The method for generating creative videos based on artificial intelligence according to claim 6, characterized in that: When the secondary upsampling processing requirement level is generated, the upsampling processing requirement index corresponding to the generated secondary upsampling processing requirement level is obtained, and the upsampling processing time of the video frame is reduced according to the upsampling processing requirement index, the second threshold of the upsampling processing requirement index, and the first threshold of the upsampling processing requirement index. The adjustment formula of the upsampling processing time is as follows: , where represents the adjusted upsampling processing time, Indicates the maximum upsampling processing time in the video frame of the first-level upsampling processing requirement level, Indicates the first threshold of the upsampling processing requirement index, Indicates the second threshold of the upsampling processing requirement index.

Images