Video processing method and device, computer device and storage medium

Abstract

The application relates to a video processing method and device, computer equipment, a storage medium and a computer program product. Each to-be-processed video frame in a to-be-processed video frame set is down-sampled through a target multi-scale reconstruction model to obtain a plurality of sampling video frames of different scales, pixel feature information of each sampling video frame is obtained through a local optimization network in the model, global dependency information of each sampling video frame is obtained through a global optimization network in the model, an adjusted target video frame set is output according to the pixel feature information, the global dependency information and the to-be-processed video frame, and a processed target video is obtained according to at least one group of target video frame sets corresponding to the to-be-processed video frame set. Compared with a traditional video processing mode based on a single network, the application parallelly processes video frames of multiple scales through a model composed of a local optimization network and a global optimization network, thereby improving the comprehensiveness of video quality improvement processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a video processing method and device, computer equipment, storage medium and computer program product. BACKGROUND

[0002] With the continuous development of computer vision technology, video as an important information carrier is widely used in online meetings, online courses, face-to-face interviews and other scenarios. In this case, the clarity of the video is extremely important. At present, the way to improve the quality of video clarity is usually through a serial processing mode of a single network structure. However, the serial processing of a single network structure cannot balance the local details and global consistency of the video, resulting in incomplete video processing.

[0003] Therefore, the current video processing method has the defect of insufficient comprehensive processing. SUMMARY

[0004] Therefore, it is necessary to provide a video processing method, device, computer equipment, computer readable storage medium and computer program product capable of improving the comprehensive processing of video.

[0005] In a first aspect, the present application provides a video processing method, which comprises:

[0006] obtaining a to-be-processed video, and extracting at least one set of to-be-processed video frames from the to-be-processed video; the set of to-be-processed video frames comprises at least one to-be-processed video frame;

[0007] inputting the set of to-be-processed video frames into a target multi-scale reconstruction model, performing down-sampling processing on each to-be-processed video frame in the set of to-be-processed video frames by the target multi-scale reconstruction model to obtain a plurality of sampled video frames of different scales, obtaining pixel feature information of each sampled video frame by a local optimization network, obtaining global dependency information of each sampled video frame by a global optimization network, and outputting an adjusted set of target video frames according to the pixel feature information, the global dependency information and the to-be-processed video frame; the target multi-scale reconstruction model comprises the local optimization network and the global optimization network;

[0008] obtaining a processed target video according to the set of target video frames corresponding to each of the at least one set of to-be-processed video frames.

[0009] In one embodiment, the extracting the at least one set of to-be-processed video frames from the to-be-processed video comprises:

[0010] performing data cleaning on the to-be-processed video to obtain a data-cleaned to-be-processed video;

[0011] The data-cleaning processed video is divided according to time sequence to obtain a plurality of to-be-processed video frames;

[0012] The plurality of to-be-processed video frames are combined into at least one set of to-be-processed video frame sets, and the number of to-be-processed video frames in each set of to-be-processed video frame sets is the same.

[0013] In one embodiment, the data cleaning of the to-be-processed video to obtain the data-cleaning processed video comprises:

[0014] At least one of removing abnormal frequency and removing pulse burr is performed on the video spectrum of the to-be-processed video, and at least one of removing abnormal frequency, removing pulse burr and removing bottom noise is performed on the audio spectrum of the to-be-processed video to obtain the data-cleaning processed video.

[0015] In one embodiment, the down-sampling processing of each to-be-processed video frame in the set of to-be-processed video frames by the target multi-scale reconstruction model to obtain a plurality of sampling video frames of different scales comprises:

[0016] The target multi-scale reconstruction model performs down-sampling on the to-be-processed video frame based on a first multiple to obtain a sampling video frame of a first scale;

[0017] The target multi-scale reconstruction model performs down-sampling on the to-be-processed video frame based on a second multiple to obtain a sampling video frame of a second scale;

[0018] The target multi-scale reconstruction model performs down-sampling on the to-be-processed video frame based on a third multiple to obtain a sampling video frame of a third scale;

[0019] The first multiple, the second multiple and the third multiple are adjacent integer multiples and increase in turn.

[0020] In one embodiment, the local optimization network is a residual block network, and the global optimization network is an attention mechanism deep learning network.

[0021] The pixel feature information of each sampling video frame is obtained by the local optimization network, and the global dependency information of each sampling video frame is obtained by the global optimization network, comprising:

[0022] The high-frequency pixel information in the plurality of sampling video frames is extracted by the residual block network as the pixel feature information of the plurality of sampling video frames.

[0023] determine the self-similarity information corresponding to each of the sampling video frames through an attention mechanism deep learning network, and determine the global information corresponding to each of the sampling video frames based on the optical flow information in each of the sampling video frames and the optical flow information in a preset number of adjacent sampling video frames;

[0024] obtain global dependency information of each of the sampling video frames according to the self-similarity information and the global information.

[0025] In one of the embodiments, the outputting of the adjusted target video frame set according to the pixel feature information, the global dependency information and the to-be-processed video frame includes:

[0026] performing scale restoration processing on each of the sampling video frames according to the pixel feature information and the global dependency information by the target multi-scale reconstruction model to obtain a plurality of restored video frames; the size of the restored video frame is the same as that of the to-be-processed video frame;

[0027] stacking the plurality of restored video frames and the to-be-processed video frame to obtain an adjusted target video frame, and outputting a target video frame set composed of a plurality of target video frames.

[0028] In one of the embodiments, the obtaining of the processed target video according to the target video frame set corresponding to each of the at least one set of to-be-processed video frame set includes:

[0029] obtaining the time information of each target video frame in the target video frame set corresponding to each of the at least one set of to-be-processed video frame set;

[0030] combining each target video frame in the target video frame set corresponding to each of the at least one set of to-be-processed video frame set according to the order of the time information to obtain a processed target video.

[0031] In one of the embodiments, the method further includes:

[0032] obtaining an initial video frame sample set and an adjusted video frame sample set corresponding to the initial video frame sample set; the initial video frame sample set includes at least one initial video frame sample; the adjusted video frame sample set includes at least one adjusted video frame sample;

[0033] input the initial video frame sample set to a to-be-trained multi-scale reconstruction model, perform downsampling processing on each initial video frame sample in the initial video frame sample set by the to-be-trained multi-scale reconstruction model, obtain a plurality of sampling video frame samples of different scales, obtain pixel feature information of each sampling video frame sample by a local optimization network, obtain global dependency information of each sampling video frame sample by a global optimization network, and output an adjusted video frame test set according to the pixel feature information, the global dependency information and the initial video frame sample; the adjusted video frame test set includes at least one test video frame sample;

[0034] According to the similarity between each test video frame sample in the video frame test set and each adjusted video frame sample in the adjusted video frame sample set, the model parameters of the to-be-trained multi-scale reconstruction model are adjusted until a target multi-scale reconstruction model is obtained when a preset training condition is met.

[0035] In a second aspect, the present application provides a video processing device, the device comprising:

[0036] The obtaining module is configured to obtain a to-be-processed video and extract at least one to-be-processed video frame set in the to-be-processed video; the to-be-processed video frame set includes at least one to-be-processed video frame.

[0037] The processing module is configured to input the to-be-processed video frame set to a target multi-scale reconstruction model, perform downsampling processing on each to-be-processed video frame in the to-be-processed video frame set by the target multi-scale reconstruction model, obtain a plurality of sampling video frames of different scales, obtain pixel feature information of each sampling video frame by a local optimization network, obtain global dependency information of each sampling video frame by a global optimization network, and output an adjusted target video frame set according to the pixel feature information, the global dependency information and the to-be-processed video frame; the target multi-scale reconstruction model includes the local optimization network and the global optimization network.

[0038] The output module is configured to obtain a processed target video according to the target video frame set corresponding to each of the at least one to-be-processed video frame set.

[0039] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0040] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the above method.

[0041] In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method described above.

[0042] The video processing method, device, computer device, storage medium and computer program product described above, by extracting at least one set of to-be-processed video frames in a to-be-processed video, inputting the set of to-be-processed video frames into a target multi-scale reconstruction model, performing down-sampling on each to-be-processed video frame in the set of to-be-processed video frames by the target multi-scale reconstruction model to obtain a plurality of sampled video frames of different scales, obtaining pixel feature information of each sampled video frame by a local optimization network in the model, obtaining global dependency information of each sampled video frame by a global optimization network in the model, outputting a set of adjusted target video frames according to the pixel feature information, the global dependency information and the to-be-processed video frames, and obtaining a processed target video according to the set of target video frames corresponding to each set of to-be-processed video frames. Compared with the traditional video processing method based on a single network, the present application improves the comprehensiveness of video quality improvement processing by using a model composed of a local optimization network and a global optimization network to process video frames of multiple scales in parallel. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A flowchart of a video processing method in an embodiment;

[0044] Figure 2 A flowchart of a preprocessing step in an embodiment;

[0045] Figure 3 A flowchart of a format conversion step in an embodiment;

[0046] Figure 4 A flowchart of a training step in an embodiment;

[0047] Figure 5 A structural diagram of a video processing system in an embodiment;

[0048] Figure 6 A flowchart of a video processing method in another embodiment;

[0049] Figure 7 A structural block diagram of a video processing device in an embodiment;

[0050] Figure 8 An internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0051] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0052] In one embodiment, as shown in Figure 1 A video processing method is provided, and the present embodiment is exemplified by the method applied to a terminal. It should be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and can be implemented through the interaction of the terminal and the server, including the following steps:

[0053] In step S202, a to-be-processed video is obtained, and at least one set of to-be-processed video frames is extracted from the to-be-processed video. The set of to-be-processed video frames includes at least one to-be-processed video frame.

[0054] The to-be-processed video can be a video that needs to be improved in quality, for example, the to-be-processed video can be a blurred video that needs to be processed into a video with higher clarity. The to-be-processed video can include multiple frames of images, so that the terminal can extract at least one to-be-processed video frame from the to-be-processed video after obtaining the to-be-processed video, and combine at least one to-be-processed video frame to obtain at least one set of to-be-processed video frames. The number of to-be-processed video frames included in each set of to-be-processed video frames can be the same or different.

[0055] In addition, after obtaining the to-be-processed video, the terminal can further pre-process the video and then extract the video frames. For example, in an embodiment, extracting at least one set of to-be-processed video frame collection from the to-be-processed video includes: performing data cleaning on the to-be-processed video to obtain a data-cleaned to-be-processed video; segmenting the data-cleaned to-be-processed video according to time sequence to obtain a plurality of to-be-processed video frames; and combining the plurality of to-be-processed video frames into at least one set of to-be-processed video frame collection, wherein the number of to-be-processed video frames in each set of to-be-processed video frame collection is the same. In this embodiment, the terminal can perform data cleaning on the to-be-processed video to obtain a data-cleaned to-be-processed video. Specifically, in some embodiments, the data cleaning can include removing abnormal frequency, removing pulse glitch, and removing bottom voice noise, etc. The to-be-processed video can include video information and audio information, and the terminal can perform data cleaning on the video and audio of the to-be-processed video respectively. For example, the terminal can determine the video spectrum of the to-be-processed video based on the video information of the to-be-processed video, and determine the audio spectrum of the to-be-processed video based on the audio information of the to-be-processed video. Thus, the terminal can perform at least one of removing abnormal frequency and removing pulse glitch on the video spectrum of the to-be-processed video, and perform at least one of removing abnormal frequency, removing pulse glitch, and removing bottom voice noise on the audio spectrum of the to-be-processed video, to obtain a data-cleaned to-be-processed video. After performing data cleaning on the to-be-processed video, the terminal can segment the data-cleaned to-be-processed video to obtain a plurality of to-be-processed video frames. The terminal can segment the data-cleaned to-be-processed video according to the time sequence of the to-be-processed video to obtain a plurality of to-be-processed video frames. For example, a data-cleaned to-be-processed video has one second, and the frame rate of this video is 30 frames per second, so the terminal can segment it into thirty to-be-processed video frames. After obtaining a plurality of to-be-processed video frames, the terminal can further combine the to-be-processed video frames. The terminal can combine the plurality of to-be-processed video frames into at least one set of to-be-processed video frame collection, and the number of to-be-processed video frames in each set of to-be-processed video frame collection is the same.

[0056] Specifically, the process of pre-processing and segmenting the to-be-processed video by the terminal can be as shown in FIG. 3, Figure 2 Figure 2 ​A flowchart of a preprocessing step in an embodiment. The terminal can first obtain the video to be processed and its related information, such as the time, frame rate, original video format and target video format of the video to be processed, etc. The target video format can be the video format that needs to be converted to, which can be set according to actual conditions. For the audio and video spectrum in the video to be processed, the terminal can perform data cleaning on the audio and video spectrum of the video to be processed, such as removing abnormal frequencies, pulse glitches, background noise, etc. The terminal can also extract video frames in the data cleaned video to be processed, a video frame can be a picture, so after the terminal extracts all the video frames in the data cleaned video to be processed, it can form a picture sequence, and the terminal can group at least one video frame in time sequence, so that each group contains a preset number of video frames, for example, each group of video frames contains 7 video frames, and if it is less than the preset number, it is filled forward, so that the terminal can obtain a video frame set that can be used by the multi-scale structure algorithm and model. The preset number can be an odd number, and the video frame corresponding to the median of the preset number can be a support frame, which is used for countermeasures.

[0057] In step S204, the target multi-scale reconstruction model is input with the video frame set to be processed, each video frame to be processed in the video frame set to be processed is down-sampled by the target multi-scale reconstruction model to obtain a plurality of sample video frames of different scales, pixel feature information of each sample video frame is obtained through a local optimization network, global dependency information of each sample video frame is obtained through a global optimization network, and an adjusted target video frame set is output according to the pixel feature information, the global dependency information and the video frame to be processed; the target multi-scale reconstruction model includes the local optimization network and the global optimization network.

[0058] The terminal can obtain at least one set of to-be-processed video frame sets based on the to-be-processed video. For each set of to-be-processed video frames, the terminal can combine the to-be-processed video frames and input them into a target multi-scale reconstruction model. The target multi-scale reconstruction model can be used to improve the definition of video frames based on the set of video frames. The target multi-scale reconstruction model includes a local optimization network and a global optimization network. The local optimization network can extract detailed information in the video frames, and the global optimization network can improve the self-similarity of a single video frame and the global consistency between multiple consecutive video frames. The target multi-scale reconstruction model can be trained based on an initial set of video frame samples and an adjusted set of video frame samples. The terminal can perform downsampling on each to-be-processed video frame in the set of to-be-processed video frames using the target multi-scale reconstruction model. The downsampling can be performed multiple times, for example, the terminal can perform multiple downsamplings on a to-be-processed video frame to obtain multiple sampled video frames, and each sampled video frame can have a different size, i.e., the terminal can perform multiple samplings with different sizes on a to-be-processed video frame to obtain multiple sampled video frames with different sizes. The size of an image refers to the size of the image, i.e., the different sizes of the sampled video frames refer to different sizes of the sampled video frames. For example, if the to-be-processed video frame is 300*300, the terminal can perform three samplings with different sizes on the to-be-processed video frame to obtain a 300*300 sampled video frame, a 150*150 sampled video frame, and a 100*100 sampled video frame. The structures of the images with different sizes are consistent because some pixels are compressed during downsampling, but the overall structure remains unchanged.

[0059] After obtaining the multiple sampled video frames with different sizes, the terminal can obtain pixel feature information of each sampled video frame using the local optimization network in the target multi-scale reconstruction model, and obtain global dependency information of each sampled video frame using the global optimization network in the target multi-scale reconstruction model. The pixel feature information can be the pixel detail information in each sampled video frame. The global dependency information can be global receptive field dependency information, which ensures consistency between the video frames in the set of video frames. The terminal can also obtain an adjusted set of target video frames output by the target multi-scale reconstruction model based on the pixel feature information, the global dependency information, and the to-be-processed video frames. The set of target video frames includes multiple adjusted target video frames. Specifically, the target multi-scale reconstruction model can improve the definition of each to-be-processed video frame based on the pixel feature information, and improve the consistency between the video frames with improved definition based on the global dependency information, so that the adjusted target video frames can be obtained by superimposing the video frames with improved definition and consistency on the original to-be-processed video frames.

[0060] In step S206, a processed target video is obtained according to the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets.

[0061] The to-be-processed video frame set corresponding to the to-be-processed video can have at least one set. After each set of to-be-processed video frame sets is processed by the target multi-scale reconstruction model, a processed target video can be obtained based on the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets. For example, the terminal can combine each target video frame set in order, and obtain a target video based on the combined target video frame set.

[0062] In the video processing method, at least one set of to-be-processed video frame sets is extracted from a to-be-processed video, the to-be-processed video frame set is input into a target multi-scale reconstruction model, each to-be-processed video frame in the to-be-processed video frame set is down-sampled by the target multi-scale reconstruction model to obtain a plurality of sampling video frames of different scales, pixel feature information of each sampling video frame is obtained by a local optimization network in the model, global dependency information of each sampling video frame is obtained by a global optimization network in the model, an adjusted target video frame set is output according to the pixel feature information, the global dependency information, and the to-be-processed video frame, and a processed target video is obtained according to the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets. Compared with the traditional video processing method based on a single network, the model composed of the local optimization network and the global optimization network is used to process video frames of multiple scales in parallel, thereby improving the comprehensiveness of video quality improvement processing.

[0063] In one embodiment, the down-sampling processing of each to-be-processed video frame in the to-be-processed video frame set by the target multi-scale reconstruction model to obtain a plurality of sampling video frames of different scales includes: down-sampling the to-be-processed video frame by the target multi-scale reconstruction model based on a first multiple to obtain a sampling video frame of a first scale; down-sampling the to-be-processed video frame by the target multi-scale reconstruction model based on a second multiple to obtain a sampling video frame of a second scale; and down-sampling the to-be-processed video frame by the target multi-scale reconstruction model based on a third multiple to obtain a sampling video frame of a third scale. The first multiple, the second multiple, and the third multiple are adjacent integer multiples and increase in order.

[0064] In this embodiment, the terminal can perform multiple down-sampling processing on each to-be-processed video frame in the to-be-processed video frame set through the target multi-scale reconstruction model. For example, for each to-be-processed video frame, the terminal can perform down-sampling of a first multiple on the to-be-processed video frame through the target multi-scale reconstruction model to obtain a sampling video frame of a first scale; the terminal can perform down-sampling of a second multiple on the to-be-processed video frame through the target multi-scale reconstruction model to obtain a sampling video frame of a second scale; and the terminal can also perform down-sampling of a third multiple on the to-be-processed video frame through the target multi-scale reconstruction model to obtain a sampling video frame of a third scale. The multiple values of the first multiple, the second multiple and the third multiple can increase in turn, and the multiple values of the first multiple, the second multiple and the third multiple can be in a non-multiple relationship, for example, can be adjacent integer multiples, such as the first multiple can be 1x, the second multiple can be 2x, and the third multiple can be 3x, and the like.

[0065] In the target multi-scale reconstruction model, a down-sampling convolutional network can also be included, and the terminal can perform down-sampling on the to-be-processed video frame through the down-sampling convolutional network in the model. Specifically, taking the scale of the to-be-processed video frame as 300*300 as an example, the terminal can perform down-sampling on the to-be-processed video frame at the original scale, down-sampling of 2 times the original scale, and down-sampling of 3 times the original scale through the target multi-scale reconstruction model, and the terminal can obtain three sampling video frames of 300*300, 150*150 and 100*100 scales, respectively, that is, the terminal can perform down-sampling on the to-be-processed video frame through the down-sampling convolutional network in the target multi-scale reconstruction model. In addition, it should be noted that the multiples of the above sampling can also be integer multiples of other values, such as combinations of 5x, 6x and 7x. The terminal can pass the three sampling video frames of different scales through the local optimization network in parallel, thereby extracting detailed information of different scales.

[0066] Through this embodiment, the terminal can perform down-sampling of multiple different scales on the original to-be-processed video frame, so that the terminal can perform video definition improvement processing based on the multiple sampling video frames obtained by down-sampling, thereby improving the comprehensiveness of video processing.

[0067] In an embodiment, the pixel feature information of each sampling video frame is obtained by a local optimization network, and the global dependency information of each sampling video frame is obtained by a global optimization network, including: high-frequency pixel information in the plurality of sampling video frames is extracted by a residual block network as the pixel feature information of the plurality of sampling video frames; self-similarity information corresponding to each sampling video frame is determined by an attention mechanism deep learning network, and global information corresponding to each sampling video frame is determined based on optical flow information in each sampling video frame and optical flow information in a preset number of adjacent sampling video frames; and the global dependency information of each sampling video frame is obtained according to the self-similarity information and the global information.

[0068] In the embodiment, the local optimization network is a Resblock network, the global optimization network is a transformer network, and the target multi-scale reconstruction model can be an encoder-decoder structure. The terminal can extract high-frequency pixel information in the plurality of sampling video frames as the pixel feature information of the plurality of sampling video frames by the residual block network in the target multi-scale reconstruction model, and the terminal can determine self-similarity information corresponding to each sampling video frame by the attention mechanism deep learning network in the target multi-scale reconstruction model, and determine global information corresponding to each sampling video frame based on optical flow information in each sampling video frame and optical flow information in a preset number of adjacent sampling video frames. The optical flow information refers to the motion change of the same object in different sampling video frames, such as the motion of an object in a video. By extracting the changes of images in the video frames of the video, the motion of the target can be determined. The self-similarity information can be the consistency information of each pixel in a single sampling video frame, and the global information can be the consistency information of pixels between two sampling video frames belonging to different to-be-processed video frames in a group of sampling video frames corresponding to the to-be-processed video frames. Thus, the terminal can obtain the global dependency information of each sampling video frame according to the self-similarity information and the global information.

[0069] Specifically, after the terminal performs downsampling on the above-mentioned to-be-processed video frames in three different scales, since the to-be-processed video frames can be multiple, the sampling video frames obtained by downsampling can be multiple, the terminal can group the obtained multiple sampling video frames according to the scales, and group the sampling video frames of the same scale into a group, so that multiple sets of sampling video frames corresponding to the multiple to-be-processed video frames can be obtained, and the terminal can pass the three sets of sampling video frame sets through the feature refinement branch of the above-mentioned same encoder-decoder structure in parallel to perform feature extraction, so as to obtain detailed information of different scales. Wherein, the detailed information can be the content contained in the pixels, and the terminal can separate the information of the picture by using some high and low frequency filters, wherein the detailed information is the part after removing the low frequency information, the low frequency information is the overall structure of the image, and the high frequency information is the detailed information of the image. Wherein, the transformer module is adopted between the encoder and the decoder, so that the global dependency information of the sequence features in the above-mentioned each set of sampling video frames can be extracted, and the global consistency and self-similarity of the video features are improved; in addition, the video sequence optical flow information of each ResBlock and transformer basic block of the encoder-decoder structure is fused, so that the continuity of the video stream is improved. Wherein, the global dependency information represents the dependency relationship of the receptive field of the convolution kernel to the range of all to-be-processed video frames in the above-mentioned input to-be-processed video frame set, that is, the global dependency relationship of the receptive field; the global consistency, also known as global information, the information of the image with global consistency is consistent in multiple frames, and the self-similarity represents the consistency within a single image frame, so that the image flow is smooth and natural.

[0070] Through the embodiment, the terminal can perform extraction of pixel feature information and adjustment of global consistency on the sampling video frames by using the model of the encoder-decoder structure containing the ResBlock and the transformer module, so as to realize the effect of improving the comprehensiveness of video processing.

[0071] In one embodiment, according to the pixel feature information, the global dependency information and the to-be-processed video frame, an adjusted target video frame set is output, including: performing scale restoration processing on each sampling video frame according to the pixel feature information and the global dependency information by the target multi-scale reconstruction model, to obtain multiple restored video frames; the size of the restored video frame is the same as that of the to-be-processed video frame; superimposing the multiple restored video frames and the to-be-processed video frame to obtain an adjusted target video frame, and outputting a target video frame set composed of multiple target video frames.

[0072] In this embodiment, the terminal obtains the pixel feature information and global dependency information of the sampling video frames of each scale through the target multi-scale reconstruction model, and can perform image reconstruction processing. The terminal can perform scale restoration processing on each sampling video frame according to the pixel feature information and global dependency information by using the target multi-scale reconstruction model, that is, restore it to the scale corresponding to the to-be-processed video frame, but with higher definition, so that the terminal can obtain a plurality of restored video frames, and the size of the restored video frame is the same as that of the to-be-processed video frame. The terminal can superimpose the plurality of restored video frames and the corresponding to-be-processed video frames in the target multi-scale reconstruction model, so that the terminal can obtain the adjusted target video frame, and obtain the target video frame set obtained by combining the plurality of target video frames output by the target multi-scale reconstruction model.

[0073] Specifically, the target multi-scale reconstruction model can further include a reconstruction module. The terminal can process the feature information according to the reconstruction module in the multi-scale reconstruction model, perform scale restoration processing on the feature internal monologue in the sampling video frame of different scales, and superimpose it into the initial to-be-processed video frame to obtain a high-quality high-definition video frame sequence and output. The multi-scale reconstruction model can further include an upsampling convolution module. The scale restoration can be achieved by upsampling convolution. The terminal performs upsampling convolution on the sampling video frame of different scales by using the upsampling convolution module in the multi-scale reconstruction model to obtain the restored video frame corresponding to the sampling video frame of different scales. The upsampling convolution can generate a detail map in the to-be-processed video frame. Since the restored video frame has the same size as the to-be-processed video frame, the terminal superimposes the restored video frame containing the detail map with the original to-be-processed video frame, which can supplement the missing detail information in the to-be-processed video frame, thereby obtaining a target video frame with higher definition.

[0074] Through this embodiment, the terminal can obtain a video frame containing detail information by using upsampling convolution processing, and obtain a video frame with higher definition by superimposing it with the original to-be-processed video frame, thereby improving the comprehensiveness of video processing.

[0075] In one embodiment, according to the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets, a processed target video is obtained, including: obtaining the time information of each target video frame in the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets; and combining each target video frame in the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets according to the order of the time information to obtain a processed target video.

[0076] In this embodiment, the terminal can input each target video frame set corresponding to each set of to-be-processed video frames of the to-be-processed video into the multi-scale reconstruction model, and the multi-scale reconstruction model can output each target video frame set corresponding to each set of video frames. Each target video frame in each target video frame set can have time information, which can be the timestamp of the target video frame in the to-be-processed video. Each target video frame set can include at least one target video frame arranged in time sequence. The terminal can combine each target video frame set, obtain the time information of each target video frame in each target video frame set, and combine the at least one target video frame according to the order of the time information, thereby obtaining the processed target video.

[0077] Specifically, the combined video can also be format-converted based on the input target video format to obtain corresponding video data. As shown in Figure 3 Figure 3 is a flowchart of the format conversion step in an embodiment. The terminal can assemble the at least one target video frame set with improved resolution, perform corresponding data conversion according to the target video format, obtain the target video, and send it to the corresponding receiving end. For example, to a user's mobile device for display.

[0078] Through this embodiment, the terminal can combine the at least one target video frame set to obtain a video with higher resolution, thereby improving the comprehensiveness of video processing.

[0079] In an embodiment, the method further includes: obtaining an initial video frame sample set and an adjusted video frame sample set corresponding to the initial video frame sample set; the initial video frame sample set includes at least one initial video frame sample; the adjusted video frame sample set includes at least one adjusted video frame sample; inputting the initial video frame sample set to the to-be-trained multi-scale reconstruction model, performing down-sampling processing on each initial video frame sample in the initial video frame sample set by the to-be-trained multi-scale reconstruction model to obtain a plurality of sampling video frame samples of different scales, obtaining pixel feature information of each sampling video frame sample through a local optimization network, obtaining global dependency information of each sampling video frame sample through a global optimization network, and outputting an adjusted video frame test set according to the pixel feature information, the global dependency information, and the initial video frame sample; the adjusted video frame test set includes at least one test video frame sample; adjusting the model parameters of the to-be-trained multi-scale reconstruction model according to the similarity between each test video frame sample in the video frame test set and each adjusted video frame sample in the adjusted video frame sample set until a target multi-scale reconstruction model is obtained when a preset training condition is met.

[0080] ​In this embodiment, the terminal can obtain the target multi-scale reconstruction model by training. The terminal can pre-set an initial video frame sample set and a corresponding adjusted video frame sample set. The initial video frame sample set can include at least one initial video frame sample, and the adjusted video frame sample set can include at least one adjusted video frame sample. The terminal can obtain a to-be-trained multi-scale reconstruction model, input the initial video frame sample set into the to-be-trained multi-scale reconstruction model, and perform downsampling processing on each initial video frame sample in the initial video frame sample set by the to-be-trained multi-scale reconstruction model to obtain a plurality of sampling video frame samples of different scales. The to-be-trained multi-scale reconstruction model can include a local optimization network and a global optimization network. The terminal obtains pixel feature information of each sampling video frame sample by the local optimization network in the model, and obtains global dependency information of each sampling video frame sample by the global optimization network in the model. Thus, the terminal outputs an adjusted video frame test set according to the pixel feature information, the global dependency information, and the initial video frame sample by the to-be-trained multi-scale reconstruction model. The adjusted video frame test set includes at least one test video frame sample. The terminal compares each test video frame sample in the video frame test set with each adjusted video frame sample in the adjusted video frame sample set for similarity, and obtains a corresponding similarity. Thus, the terminal can adjust model parameters of the to-be-trained multi-scale reconstruction model according to the similarity until a target multi-scale reconstruction model is obtained when a preset training condition is met.

[0081] Specifically, the training process can be as shown in Figure 4 Figure 4 a flowchart of the training steps in an embodiment. The initial video frame sample set can be a low-quality video frame sample set, and the adjusted video frame sample set can be a high-quality video frame sample set corresponding to the low-quality video frame sample set. The terminal can construct a training data set based on the two sets, and obtain training data by cleaning data in the training data set. The terminal can input the low-quality video frame sample set as input and the high-quality video frame sample set as label. The terminal can train the multi-scale reconstruction model of the encoder-decoder framework by the training data until the model accuracy reaches a preset threshold, for example, the similarity between each test video frame sample in the video frame test set and each adjusted video frame sample in the adjusted video frame sample set reaches a preset similarity threshold, and output a target multi-scale reconstruction model trained. The target multi-scale reconstruction model can be a quality improvement model for improving the clarity of the video.

[0082] ​Through the embodiment, the terminal can train a target multi-scale reconstruction model through the constructed training data, so that the terminal can perform video processing based on the target multi-scale reconstruction model, thereby improving the comprehensiveness of video processing.

[0083] In one embodiment, as Figure 5 shown, Figure 5 is a structural schematic diagram of a video processing system in one embodiment. It includes a data preprocessing module 2: used to collect original video related information, target video format, perform data cleaning, extract video frames, sequence grouping, so as to obtain a video sequence group available for a multi-scale reconstruction model. A video quality improvement module 3: this module constructs a multi-scale video frame group repair network, i.e. the above-mentioned multi-scale reconstruction model, which can effectively improve the quality of the video sequence. Specifically, it is used to perform 2x and 3x downsampling convolution on a group of 7 video frames, so as to obtain three groups of video frames of different scales; the three groups of video frames are processed in parallel through the feature refinement branches of the same encoder-decoder structure, so as to obtain detailed information of different scales; the transformer module is adopted between the encoder and the decoder to extract the global dependency information of the sequence features, thereby improving the global consistency and self-similarity of the video features; in addition, the video sequence optical flow information of each ResBlock and transformer basic block of the encoder-decoder structure is fused, thereby improving the continuity of the video stream; finally, the features of different scales are upsampled and convolved, superimposed with the original video frames, and the video sequence group is reconstructed. A result processing module 4: used to process the data output by the improvement module, assemble the video sequences in order, and perform appropriate video type conversion according to the target video type, and transmit the video data of the target format to the outside.

[0084] The specific process of the above-mentioned video processing can be as Figure 6 shown, Figure 6 is a flowchart of a video processing method in another embodiment. It includes the following steps: obtaining a group of preprocessed video sequence data, i.e. the above-mentioned set of video frames to be processed. Using a downsampling convolution module to process the data features in the video frames to be processed, to obtain three groups of video sequences of different scales. The three groups of sequences composed of video frames of different scales are processed in parallel, each group of data extracts corresponding optical flow information, and the data features are processed and optimized through the encoder-decoder structure with ResBlock module and transformer module and fused with the optical flow information, to improve the feature quality. The terminal uses the reconstruction module in the multi-scale reconstruction model to process the data features, restores the features of different scales, superimposes them on the initial video frames, and generates high-quality video sequence output.

[0085] Among them, Resblock is good at processing local information, that is, information within the receptive field, transformer captures global information, maintains overall consistency, and the whole is more smooth and natural. Therefore, the combination of the two can optimize the global and local features of the sampling video frame corresponding to the video frame to be processed. And the terminal does not completely stack the transformer module, but uses a small amount of transformer to obtain global information, and the detailed information is processed by the stacked resblock as a cnn structure. Resblock focuses on local information and has a small receptive field, so it occupies less resources and runs faster relatively. The skip connection of resblock also complements the information locally, but it does not have as much global information as transformer.

[0086] Through the above embodiment, the terminal processes multiple scale video frames in parallel by using a model composed of a local optimization network and a global optimization network, thereby improving the comprehensiveness of video quality improvement processing. And, combining the transformer structure with global attention mechanism and the ResBlock convolutional neural network with local information extraction capability, the global and local information can be combined to extract perfect structure and detail information. This method uses a small amount of transformer, and captures global dependence on processed data features rather than original image data, uses relatively less resources, and can limit the feature space, so as not to cause the generalization performance to decline due to insufficient data; and the feature space is expanded from single scale to custom multi-scale, which can extract multi-scale structure information, and the detail information is preserved through multiple global skip connections, better improving the structure and detail consistency of video sequence. And parallel processing of different scale picture sequences avoids the detail blur and error accumulation caused by multiple downsampling operations, while improving the parallelism of the network and the real-time performance of the lossless network.

[0087] It should be understood that although each step in the flowchart involved in each embodiment as described above is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps has no strict sequence limitation, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps.

[0088] Based on the same inventive concept, the embodiments of the present application also provide a video processing device for implementing the video processing method described above. The implementation scheme of the device for solving the problem is similar to the implementation scheme described in the above method, so the specific limitations in one or more video processing device embodiments provided below can refer to the limitations of the video processing method described above, which will not be repeated here.

[0089] In one embodiment, as shown in Figure 7 A video processing device is provided, comprising: an acquisition module 500, a processing module 502 and an output module 504, wherein:

[0090] The acquisition module 500 is configured to acquire a to-be-processed video and extract at least one set of to-be-processed video frames from the to-be-processed video. The set of to-be-processed video frames includes at least one to-be-processed video frame.

[0091] The processing module 502 is configured to input the set of to-be-processed video frames into a target multi-scale reconstruction model, perform down-sampling processing on each to-be-processed video frame in the set of to-be-processed video frames by the target multi-scale reconstruction model, obtain a plurality of sampled video frames of different scales, obtain pixel feature information of each sampled video frame by a local optimization network, obtain global dependency information of each sampled video frame by a global optimization network, and output an adjusted set of target video frames according to the pixel feature information, the global dependency information and the to-be-processed video frame. The target multi-scale reconstruction model includes the local optimization network and the global optimization network.

[0092] The output module 504 is configured to obtain a processed target video according to the set of target video frames corresponding to each of the at least one set of to-be-processed video frames.

[0093] In one embodiment, the acquisition module 500 is specifically configured to perform data cleaning on the to-be-processed video to obtain a data-cleaned to-be-processed video, segment the data-cleaned to-be-processed video according to a time sequence to obtain a plurality of to-be-processed video frames, combine the plurality of to-be-processed video frames into at least one set of to-be-processed video frames, and the number of to-be-processed video frames in each set of to-be-processed video frames is the same.

[0094] In one embodiment, the acquisition module 500 is specifically configured to perform at least one of abnormal frequency removal and impulse burr removal on the video spectrum of the to-be-processed video, and perform at least one of abnormal frequency removal, impulse burr removal and bottom voice noise removal on the audio spectrum of the to-be-processed video, to obtain a data-cleaned to-be-processed video.

[0095] In an embodiment, the processing module 502 described above is specifically configured to: perform downsampling on the to-be-processed video frame based on a first multiple by the target multi-scale reconstruction model to obtain a sampling video frame of a first scale; perform downsampling on the to-be-processed video frame based on a second multiple by the target multi-scale reconstruction model to obtain a sampling video frame of a second scale; perform downsampling on the to-be-processed video frame based on a third multiple by the target multi-scale reconstruction model to obtain a sampling video frame of a third scale; the first multiple, the second multiple and the third multiple are adjacent integer multiples and sequentially increase.

[0096] In an embodiment, the processing module 502 described above is specifically configured to: extract high-frequency pixel information in the plurality of sampling video frames as pixel feature information of the plurality of sampling video frames by the residual block network; determine self-similarity information corresponding to each sampling video frame by the attention mechanism deep learning network respectively, and determine global information corresponding to each sampling video frame based on optical flow information in each sampling video frame and optical flow information in a preset number of adjacent sampling video frames; and obtain global dependency information of each sampling video frame according to the self-similarity information and the global information.

[0097] In an embodiment, the processing module 502 described above is specifically configured to: perform scale restoration processing on each sampling video frame according to the pixel feature information and the global dependency information by the target multi-scale reconstruction model to obtain a plurality of restored video frames; the size of the restored video frame is the same as that of the to-be-processed video frame; superimpose the plurality of restored video frames and the to-be-processed video frame to obtain an adjusted target video frame, and output a target video frame set composed of a plurality of target video frames.

[0098] In an embodiment, the output module 504 described above is specifically configured to: obtain time information of each target video frame in each target video frame set corresponding to each set of to-be-processed video frames; and combine each target video frame in each target video frame set corresponding to each set of to-be-processed video frames according to the order of the time information to obtain a processed target video.

[0099] In an embodiment, the apparatus further includes a training module configured to obtain an initial video frame sample set and an adjusted video frame sample set corresponding to the initial video frame sample set, the initial video frame sample set including at least one initial video frame sample, the adjusted video frame sample set including at least one adjusted video frame sample, input the initial video frame sample set into the multi-scale reconstruction model to be trained, perform down-sampling on each initial video frame sample in the initial video frame sample set by the multi-scale reconstruction model to be trained to obtain a plurality of sample video frame samples of different scales, obtain pixel feature information of each sample video frame sample by a local optimization network, obtain global dependency information of each sample video frame sample by a global optimization network, and output an adjusted video frame test set according to the pixel feature information, the global dependency information, and the initial video frame sample, the adjusted video frame test set including at least one test video frame sample, adjust model parameters of the multi-scale reconstruction model to be trained according to a similarity between each test video frame sample in the video frame test set and each adjusted video frame sample in the adjusted video frame sample set until a target multi-scale reconstruction model is obtained when a preset training condition is met.

[0100] Each module in the video processing apparatus can be implemented wholly or partially by software, hardware, and combinations thereof. Each module can be embedded in or independent of a processor in the computer device in hardware form, or stored in a memory in the computer device in software form, so as to be called and executed by the processor to perform operations corresponding to each module.

[0101] In an embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in FIG. 1. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, mobile cellular network, NFC (near field communication), or other technologies. The computer program is executed by the processor to implement a video processing method. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the shell of the computer device. The input device can also be an external keyboard, touchpad, or mouse, etc. Figure 8 In an embodiment, a computer device is provided, which can be a terminal, and an internal structure diagram of the computer device can be as shown in FIG. 1. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved by WIFI, mobile cellular network, NFC (near field communication), or other technologies. The computer program is executed by the processor to implement a video processing method. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the shell of the computer device. The input device can also be an external keyboard, touchpad, or mouse, etc.

[0102] Those skilled in the art can understand that Figure 8 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0103] In one embodiment, a computer device is provided, including a memory and a processor, the memory stores a computer program, and the processor implements the video processing method described above when executing the computer program.

[0104] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the video processing method described above.

[0105] In one embodiment, a computer program product is provided, including a computer program, and the computer program is executed by a processor to implement the video processing method described above.

[0106] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0108] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

[0109] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method of video processing, the method comprising: The method comprises: acquiring a to-be-processed video, and extracting at least one set of to-be-processed video frames in the to-be-processed video; the set of to-be-processed video frames comprises at least one to-be-processed video frame; inputting the set of to-be-processed video frames into a target multi-scale reconstruction model, performing different-scale downsampling processing on each to-be-processed video frame in the set of to-be-processed video frames by the target multi-scale reconstruction model, obtaining a plurality of sampling video frames of different scales, grouping the plurality of sampling video frames according to scales, and obtaining pixel feature information of each sampling video frame by a local optimization network and global dependency information of each sampling video frame by a global optimization network; the target multi-scale reconstruction model comprises the local optimization network and the global optimization network, and the global optimization network is used to improve self-similarity in a single video frame and global consistency between a plurality of continuous video frames; performing scale restoration processing on the sampling video frames according to the pixel feature information and the global dependency information by the target multi-scale reconstruction model, respectively, to obtain a plurality of restored video frames; the size of the restored video frames is the same as that of the to-be-processed video frames; superimposing the plurality of restored video frames and the to-be-processed video frames to obtain adjusted target video frames, and outputting a set of target video frames combined by the target video frames; obtaining a processed target video according to the set of target video frames corresponding to each of the at least one set of to-be-processed video frames.

2. The method of claim 1, wherein, The extraction of the at least one set of to-be-processed video frames in the to-be-processed video comprises: performing data cleaning on the to-be-processed video to obtain a data-cleaned to-be-processed video; segmenting the data-cleaned to-be-processed video according to a time sequence to obtain a plurality of to-be-processed video frames; combining the plurality of to-be-processed video frames into at least one set of to-be-processed video frames; the number of to-be-processed video frames in each set of to-be-processed video frames is the same.

3. The method of claim 2, wherein, The data cleaning of the to-be-processed video to obtain the data-cleaned to-be-processed video comprises: performing at least one of abnormal frequency removal and impulse burr removal on a video spectrum of the to-be-processed video, and performing at least one of abnormal frequency removal, impulse burr removal and bottom noise removal on an audio spectrum of the to-be-processed video, to obtain the data-cleaned to-be-processed video.

4. The method of claim 1, wherein, The downsampling processing of each to-be-processed video frame in the set of to-be-processed video frames by the target multi-scale reconstruction model to obtain a plurality of sampling video frames of different scales comprises: performing downsampling on the to-be-processed video frame by the target multi-scale reconstruction model based on a first multiple to obtain a sampling video frame of a first scale; performing downsampling on the to-be-processed video frame by the target multi-scale reconstruction model based on a second multiple to obtain a sampling video frame of a second scale; performing downsampling on the to-be-processed video frame by the target multi-scale reconstruction model based on a third multiple to obtain a sampling video frame of a third scale; the first multiple, the second multiple and the third multiple are adjacent integer multiples and increase sequentially.

5. The method of claim 1, wherein, The local optimization network is a residual block network, and the global optimization network is an attention mechanism deep learning network; The pixel feature information of each sampling video frame is obtained through the local optimization network, and the global dependency information of each sampling video frame is obtained through the global optimization network, including: High-frequency pixel information in the plurality of sampling video frames is extracted through the residual block network as the pixel feature information of the plurality of sampling video frames; The self-similarity information corresponding to each sampling video frame is determined through the attention mechanism deep learning network, and the global information corresponding to each sampling video frame is determined based on the optical flow information in each sampling video frame and the optical flow information in a preset number of adjacent sampling video frames; The global dependency information of each sampling video frame is obtained according to the self-similarity information and the global information.

6. The method of claim 2, wherein, The target video after processing is obtained according to the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets, including: Obtaining the time information of each target video frame in the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets; Combining each target video frame in the target video frame set corresponding to each of the at least one set of to-be-processed video frame sets according to the order of the time information to obtain the target video after processing.

7. The method of claim 1, wherein, The method further includes: Obtaining an initial video frame sample set and an adjusted video frame sample set corresponding to the initial video frame sample set; the initial video frame sample set includes at least one initial video frame sample; the adjusted video frame sample set includes at least one adjusted video frame sample; Inputting the initial video frame sample set into a to-be-trained multi-scale reconstruction model, performing down-sampling processing on each initial video frame sample in the initial video frame sample set by the to-be-trained multi-scale reconstruction model to obtain a plurality of sampling video frame samples of different scales, obtaining pixel feature information of each sampling video frame sample through a local optimization network, obtaining global dependency information of each sampling video frame sample through a global optimization network, and outputting an adjusted video frame test set according to the pixel feature information, the global dependency information, and the initial video frame sample; the adjusted video frame test set includes at least one test video frame sample; Adjusting the model parameters of the to-be-trained multi-scale reconstruction model according to the similarity between each test video frame sample in the video frame test set and each adjusted video frame sample in the adjusted video frame sample set until a target multi-scale reconstruction model is obtained when a preset training condition is met.

8. A video processing apparatus, comprising: The device includes: An acquisition module is configured to acquire a to-be-processed video and extract at least one set of to-be-processed video frame sets from the to-be-processed video; the set of to-be-processed video frame sets includes at least one to-be-processed video frame. The processing module is configured to input the set of video frames to be processed into a target multi-scale reconstruction model, perform different scale down-sampling processing on each video frame to be processed in the set of video frames to be processed respectively by the target multi-scale reconstruction model, obtain a plurality of sampled video frames of different scales, group the plurality of sampled video frames of different scales according to scales, obtain pixel feature information of each sampled video frame by a local optimization network, and obtain global dependency information of each sampled video frame by a global optimization network. The target multi-scale reconstruction model includes the local optimization network and the global optimization network. The global optimization network is configured to improve self-similarity in a single video frame and global consistency between a plurality of continuous video frames. The target multi-scale reconstruction model performs scale restoration processing on each sampled video frame according to the pixel feature information and the global dependency information, respectively, to obtain a plurality of restored video frames. The size of the restored video frames is the same as that of the video frames to be processed. The plurality of restored video frames and the video frames to be processed are superimposed to obtain adjusted target video frames, and a set of target video frames obtained by combining a plurality of target video frames is output. The output module is configured to obtain target video processed according to the set of target video frames corresponding to each of the at least one set of video frames to be processed. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

11. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 7.

Images