A model training method and system for video super-resolution reconstruction

Abstract

The application relates to a model training method and system for video super-resolution reconstruction. The method comprises: preprocessing a video sample set to obtain a training data set, wherein the training data set comprises a plurality of low-resolution frames and a real high-resolution frame corresponding to each low-resolution frame; performing super-resolution reconstruction on each low-resolution frame according to a convolutional neural network constructed based on three stages of feature extraction, inter-frame fusion and super-resolution enhancement to obtain a predicted high-resolution frame in the current round; and performing iterative updating on the convolutional neural network according to the difference between the predicted high-resolution frame in each round and the corresponding real high-resolution frame to obtain a trained video reconstruction model. The method can form a video super-resolution reconstruction process with the synergistic effect of feature extraction, inter-frame fusion and super-resolution enhancement, so that the video super-resolution reconstruction process has high efficiency and accuracy at the current training level and the subsequent application level.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a model training method and system for video super-resolution reconstruction. Background Technology

[0002] In the field of video processing technology, this involves super-resolution upscaling and detail enhancement of existing videos to achieve super-resolution reconstruction of the original videos.

[0003] Among the relevant video super-resolution reconstruction methods, on the one hand, there are traditional interpolation amplification techniques such as bilinear interpolation, bicubic interpolation, and Lanczos interpolation, but these can only achieve pixel-level filling and cannot generate real details at the feature level. On the other hand, there are early single-frame deep learning super-resolution techniques such as SRCNN, VDSR, and ESPCN, but these all focus on single-frame processing and do not consider the correlation between video frames. Furthermore, there are video-specific super-resolution techniques such as optical flow-based video super-resolution algorithms and early video CNN models, but these have poor adaptability to complex scenes such as low light, noise pollution, and low texture, and lack matching super-resolution enhancement processing. Summary of the Invention

[0004] Therefore, it is necessary to provide a model training method, system, computer device, and computer-readable storage medium for video super-resolution reconstruction to address the aforementioned technical problems.

[0005] Firstly, this application provides a model training method for video super-resolution reconstruction, comprising: A training dataset is obtained by preprocessing a preset video sample set. The training dataset includes multiple low-resolution frames and the real high-resolution frames corresponding to each low-resolution frame. Based on the convolutional neural network pre-constructed in three stages—feature extraction, inter-frame fusion, and super-resolution enhancement—super-resolution reconstruction is performed on each low-resolution frame to obtain the predicted high-resolution frame of the convolutional neural network in the current round. Based on the difference between the predicted high-resolution frames and the corresponding real high-resolution frames in each round, the convolutional neural network is iteratively updated, and the convolutional neural network that meets the preset evaluation conditions is used as a trained video reconstruction model for super-resolution reconstruction of any video.

[0006] Secondly, this application also provides a model training system for video super-resolution reconstruction, comprising: The acquisition module is used to preprocess a preset video sample set to obtain a training dataset, which includes multiple low-resolution frames and the real high-resolution frames corresponding to each low-resolution frame. The training module is used to perform super-resolution reconstruction on each low-resolution frame based on a convolutional neural network constructed in advance based on three stages: feature extraction, inter-frame fusion, and super-resolution enhancement, so as to obtain the predicted high-resolution frame of the convolutional neural network in the current round. The update module is used to iteratively update the convolutional neural network based on the difference between the predicted high-resolution frames and the corresponding real high-resolution frames in each round, and to use the convolutional neural network that meets the preset evaluation conditions as a trained video reconstruction model for super-resolution reconstruction of any video.

[0007] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the above steps.

[0008] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the above steps.

[0009] The aforementioned model training method, system, computer equipment, and computer-readable storage medium for video super-resolution reconstruction firstly preprocesses the video sample set to obtain a training dataset containing each pair of low-resolution frames, thus providing an input and comparison basis with a clear temporal alignment relationship for subsequent network processing. Secondly, a convolutional neural network constructed sequentially from three stages—feature extraction, inter-frame fusion, and super-resolution enhancement—processes the low-resolution frames step by step, thereby extracting information from the low-resolution frames, fusing information between adjacent frames, and generating a predicted high-resolution frame that is spatially consistent with the real high-resolution frame. Thirdly, based on the predicted high-resolution... The difference between the frame and the real high-resolution frame iteratively updates the convolutional neural network and outputs the trained model when the evaluation conditions are met, so that the model parameter update has a clear basis and ensures that the output model reaches a usable training completion state. Based on this, in the whole technical solution, intra-frame information is extracted at the feature level to generate real details, inter-frame information is fused in the temporal dimension to consider the inter-frame correlation of video, and high-resolution output is generated on this basis to present size restoration and detail enhancement. This forms a video super-resolution reconstruction process with the synergistic effect of feature extraction, inter-frame fusion and super-resolution enhancement, so that the video super-resolution reconstruction processing is efficient and accurate at the current training level and the subsequent application level. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a model training method for video super-resolution reconstruction in one embodiment. Figure 2 This is an architecture diagram of a video reconstruction model during the deployment phase in one embodiment; Figure 3 This is a block diagram of a model training system for video super-resolution reconstruction in one embodiment. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] In one embodiment, such as Figure 1 As shown, a model training method for video super-resolution reconstruction is provided. This embodiment illustrates the method by applying it to a server. It is understood that the method can also be applied to a terminal, or to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps S101 to S103.

[0014] Step S101: Preprocess the preset video sample set to obtain the training dataset. The training dataset includes multiple low-resolution frames and the corresponding real high-resolution frames for each low-resolution frame.

[0015] The video sample set refers to the set of video data used for model training, which provides video content that has a temporal continuity and a resolution correspondence. For example, it includes multiple different video contents, and for the same video content, it includes both low-resolution video streams and high-resolution video streams.

[0016] Low-resolution frames refer to video frames extracted in chronological order from low-resolution video streams in the video sample set. Low resolution means that the horizontal and vertical pixel counts of the video frame are at a low level, such as video frames with a resolution of no more than 1280×720. This type of resolution corresponds to video content in media platforms that is identified as high-definition or lower resolution.

[0017] Among them, true high-resolution frames refer to video frames extracted in chronological order from high-resolution video streams in the video sample set. High resolution means that the number of horizontal and vertical pixels of the video frame is higher than that of the low-resolution frame corresponding to the same frame content. For example, video frames with resolutions of 1920×1080, 2560×1440 or 3840×2160. This type of resolution corresponds to video content in media platforms that is identified as Full HD, Ultra HD or higher resolution.

[0018] For example, a pre-defined video sample set is uniformly organized to transform the original video content into a data format suitable for subsequent model training. This involves establishing a clear and stable correspondence between low-resolution frames and true high-resolution frames while maintaining temporal consistency of the video content. Specifically, for each video stream in the video sample set, the stream is split frame-by-frame according to its temporal order, converting continuous video data into a set of time-arranged video frames, facilitating subsequent frame-by-frame processing. After frame-by-frame splitting, in a low-resolution video stream and a high-resolution video stream corresponding to the same video content, the video frames in the low-resolution video stream are designated as low-resolution frames, and the video frames in the high-resolution video stream at the same time position are designated as true high-resolution frames, ensuring that each video frame in both types of video streams maintains temporal consistency.

[0019] Furthermore, within the same video content, low-resolution frames corresponding to the same time position and real high-resolution frames maintain consistency in imaging environment conditions and image motion states. That is, the two strictly correspond in the time dimension, reflecting the scene content at the same moment. This ensures that the difference between the two is only reflected at the resolution level and is not affected by factors such as changes in lighting or motion. Different video content covers different imaging environment conditions and image motion states to ensure that the model training process maintains high accuracy in complex scenarios such as low light and fast motion, taking into account both real-time performance and robustness.

[0020] Furthermore, the low-resolution frames and the real high-resolution frames are standardized in terms of data format, spatial size organization, and storage structure to ensure they meet the consistency requirements of the input data in subsequent networks. Through this process, the video sample set is transformed into a training dataset containing multiple low-resolution frames and their one-to-one corresponding real high-resolution frames, providing a clear data foundation for comparing the prediction results with the reference results during subsequent training.

[0021] Step S102: Based on the convolutional neural network pre-constructed in three stages—feature extraction, inter-frame fusion, and super-resolution enhancement—super-resolution reconstruction is performed on each low-resolution frame to obtain the high-resolution frame predicted by the convolutional neural network in the current round.

[0022] Convolutional neural networks (CNNs) are network models that are structurally divided into multiple stages according to the data processing flow. They are used to process low-resolution input frames step by step to generate predicted high-resolution frames.

[0023] The predicted high-resolution frame refers to the output frame data generated by the convolutional neural network based on a low-resolution frame in a certain training round. The predicted high-resolution frame is consistent with the real high-resolution frame corresponding to the low-resolution frame in terms of spatial resolution, so as to be used for difference comparison with the real high-resolution frame.

[0024] For example, based on a pre-built convolutional neural network, super-resolution reconstruction is performed on low-resolution frames in the training dataset. This involves a phased unfolding of the processing of low-resolution frames within the network to obtain the predicted high-resolution frame corresponding to the current training epoch. This ensures that the information of the input frames is fully utilized and transformed into a high-resolution form according to a predetermined structural order. Specifically, low-resolution frames are input into the feature extraction stage of the convolutional neural network. In this stage, features are extracted from the image information contained in the low-resolution frames, transforming the original frame data into a feature representation for subsequent processing, thus providing a basis for feature alignment between different frames. After feature extraction, the network enters the inter-frame fusion stage. In this stage, based on the temporal correlation of the low-resolution frames, the feature representations corresponding to multiple frames are fused, enabling the reconstruction process of the current frame to unfold based on information within a continuous time range, ensuring the consistency of the reconstruction input in the temporal dimension. After completing the inter-frame fusion, the network enters the super-resolution enhancement stage. In this stage, the aforementioned processing results are further super-resolution reconstructed and enhanced to reconstruct them in a spatial size consistent with the real high-resolution frame. In terms of details, the texture structure and edge information in the reconstructed results are enhanced and presented, thereby forming the corresponding output frame.

[0025] Through the above sequential processing flow, each low-resolution frame can be mapped to the corresponding predicted high-resolution frame under the same network structure, and the resulting predicted high-resolution frame meets the requirements for subsequent comparison with the real high-resolution frame at the data structure and resolution levels, thereby providing clear input data for subsequent difference comparison and iterative updates based on the prediction results and the real results.

[0026] Step S103: Based on the difference between the predicted high-resolution frames and the corresponding real high-resolution frames in each round, the convolutional neural network is iteratively updated, and the convolutional neural network that meets the preset evaluation conditions is used as the trained video reconstruction model for super-resolution reconstruction of any video.

[0027] The evaluation criteria refer to the criteria used to determine the training status of the convolutional neural network, in order to determine whether to terminate the iterative update of the convolutional neural network and output it as a trained model. For example, the criteria are used to determine whether the reconstruction accuracy of the verification dataset, the temporal coherence between adjacent frames, and the detail consistency of the reconstruction results reach a preset threshold, so as to determine whether the convolutional neural network meets the training completion requirements.

[0028] For example, after obtaining the predicted high-resolution frame output by the convolutional neural network in the current training epoch, the predicted high-resolution frame is compared with the corresponding real high-resolution frame in time position, which serves as the basis for iteratively updating the convolutional neural network. Specifically, the predicted high-resolution frame and the real high-resolution frame are matched one by one, and the differences between them at the pixel level are sorted out to form a difference result reflecting the current training epoch. Subsequently, based on the difference result, the parameter state inside the convolutional neural network is updated, so that the network can adjust according to the previous differences when processing low-resolution frames in subsequent training epochs.

[0029] The above process is executed cyclically in multiple training rounds. After each training round, the training state of the convolutional neural network is uniformly evaluated. When the network output meets the preset evaluation conditions on the validation dataset, the iterative update process is terminated, and the convolutional neural network with the corresponding parameter state is determined as the trained video reconstruction model. This ensures that the model training is completed based on the continuous comparison between the predicted results and the actual results, and ensures that the final model can be used for subsequent super-resolution reconstruction processing of actual videos.

[0030] Optionally, based on the differences between each predicted high-resolution frame and the corresponding real high-resolution frame, the network parameters are iteratively optimized through backpropagation until the super-resolution accuracy (PSNR, Peak Signal-to-Noise Ratio) of the validation dataset reaches 38dB, the temporal consistency score (TCS) reaches 90 points, and the detail fidelity score (DFS) reaches 92 points. Based on this, the convolutional neural network is used as the trained video reconstruction model. The trained video reconstruction model can quickly infer low-resolution video at a rate of 30ms / frame and output high-resolution video, achieving 4K resolution upscaling, realistic restoration of texture details, and smooth inter-frame continuity. After testing, the target recognition accuracy is improved by 40% and the image quality score is improved by 25% after the video resolution is upscaled, making it suitable for core fields such as live streaming, intelligent monitoring, short video creation, and film and television post-production.

[0031] In the above-mentioned model training method for video super-resolution reconstruction, in step S101, the video sample set is preprocessed to obtain a training dataset containing each pair of low-resolution frames, thus providing an input and comparison basis with a clear temporal alignment relationship for subsequent network processing; in step S102, the low-resolution frames are processed step by step according to the convolutional neural network constructed sequentially by the three stages of feature extraction, inter-frame fusion, and super-resolution enhancement, so that information in the low-resolution frames can be extracted, information between adjacent frames can be fused, and a predicted high-resolution frame with the same spatial size as the real high-resolution frame can be generated on this basis; in step S103, the predicted high-resolution frame is processed according to the predicted high-resolution frame. The difference between the frame and the real high-resolution frame iteratively updates the convolutional neural network and outputs the trained model when the evaluation conditions are met, so that the model parameter update has a clear basis and ensures that the output model reaches a usable training completion state. Based on this, in the whole technical solution, intra-frame information is extracted at the feature level to generate real details, inter-frame information is fused in the temporal dimension to consider the inter-frame correlation of video, and high-resolution output is generated on this basis to present size restoration and detail enhancement. This forms a video super-resolution reconstruction process with the synergistic effect of feature extraction, inter-frame fusion and super-resolution enhancement, so that the video super-resolution reconstruction processing is efficient and accurate at the current training level and the subsequent application level.

[0032] In an exemplary embodiment, a preset video sample set is preprocessed to obtain a training dataset, including steps S201 to S204.

[0033] Step S201: Extract the initial frame sequence corresponding to each video sample in the video sample set, and perform noise filtering on each initial frame sequence to obtain the first frame sequence corresponding to each initial frame sequence.

[0034] For example, frame-level parsing is performed on each video sample in the video sample set to convert the continuous original video content into an initial frame sequence arranged in chronological order. This process uses the time-coding information of the video samples as a basis to extract the scene content frame by frame from the original video content, ensuring that each frame corresponds to a specific time position, thereby guaranteeing that the obtained initial frame sequence can completely reflect the temporal structure of the original video content. After completing the frame-level extraction, in order to reduce random interference factors introduced during the acquisition, compression, or transmission of the original video content, noise filtering is performed on each video frame in the initial frame sequence; that is, the noise filtering process revolves around suppressing unstructured pixel fluctuations. By uniformly organizing the pixel distribution within the frame, interference information unrelated to the video content is reduced, while maintaining the continuity of the main structural information in the frame.

[0035] For example, when filtering noise in the initial frame sequence, the pixel distribution of each frame is locally smoothed based on the scattered pixel fluctuations caused by video acquisition or compression. This suppresses abnormal jumps between adjacent pixels while maintaining the continuous distribution of contour and texture regions in the image. By performing this noise filtering process frame by frame, random interference components in the frame image are reduced, while the original structural form of the video content is continuously preserved in time.

[0036] Based on this, by combining frame-level extraction with noise filtering, the original video samples are converted into a first frame sequence with controlled noise levels and clear temporal relationships. This first frame sequence has a more stable intra-frame data state while maintaining the basic structure of the video content.

[0037] Step S202: Based on the difference in light intensity between each first frame sequence, perform light normalization on each first frame sequence frame by frame to obtain the second frame sequence corresponding to each first frame sequence.

[0038] For example, after obtaining the first frame sequence corresponding to each video sample, to address the issue of inconsistent illumination intensity between different first frame sequences due to differences in shooting environment, acquisition conditions, or content scene, illumination normalization processing is performed on each first frame sequence to ensure that subsequent processing is based on a relatively consistent brightness foundation. Specifically, the overall brightness distribution of the frame images in each first frame sequence is analyzed and organized to determine the differences in brightness intensity among different first frame sequences, and this is used as the basis for performing normalization processing. Subsequently, the brightness of the video frames in each first frame sequence is adjusted in a frame-by-frame processing manner to ensure that the brightness changes in the time dimension within the same frame sequence remain continuous, while simultaneously making the brightness levels between different frame sequences tend to be consistent.

[0039] For example, when processing the first frame sequence, the brightness value distribution of each frame image is first statistically analyzed to obtain the brightness distribution characteristics of each first frame sequence in terms of overall brightness level. The brightness distribution characteristics of different first frame sequences are then compared to determine the differences in brightness intensity. Subsequently, during frame-by-frame processing, the brightness values ​​of each frame image are adjusted accordingly based on the above differences, so that the brightness changes of adjacent frames in the same first frame sequence remain smooth and continuous, while ensuring that different first frame sequences are within a relatively consistent range in terms of overall brightness level.

[0040] Based on this, the first frame sequence, which originally had differences in illumination intensity, was organized into a second frame sequence, so that each second frame sequence had uniformity in brightness distribution, thereby reducing the interference of illumination differences on the subsequent high-resolution and low-resolution frame pair annotation process and model training process.

[0041] Step S203: Each frame of the second frame sequence corresponding to the same video content and corresponding to high resolution and low resolution respectively is labeled with real high-resolution frames and low-resolution frames to obtain the corresponding high-low resolution frame pairs.

[0042] For example, after noise filtering and illumination normalization are performed on each video sample in the video sample set, the second frame sequences corresponding to the same video content and originating from high-resolution and low-resolution video streams respectively are used as processing objects to establish a clear and stable correspondence between real high-resolution frames and low-resolution frames. Specifically, based on the consistency of video content, high-resolution and low-resolution second frame sequences belonging to the same video content are matched and organized so that the two sets of second frame sequences are under the same reference system in the time dimension. On this basis, according to the time order of video frames in their respective second frame sequences, frame-by-frame alignment processing is performed on the two sets of second frame sequences, that is, video frames at the same time position are associated and organized as a group of corresponding frames, thereby ensuring that no temporal misalignment is introduced in the subsequent annotation process.

[0043] After frame-by-frame alignment is completed, the annotation process of real high-resolution frames and low-resolution frames is performed on each group of corresponding frames, so that the video frames in the high-resolution video stream are clearly identified as real high-resolution frames, and the video frames in the corresponding low-resolution video stream are identified as low-resolution frames.

[0044] Based on this, by performing the above frame-by-frame annotation process on all second-frame sequences, the originally independent high-resolution second-frame sequences and low-resolution second-frame sequences are organized into a set of high-low resolution frame pairs with clear structure and clear correspondence, so that each low-resolution frame has a clear reference frame source, thereby providing basic data units for the construction of subsequent training datasets.

[0045] Step S204: Perform data augmentation on each high-low resolution frame pair to obtain multiple extended high-low resolution frame pairs, and combine all high-low resolution frame pairs with all extended high-low resolution frame pairs to form a training dataset.

[0046] For example, after obtaining the high- and low-resolution frame pairs formed by the previous steps, data augmentation processing is performed on each high- and low-resolution frame pair to expand the scale of the training data and enrich the distribution of frame pairs. Specifically, each high- and low-resolution frame pair is used as a basic processing unit. While maintaining the correspondence between the real high-resolution frames and low-resolution frames, the frame pair data is standardized to ensure that the temporal position and resolution correspondence of each frame pair remains stable before and after the data augmentation processing. Based on this, expansion processing is performed synchronously on each high- and low-resolution frame pair around the spatial arrangement and value distribution of the frame images, so that the same frame pair data forms multiple expanded high- and low-resolution frame pairs in different data formats, thereby increasing the diversity of frame pair samples in the training data. In addition, this processing is performed on the frame pair data as a whole, rather than processing individual frames separately, thereby avoiding the disruption of the pairing relationship between high- and low-resolution frames.

[0047] For example, when processing a high-low resolution frame pair, the spatial dimensions and pixel arrangement of the two frames are first standardized, treating the frame pair as a whole. This ensures that the actual high-resolution frame and the low-resolution frame maintain consistency in the cropped area and arrangement position, thereby guaranteeing the stability of the correspondence within the frame pair data. After standardization, the high-low resolution frame pair is simultaneously extended. This involves applying the same spatial transformation and value adjustment to both frames in the frame pair data, ensuring that the temporal and resolution correspondences of the frame pair data remain unchanged before and after the extension process. This results in multiple extended high-low resolution frame pairs that maintain consistent correspondence.

[0048] After data augmentation, the original high-low resolution frame pairs and all extended high-low resolution frame pairs obtained through the above processing are combined according to a preset data organization method. For example, the high-low resolution frame pairs are categorized and organized according to video content categories. Within the same video content category, the original high-low resolution frame pairs and extended high-low resolution frame pairs are further distinguished and organized according to the data source of the frame pairs, thus forming a complete training dataset. This allows the training dataset to maintain the original frame pair correspondence while possessing a more sufficient amount of data and a more varied data distribution, thereby meeting the requirements for data scale and data organization consistency in the subsequent model training stage.

[0049] In this embodiment, in step S201, the video samples are analyzed at the frame level and noise filtering is performed frame by frame to form a first frame sequence, thereby reducing the impact of random interference on intra-frame data while maintaining the temporal order of the training data. In step S202, the first frame sequences are normalized frame by frame according to the differences in illumination intensity between different first frame sequences to form a second frame sequence, thereby ensuring that the training data has a consistent data basis in terms of brightness intensity and maintaining the continuity within the frame sequence. In step S203, the high-resolution second frame sequence and the low-resolution second frame sequence under the same video content are labeled frame by frame, thereby establishing a clear and stable correspondence between real high-resolution frames and low-resolution frames. In step S204, data augmentation processing is performed using frame pairs as the basic processing unit, thereby expanding the scale of the training data without destroying the frame pair correspondence. Based on this, in the entire technical solution, a training dataset with a stable structure and clear correspondence is formed through a data construction process that features controlled noise, unified illumination, temporal alignment, and consistent expansion.

[0050] In an exemplary embodiment, before extracting the initial frame sequence corresponding to each video sample in the video sample set, the method further includes step S301.

[0051] Step S301: Based on the sampling scene features of the current video sample, determine the preprocessing strategy matched to the current video sample in the preset scene classifier, so as to perform noise filtering, illumination normalization and data augmentation processing on the current video sample.

[0052] Among them, the sampling scene features represent the characteristic information used to characterize the environment and content status of the current video sample during the acquisition stage, such as the characteristic performance of the video sample in terms of overall brightness level, image stability and content complexity.

[0053] The scene classifier refers to a classification model used to identify and classify the sampling scene to which a video sample belongs based on the sampling scene features of the video sample. For example, a rule-based classification model or a learning-based classification model can be used to comprehensively determine the sampling scene features and output the corresponding sampling scene category. The preprocessing strategy refers to the pre-configured processing scheme for different sampling scenes, which guides how to perform noise filtering, illumination normalization and data augmentation processing on video samples in a specified sampling scene.

[0054] For example, by analyzing the overall image performance of the current video sample, feature information reflecting its brightness distribution, image stability, and content complexity is obtained. This feature information is then used as the basis for characterizing the sampling scene, thereby forming the sampling scene features of the current video sample. For instance, by analyzing the concentration of brightness values ​​in each frame of the current video sample, the magnitude of image changes between adjacent frames, and the richness of structural information in the image, a description of the sampling scene features of the current video sample is formed.

[0055] After obtaining the sampling scene features of the current video sample, these features are input into a pre-defined scene classifier. The scene classifier then determines the sampling scene category to which the current video sample belongs based on its internally established scene determination rules. For example, based on the feature performance of video samples from multiple historical time periods under their respective sampling scenes, different sampling scenes are summarized and organized. Corresponding scene determination rules are then established based on this, enabling the scene classifier to determine the sampling scene category to which the current video sample belongs based on the matching relationship between the sampling scene features of the current video sample and these scene determination rules.

[0056] After determining the sampling scene category to which the current video sample belongs, the scene classifier selects a matching preprocessing strategy from a variety of pre-configured preprocessing strategies to determine the processing rules for subsequent noise filtering, illumination normalization, and data augmentation of the current video sample. These rules include the processing sequence, processing intensity, and corresponding parameter configurations.

[0057] Based on this, the above processing flow enables the identification of the sampling scene and the matching of the preprocessing method to be completed before the video samples enter the specific preprocessing steps. This ensures that the subsequent preprocessing process is carried out in an orderly manner under the constraints of a unified strategy, and enables video samples under different sampling scenes to be adaptively preprocessed according to their respective preprocessing strategies. This provides a stable and consistent processing foundation for the subsequent frame sequence construction and training dataset generation.

[0058] In this embodiment, in step S301, the sampling scene features of the current video sample are analyzed, so that the sampling scene category of the video sample is determined before entering the specific preprocessing. Then, a matching preprocessing strategy is selected from a variety of pre-configured preprocessing strategies according to the determined sampling scene category, thereby providing a reasonable processing method for subsequent preprocessing. Based on this, in the whole technical solution, by introducing a feature classification and strategy matching mechanism in the preprocessing stage, video samples under different sampling scenes can be adaptively processed according to the corresponding preprocessing strategy, providing a stable and consistent data foundation for the construction of the subsequent training dataset.

[0059] In an exemplary embodiment, based on a convolutional neural network pre-constructed using three stages—feature extraction, inter-frame fusion, and super-resolution enhancement—super-resolution reconstruction is performed on each low-resolution frame to obtain the high-resolution frame predicted by the convolutional neural network in the current round, including steps S401 to S403.

[0060] Step S401: In the feature extraction stage of the convolutional neural network, the features formed by each low-resolution frame at different feature levels are enhanced and extracted to obtain the multi-scale features corresponding to each low-resolution frame.

[0061] Among them, the feature representations formed at different feature levels represent the feature expressions obtained by low-resolution frames through different processing depths during the feature extraction stage. These expressions are used to reflect the information structure contained in the frame image from different levels, such as the local structural features formed in shallow processing and the overall structural features formed in deep processing.

[0062] Among them, multi-scale features represent a set of features formed by features at different feature levels, which are used to simultaneously characterize the information distribution of low-resolution frames at different spatial scales. For example, features that reflect local structure and features that reflect overall structure are combined to form a feature expression.

[0063] For example, after entering the feature extraction stage, the convolutional neural network takes each low-resolution frame as the basic input and processes and enhances the features formed in the low-resolution frames layer by layer around different feature levels to construct an information representation that can reflect the multi-level structure of the frame image. Specifically, the low-resolution frames are processed sequentially through multiple feature levels in the network, so that the information in the frame image is unfolded and reorganized at different processing depths, thereby forming feature representations for characterizing the local structural distribution at shallower levels and feature representations for characterizing the overall structural relationships at deeper levels.

[0064] Furthermore, by advancing and integrating the formed features at different feature levels, the information contained in each feature level is further highlighted, so that the information in the same low-resolution frame is continuously extracted at multiple scales and forms a stable representation, avoiding the problem of insufficient information representation caused by forming features only at a single feature level.

[0065] As processing at each feature level progresses, the features formed at different feature levels are unified and organized to form multi-scale features while maintaining spatial consistency. These multi-scale features are used to comprehensively reflect the content distribution of the low-resolution frame at both the detail and overall levels. Thus, through the above processing, each low-resolution frame completes the transformation from a single image representation to a multi-scale feature representation during the feature extraction stage, thereby providing a clear and information-rich input foundation for the subsequent inter-frame fusion stage.

[0066] Step S402: In the inter-frame fusion stage of the convolutional neural network, based on the motion features between adjacent frames, the multi-scale features corresponding to each low-resolution frame are fused to obtain the fused features corresponding to each low-resolution frame.

[0067] Among them, the motion feature representation between adjacent frames is used to reflect the feature information of the content change relationship between adjacent low-resolution frames in time sequence, and is used to guide the alignment and fusion of features between frames, such as feature expression reflecting the positional or structural changes of the same content in consecutive frames.

[0068] Among them, the fusion feature refers to the feature expression obtained by integrating the multi-scale features corresponding to each low-resolution frame with the motion features of adjacent frames in the inter-frame fusion stage, which is used as the reconstruction input in the super-resolution enhancement stage. For example, it includes feature expressions that contain information of the current frame and combine supplementary information from adjacent frames.

[0069] For example, after extracting multi-scale features corresponding to each low-resolution frame, the convolutional neural network enters the inter-frame fusion stage, and performs unified inter-frame fusion processing on the multi-scale features based on the temporal correlation between adjacent frames. Specifically, based on the temporal position of the low-resolution frame in the corresponding frame sequence, the correlation between adjacent frames is organized so that multi-scale features from different time positions are under the same temporal reference system, thereby providing a basis for subsequent fusion.

[0070] Based on this, the multi-scale features corresponding to each low-resolution frame are fused according to the motion characteristics reflected between adjacent frames, so that the current frame can combine the information contained in the preceding and following frames at the feature level. That is, at the feature level, the current frame and its adjacent frames are time-aligned and information-integrated, so that the multi-scale features that originally existed at different time positions are associated and organized under a unified time reference system, thereby incorporating the feature information of the current frame from the adjacent frames into the feature representation process of the current frame.

[0071] As the fusion process progresses, each low-resolution frame forms its own fused feature based on its corresponding multi-scale features and combined with relevant features from adjacent frames. This fused feature maintains the structural continuity within the frame while incorporating supplementary information from adjacent frames. Thus, low-resolution frames complete temporal integration at the feature level before entering the super-resolution enhancement stage, providing a stable and consistent input foundation for subsequent super-resolution restoration based on the fused features.

[0072] Step S403: In the super-resolution enhancement stage of the convolutional neural network, the fusion features corresponding to each low-resolution frame are subjected to convolution-based super-resolution restoration to obtain the predicted high-resolution frames corresponding to each low-resolution frame in the current round.

[0073] Among them, the convolution-based super-resolution restoration representation performs convolution processing on the fused features in the super-resolution enhancement stage to complete the mapping of the fused features to the high-resolution space, and further enhances the local structural information in the output result during the mapping process, thereby forming the corresponding predicted high-resolution frame.

[0074] For example, after constructing the fused features corresponding to each low-resolution frame, the convolutional neural network enters the super-resolution enhancement stage. Using each fused feature as a unified input, it performs super-resolution reconstruction and enhancement processing to generate predicted high-resolution frames that correspond one-to-one with each low-resolution frame. Specifically, the fused features are processed layer by layer within the predetermined structure of the convolutional neural network. Through continuous convolution operations, the feature representation is reconstructed, gradually mapping the features originally in the low-resolution space to the high-resolution space. This completes the spatial size reconstruction of the fused features, ensuring that the output results are consistent with the true high-resolution frames in terms of spatial size.

[0075] During the spatial size reconstruction process, the local structural information in the output result is simultaneously enhanced. This involves selectively strengthening the detailed areas formed during reconstruction while maintaining overall spatial structural stability. This makes the structural distribution in textured areas clearer and the contour transitions in edge areas more seamless. Thus, through this enhancement process, the output result achieves consistent spatial dimensions while further refining and presenting its internal structural information, resulting in an output frame that meets both structural integrity and dimensional consistency requirements.

[0076] Based on this, through the above processing, the fusion features corresponding to each low-resolution frame can independently complete the transformation from feature representation to predicted high-resolution frames, and serve as the final output of the convolutional neural network in the current training round, providing basic data for comparison with real high-resolution frames in subsequent training stages.

[0077] In this embodiment, in step S401, during the feature extraction stage, the features formed by each low-resolution frame at different feature levels are enhanced and extracted, and multi-scale features are constructed. This allows the structural information in the low-resolution frame to be fully expressed at different levels, providing a clear feature foundation for subsequent processing. In step S402, during the inter-frame fusion stage, based on the motion features between adjacent frames, the multi-scale features corresponding to each low-resolution frame are fused. This allows each low-resolution frame to maintain its own feature integrity while combining supplementary information from adjacent frames to form corresponding fused features. In step S403, during the super-resolution enhancement stage, the fused features corresponding to each low-resolution frame are subjected to convolution-based super-resolution restoration processing to form a predicted high-resolution frame that meets the requirements in terms of structural integrity and size consistency. Based on this, in the entire technical solution, by sequentially implementing the three stages of feature extraction, inter-frame fusion, and super-resolution enhancement, a staged processing mechanism for video super-resolution reconstruction is formed.

[0078] In an exemplary embodiment, during the feature extraction stage of the convolutional neural network, the features formed by each low-resolution frame at different feature levels are enhanced and extracted to obtain the multi-scale features corresponding to each low-resolution frame, including steps S501 to S502.

[0079] Step S501: Based on the preset improved RDN network, densely connect the features formed by each low-resolution frame at different feature levels to obtain the intermediate features corresponding to each low-resolution frame.

[0080] Among them, the improved RDN network refers to the network structure after adjusting the structure or connection method based on the RDN network (Residual Dense Network) to meet the needs of the feature extraction stage. It is used to centrally organize the features at each level by establishing dense connections between features at multiple levels within the network.

[0081] The intermediate features represent the feature representation obtained by densely connecting the features formed at different feature levels of each low-resolution frame through the improved RDN network, and are used as the basic input for subsequent enhancement extraction processing.

[0082] For example, in the feature extraction stage of the convolutional neural network, the features already formed at different feature levels of each low-resolution frame are used as processing objects, and dense connection processing is performed on the features based on the improved RDN network to achieve orderly integration of cross-level features. Specifically, for the same low-resolution frame, the features formed at each level after processing through multiple feature levels are associated with the features of the previous level layer by layer according to the preset dense connection relationship, so that each feature level can receive feature inputs from multiple previous levels when generating new features, thereby forming a dense connection structure between levels.

[0083] In other words, during the feature extraction process of the same low-resolution frame, it is pre-set that when the current feature level forms features, the features already formed in the previous few feature levels should be used as input sources, and the features from different levels should be spliced ​​and organized in a predetermined order before being used for the feature generation of the current level. This allows the features formed in the early and middle levels to participate in the feature construction of subsequent levels at the same time, thereby reflecting the dense connection relationship between feature levels.

[0084] Furthermore, the dense connections between feature levels can include progressive connections that pass from shallow features to deep features, allowing early-formed features to continuously participate in the construction of features at subsequent levels; round-trip connections that allow deep features to be passed back to shallow features for further processing, allowing features at different processing depths to repeatedly participate in integration; and cross connections that are established simultaneously between multiple shallow and multiple deep levels, allowing features at different levels to be jointly utilized in the same processing, thereby forming a dense connection relationship involving multiple directions and multiple levels.

[0085] Thus, through the aforementioned dense connection process, features formed at different feature levels no longer exist in isolation, but are gradually aggregated within the network in a continuous transmission manner, enabling information sharing among features at each level while maintaining their original hierarchical order. After the dense connections are completed, the features formed at different feature levels in each low-resolution frame are uniformly organized into structurally continuous intermediate features, allowing features formed at different levels during the feature extraction stage to be fully transmitted and integrated within the network, thereby completing the transformation from dispersed hierarchical features to centralized intermediate features.

[0086] Step S502: Based on the attention weights of each low-resolution frame at different feature levels, the intermediate features corresponding to each low-resolution frame are enhanced and extracted to obtain the multi-scale features corresponding to each low-resolution frame.

[0087] The attention weights of each low-resolution frame at different feature levels represent weight information used to reflect the relative importance of the same low-resolution frame at different feature levels, in order to guide the differential enhancement extraction of intermediate features.

[0088] For example, after obtaining the intermediate features corresponding to each low-resolution frame, the intermediate features are enhanced and extracted based on the differences in the roles of different feature levels in the current frame, so that key feature levels are highlighted in the output. Specifically, for the same low-resolution frame, the intermediate features contain feature information formed by dense connections of multiple feature levels, and the attention weights corresponding to each feature level are used to characterize their relative importance in the current frame. Therefore, in the enhancement extraction process, the attention weights are used as a guide to process the information of different feature levels in the intermediate features in a targeted manner, so that the feature levels with higher weights are repeatedly processed and enhanced in the feature integration process.

[0089] For example, when assigning attention weights to different feature levels, feature levels that can reflect texture structure and edge changes are given relatively high attention weights, so that these types of features are given priority in the processing and output during the enhancement extraction process. Meanwhile, feature levels that mainly reflect the overall contour or background information are given relatively low attention weights, so that they play an auxiliary role in feature construction.

[0090] Based on this, features with a high degree of correlation with the current frame content are continuously extracted and enhanced, while other hierarchical features participate in the overall construction according to their weights, thus forming a feature structure with clear hierarchical distinctions. As the enhancement extraction process is completed, the intermediate features corresponding to each low-resolution frame are transformed into multi-scale features. These multi-scale features, while maintaining hierarchical correlation, reflect the degree of enhancement of different feature levels in the overall representation.

[0091] Therefore, this technical solution combines dense connections between different feature levels with a hierarchical differentiated attention weight allocation mechanism, enabling the feature extraction process to move beyond relying solely on parallel or independent multi-scale feature acquisition methods. Instead, it forms a feature construction process characterized by continuous transmission and repeated participation between levels. Essentially, in the feature connection stage, shallow and deep features participate in subsequent feature processing through pre-defined dense connections. Furthermore, attention weights are introduced for different feature levels, allowing specific level features to be emphasized during integration, while other level features participate in the overall construction in an auxiliary capacity. In this way, the feature extraction process simultaneously reflects information sharing between levels and differentiated enhancement within levels, resulting in structurally continuous and content-specific features, thus distinguishing it from multi-scale feature processing methods that rely solely on scale stacking or simple fusion.

[0092] In this embodiment, in step S501, dense connection processing is performed on the features formed by each low-resolution frame at different feature levels, so that features from different levels can be continuously transmitted and integrated in a unified structure, thereby avoiding the fragmentation of features in layer-by-layer processing and preserving complete hierarchical association information for subsequent processing. In step S502, according to the attention weights of each low-resolution frame at different feature levels, the intermediate features obtained after dense connection are enhanced and extracted, so that features of specific levels are highlighted in the feature construction process, while features of other levels participate in the overall construction according to their weights, thereby forming multi-scale features with clear hierarchical distinctions. Based on this, in the entire technical solution, by first establishing dense connection relationships across levels and then introducing a hierarchical differentiated enhancement extraction mechanism, a feature extraction method that combines continuous integration and highlighting of key features between feature levels is realized.

[0093] In an exemplary embodiment, during the inter-frame fusion stage of the convolutional neural network, based on the motion characteristics between adjacent frames, the multi-scale features corresponding to each low-resolution frame are fused to obtain the fused features corresponding to each low-resolution frame, including steps S601 to S603.

[0094] Step S601: Based on the preset motion estimation based on lightweight optical flow, obtain the motion characteristics of each low-resolution frame between frames.

[0095] Among them, the motion estimation representation based on lightweight optical flow is used to describe the motion estimation method of the relative change relationship between adjacent frames in time order, so as to obtain the motion features between frames while controlling the computational complexity. For example, by analyzing the pixel changes between adjacent frames, a motion feature representation describing the displacement relationship between frames can be formed.

[0096] For example, based on the temporal order of low-resolution frames in the frame sequence, low-resolution frames at adjacent time positions are taken as processing objects. Using a motion estimation method based on lightweight optical flow, the pixel changes between adjacent frames are analyzed and organized to extract information reflecting the relative displacement and change trends between frames. In this process, by uniformly calculating the direction and magnitude of pixel position changes in adjacent frames, the motion relationships between frames are transformed into motion features that can be used for subsequent processing, thereby clarifying the temporal change state of low-resolution frames in the time dimension.

[0097] For example, when processing two low-resolution frames at consecutive time points, the process first involves corresponding regions at the same spatial location in both frames, comparing their pixel changes between the preceding and following frames, and then organizing the direction and magnitude of the overall pixel displacement to form information describing the relative temporal change between the two frames. Simultaneously, the degree of change in different regions within the frame is summarized, allowing regions with more pronounced changes to be distinguished from those with more gradual changes in the motion features. Thus, this method unifies the scattered pixel changes between adjacent frames into motion features that reflect the overall motion trend between frames.

[0098] Furthermore, the purpose of employing a lightweight optical flow approach is to reduce unnecessary computational overhead when describing the temporal changes of low-resolution frames, keeping the complexity of motion feature extraction controllable and thus avoiding excessive burden on the overall processing flow. In addition, it ensures that the obtained motion features, while maintaining the representation of inter-frame changes, are compatible with the subsequent processing flow of the convolutional neural network, allowing the motion feature acquisition process to be smoothly embedded into the overall network structure and serve subsequent feature fusion processing.

[0099] Step S602: Perform image analysis on the multi-scale features corresponding to each low-resolution frame to obtain the image features of each low-resolution frame.

[0100] For example, focusing on the spatial distribution of low-resolution frames in the spatial dimension, the spatial information contained in multi-scale features is organized layer by layer, so that the spatial information reflected at different scales is uniformly parsed and transformed into image feature representations with clear expressive forms. In this process, by analyzing the spatial distribution relationship corresponding to multi-scale features, the performance of different regions within the frame at different scales is summarized into a unified parsing result, thereby clarifying the spatial structural state of the low-resolution frame in terms of overall structure and local details, and obtaining the image features of the low-resolution frame.

[0101] For example, when analyzing the multi-scale features of a low-resolution frame, the regions reflected in the features at different scales are first aligned and organized, and the feature representations of the same region at coarse and fine scales are summarized. At the same time, the feature changes of edge regions, texture regions and relatively flat regions in the frame at each scale are summarized, so that these regions can be clearly distinguished in the analysis results. In this way, the spatial information scattered in the multi-scale features is unified and organized into image features that can characterize the overall structure and local details of the frame.

[0102] Step S603: Based on the attention weights of each low-resolution frame on different motion feature types and different image feature types, perform inter-frame feature fusion on the multi-scale features corresponding to each low-resolution frame to obtain the fused features corresponding to each low-resolution frame.

[0103] The attention weights of each low-resolution frame on different motion feature types represent weight information used to reflect the relative importance of the same low-resolution frame on different motion feature types, so as to guide the degree of participation of different motion features in the current frame during the inter-frame feature fusion process.

[0104] The attention weights of each low-resolution frame on different image feature types represent weight information used to reflect the relative importance of the same low-resolution frame in different image feature types, so as to guide the degree of participation of different image features in the current frame during the inter-frame feature fusion process.

[0105] For example, motion features that reflect significant displacement changes, local area motion, and overall motion trends between adjacent frames are classified into different motion feature types, and attention weights are assigned according to their correlation with the temporal changes of the current frame. Among them, motion feature types that can accurately reflect the changes between the current frame and adjacent frames are given higher weights, while motion feature types with smaller changes or weaker correlations are given relatively lower weights, so as to highlight motion information that is more representative of inter-frame changes.

[0106] For example, image features in low-resolution frames that reflect edge contours, texture distribution, and regional flatness can be divided into different image feature types, and attention weights can be assigned according to their importance to the structural expression of the current frame. Image feature types that depict edge and texture structures are given higher weights, while image feature types that mainly reflect the background or flat areas are given relatively lower weights, so that the fusion process focuses more on information related to structure and detail.

[0107] For example, based on the attention weights of each low-resolution frame on different motion feature types, the motion features reflected between adjacent frames are differentiated, so that motion features more closely related to the temporal changes of the current frame are preferentially introduced during the fusion process to dominate inter-frame fusion. Simultaneously, based on the attention weights of each low-resolution frame on different image feature types, the image features reflected in the current frame are differentiated, so that image features more critical to the structural expression of the current frame are also preferentially introduced during the fusion process to assist inter-frame fusion. On this basis, the information after weight adjustment of motion features and image features is jointly applied to multi-scale features, so that multi-scale features are simultaneously constrained and guided by both temporal change state and spatial structural state during fusion.

[0108] With the completion of the fusion process, a corresponding fusion feature is formed for each low-resolution frame. This fusion feature maintains the continuity of the intra-frame structure while comprehensively reflecting the change information of adjacent frames, providing a stable and consistent input basis for the subsequent super-resolution enhancement stage.

[0109] Specifically, when performing inter-frame fusion on multi-scale features, the weights of different motion features are first adjusted based on their respective attention weights, ensuring that motion features more closely related to the temporal changes of the current frame occupy a higher proportion in the adjustment result. Simultaneously, the weights of different image features are adjusted based on their respective attention weights, ensuring that image features more critical to the structural representation of the current frame occupy a higher proportion in the adjustment result. After adjusting the weights of both types of features, the weight adjustment result of the motion features is used as the primary information introduced into the inter-frame fusion process, while the weight adjustment result of the image features is used as auxiliary information in the fusion process.

[0110] Furthermore, the weight adjustment results of motion features are first used to guide the alignment and cross-frame introduction of multi-scale features at different time positions, so that multi-scale features prioritize the inter-frame variation relationship during the fusion process to complete the temporal integration. Subsequently, the weight adjustment results of image features are superimposed on the above fusion process to supplement and correct the multi-scale features that have completed temporal integration at the spatial structure level, so that the intra-frame structural information is maintained and improved in the fusion result. Thus, the inter-frame fusion process of multi-scale features is completed under the synergistic effect of the dominant and auxiliary functions.

[0111] Therefore, this technical solution demonstrates that the inter-frame fusion process does not solely rely on inter-frame relationships in the temporal dimension for feature integration. Instead, it simultaneously introduces a mechanism to differentiate between temporal changes and spatial structural states, resulting in a clearer hierarchical division of labor within the fusion process. Specifically, motion features and image features are first weighted separately, allowing each type of feature to be selected and highlighted in its respective dimension. Then, in the fusion stage, the weighting results of motion features guide the introduction and alignment of cross-frame features, ensuring accurate depiction of temporal relationships. Simultaneously, the adjustment results of image features supplement and constrain the fusion result structurally, thus forming a synergistic relationship between temporal consistency and spatial structural integrity. In this way, the inter-frame fusion process reflects both the main thread of inter-frame changes and the intra-frame structural expression, making the fusion result more consistent with the comprehensive characteristics of video data in both temporal and spatial dimensions.

[0112] In this embodiment, in step S601, motion features between adjacent low-resolution frames are extracted using a lightweight optical flow method, thereby clarifying the temporal change state between frames while controlling computational complexity, providing a reliable temporal correlation basis for subsequent fusion; in step S602, image analysis is performed on the multi-scale features corresponding to each low-resolution frame to obtain image features that can characterize the intra-frame structural distribution, thus enabling the fusion process to have a reliable spatial structural basis while introducing inter-frame information; in step S603, inter-frame feature fusion is performed on the multi-scale features according to the attention weights corresponding to different motion feature types and different image feature types, so that the obtained fused features maintain the intra-frame structural continuity while comprehensively reflecting the change information of adjacent frames; based on this, the entire technical solution realizes an inter-frame feature fusion process that simultaneously takes into account the consistency of temporal change and the integrity of spatial structure.

[0113] In an exemplary embodiment, during the super-resolution enhancement stage of the convolutional neural network, the fusion features corresponding to each low-resolution frame are subjected to convolution-based super-resolution restoration to obtain the predicted high-resolution frames corresponding to each low-resolution frame in the current round, including steps S701 to S702.

[0114] Step S701: Perform sub-pixel convolution processing on the fusion features corresponding to each low-resolution frame to obtain the restored frames corresponding to each low-resolution frame.

[0115] For example, the spatial size of the fused features remains at a low resolution. Therefore, sub-pixel convolution is needed to process the fused features to increase their spatial size without disrupting the internal relationships between features. In this process, convolution operations are performed on the fused features to form a rearrangeable representation in the channel dimension. Subsequently, through feature recombination operations, the information originally distributed in the channel dimension is mapped in an orderly manner to the spatial dimension, thereby expanding the fused features in spatial position and forming a frame structure corresponding to the specified resolution.

[0116] Through the above processing, the information in the fused features is rearranged into a higher resolution spatial grid, so that the low-resolution frame is restored in terms of spatial size, and the corresponding restored frame is obtained. The restored frame already has the basic form of the high-resolution frame in terms of overall structural layout, but still retains room for further processing in terms of detail representation. Therefore, it provides a consistent and structurally continuous input basis for the subsequent detail restoration process.

[0117] Step S702: Based on the preset improved residual block and attention mechanism, perform detail repair on the restored frames corresponding to each low-resolution frame to obtain the predicted high-resolution frames corresponding to each low-resolution frame in the current round.

[0118] For example, although the restored frame has been expanded in terms of spatial size, there is still room for further processing in terms of texture continuity, edge sharpness, and local structural integrity. Therefore, the restored frame is processed layer by layer by an improved residual block so that the original structural information in the restored frame can be preserved during the transmission process. On this basis, an attention mechanism is combined to distinguish the features of different regions in the restored frame, so that the texture regions and edge regions that are more critical to visual expression can be processed more fully in the detail restoration process.

[0119] In essence, when performing detail restoration on the restored frame, the frame is first processed layer by layer using an improved residual block. This allows the restored frame to directly transfer its original spatial structure information to subsequent layers, thus maintaining the overall outline and basic structure without shifting during multiple processing steps. Subsequently, an attention mechanism is introduced to distinguish the features of different spatial regions in the restored frame, giving higher weights to regions with denser texture changes or more pronounced edge transitions, while regions with smaller structural changes retain their original expression. Based on the similar features between adjacent frames, the missing or weakened details in the current frame are identified to be completed. Based on this, the corresponding content to be completed is then processed and supplemented in a targeted manner, combined with the corresponding weight allocation results.

[0120] Through the above methods, the detailed information in the restored frame is gradually repaired and enhanced, so as to further improve the expression integrity of the local area while maintaining the overall structural stability. As the detail repair process is completed, the restored frame corresponding to each low-resolution frame is converted into a predicted high-resolution frame. The predicted high-resolution frame is consistent with the corresponding real high-resolution frame in terms of spatial size and achieves a more complete presentation in terms of detail compared to the corresponding low-resolution frame, so as to serve as the output result of the super-resolution enhancement stage.

[0121] Therefore, in this technical solution, the super-resolution enhancement process does not simply expand the spatial size through a single magnification operation. Instead, it divides size reconstruction and detail restoration into two interconnected processing layers and implements them collaboratively. First, sub-pixel convolution is used to map features to the high-resolution space, allowing information in the channel dimension to be orderly rearranged into the spatial dimension, thus achieving resolution improvement while maintaining the continuity of the feature structure. Based on this, the result of the size expansion is then subjected to detail restoration processing. This involves differentiated processing of different regions while maintaining the overall structural continuity, specifically supplementing key areas such as textures and edges. Through this combination of methods, the resolution enhancement process goes beyond mere pixel-level expansion, focusing on the improvement of structure and details, resulting in an enhanced result that better meets the requirements of high-resolution representation.

[0122] In this embodiment, in step S701, the restored frame is obtained by performing sub-pixel convolution processing on the fused features, thereby achieving effective reconstruction of spatial dimensions and enabling the low-resolution frame to obtain a structural basis consistent with the specified resolution. In step S702, the restored frame is repaired for details based on the improved residual block and attention mechanism, thereby maintaining the continuity of the overall structure while performing targeted enhancement on key areas such as texture and edges, so that the reconstruction result is improved at the detail level. Based on this, in the entire technical solution, by first completing the spatial dimension restoration and then performing detail-level repair and enhancement, the synergistic improvement of size consistency and detail expression capability is achieved, so that the predicted high-resolution frame is effectively guaranteed in terms of structural integrity and visual performance.

[0123] In an exemplary embodiment, subpixel convolution processing is performed on the fusion features corresponding to each low-resolution frame to obtain the restored frames corresponding to each low-resolution frame, including steps S801 and S803.

[0124] Step S801: Perform spatial convolution on the fusion features corresponding to each low-resolution frame channel by channel to obtain the channel feature information corresponding to each low-resolution frame.

[0125] For example, the different channels in the fused features correspond to the feature representations formed by low-resolution frames under different expressive emphases. Some channels mainly reflect the overall structure, some mainly reflect local changes, and some mainly reflect edge transitions or regional differences. Based on this, by using channel-wise spatial convolution, each channel maintains its original expressive emphase while further modeling and organizing the feature relationships within each channel. This makes the information expressed by the channel more coherent and consistent in spatial structure and avoids interference between different channels in subsequent processing stages due to unclear relationships.

[0126] In essence, during spatial convolution processing, different channels are processed independently, allowing each channel to complete spatial correlation modeling while maintaining its own expressive focus. Specifically: for channels emphasizing overall structural expression, a spatial convolution window matching the structural scale is set within its feature distribution range and slides along the spatial dimension, allowing structurally relevant features within the window to participate in the combined calculation, thus forming a continuous and consistent spatial relationship of the overall structure within the channel; for channels emphasizing local variation expression, a spatial convolution window matching the variation range is set within its feature distribution range and slides along the spatial dimension, allowing variation-related features within the window to participate in the combined calculation, thus forming a continuous transmission of local variation trends within the channel; for channels emphasizing edge transition expression, a spatial convolution window matching the transition region is set within its feature distribution range and slides along the spatial dimension, allowing transition-related features within the window to participate in the combined calculation, thus forming a smooth connection between different regions within the channel.

[0127] Based on this, as the channel-wise spatial convolution process is completed, the fused features are converted into corresponding channel feature information. This channel feature information maintains the same number of channels as the original fused features, but its internal correlation expression is clearer and more orderly.

[0128] In step S802, the channel feature information corresponding to each low-resolution frame is convolved and fused point by point to obtain the feature map corresponding to each low-resolution frame.

[0129] For example, channel feature information still exists in a multi-channel form, with different channels carrying feature representations with different emphases. Therefore, it is necessary to unify and integrate the channel feature information along the channel dimension while maintaining the spatial structural relationship. Based on this, pointwise convolution is used to combine and calculate the channel feature information from different channels at each spatial location, so that the feature representations scattered across multiple channels at the same spatial location are converged into a consistent representation. In this process, pointwise convolution does not introduce spatial expansion but processes around the channel dimension, thereby ensuring the stability of the spatial structural relationship and clearly expressing the feature correlations between channels.

[0130] Specifically, when performing pointwise convolutional fusion on channel feature information, a fixed spatial location is used as the processing benchmark. The feature values ​​corresponding to this spatial location in different channels are simultaneously used as inputs for calculation. These feature values ​​are weighted and combined through pointwise convolution to form a unified feature representation at that spatial location. The weighting is based on the principle that features that contribute more to the representation of the current spatial location in different channels have a higher weight, while features that contribute less are relatively weakened. Subsequently, the above process is repeated for other spatial locations, so that the channel features at each spatial location are fused into the corresponding unified representation.

[0131] Furthermore, the weight adjustment in the pointwise convolution stage is used to complete the basic integration of multi-channel features, only determining the combination ratio of each channel when forming the restored frame, without making a final selection of detail information; the subsequent detail restoration stage will refocus on texture and edge regions based on the restoration result, so the change of the weight of the preceding multi-channel features will not limit the subsequent detail enhancement effect.

[0132] In this process, the fusion operation is always confined to the same spatial location and does not involve information interaction between adjacent locations. This allows for feature integration and correlation expression in the channel dimension while maintaining the spatial structure. Based on this, through the above processing, the channel feature information corresponding to each low-resolution frame is converted into a corresponding feature map. This feature map remains consistent with the input in the spatial dimension, while simultaneously undergoing information compression and reorganization in the channel dimension, making the feature expression more concentrated and orderly.

[0133] Step S803: Perform sub-pixel convolution processing on the feature maps corresponding to each low-resolution frame to obtain the restored frames corresponding to each low-resolution frame.

[0134] For example, when the feature map is in a low spatial resolution form, the information in the feature map is organized and reconstructed by subpixel convolution, so that it completes spatial expansion while maintaining the original spatial arrangement relationship. The expressions originally concentrated inside the feature map are orderly mapped to a higher resolution spatial position, so that the feature map is expanded in spatial dimension and forms a frame structure that matches the specified spatial resolution, so as to obtain the corresponding restored frame, thereby providing a consistent and structurally stable input basis for subsequent detail repair processing.

[0135] Therefore, in this technical solution, the spatial reconstruction process does not directly involve convolving and rearranging all features. Instead, it first decomposes the features using depthwise separable convolution, modeling the relationships between features in both the channel and spatial dimensions separately. This reduces computational complexity while preserving the expressive emphasis within each channel. Based on this, sub-pixel convolution is then used to spatially rearrange the processed features, ensuring that the information used for high-resolution representation is mapped sequentially to the spatial dimension. In this way, sub-pixel convolution no longer undertakes the entire feature arrangement task but builds upon the features whose channel and spatial relationships have already been clarified, thus making the spatial expansion process more stable.

[0136] In this embodiment, in step S801, spatial convolution is performed on the fused features channel by channel to strengthen the feature correlation within each channel, thereby enabling channels with different expression focuses to form channel feature information with clear structure and stable relationship before entering fusion; in step S802, point-by-point fusion is performed on the channel feature information to orderly integrate features from different channels at the same spatial location, thereby forming a feature map for spatial reconstruction, making the feature expression more concentrated and consistent; in step S803, sub-pixel convolution processing is performed on the feature map to achieve orderly expansion of the feature map in the spatial dimension, thereby completing the conversion from low-resolution feature expression to a frame structure of a specified resolution, and obtaining a restored frame with consistent spatial size; based on this, in the entire technical solution, a super-resolution restoration process with stable structure and continuous expression is constructed through channel internal correlation strengthening, orderly fusion of channel information, and feature map-driven spatial size reconstruction.

[0137] In an exemplary embodiment, the convolutional neural network is iteratively updated based on the difference between the predicted high-resolution frames and the corresponding real high-resolution frames in each round, including step S901.

[0138] Step S901: Based on the composite loss function constructed from perceptual loss, adversarial loss, and temporal loss, calculate the loss of the predicted high-resolution frame and the corresponding real high-resolution frame in each round to obtain the composite loss of each round, which is then used to iteratively update the convolutional neural network.

[0139] For example, after obtaining the predicted high-resolution frames output by the convolutional neural network in the current training epoch, each predicted high-resolution frame is aligned frame by frame with the corresponding real high-resolution frame at the time position to ensure that subsequent loss calculations are based on a one-to-one correspondence. Subsequently, for each pair of predicted high-resolution frames and real high-resolution frames, perceptual loss, adversarial loss, and temporal loss are calculated, where: perceptual loss represents the loss term for the difference in feature representation between the predicted high-resolution frame and the real high-resolution frame, used to constrain the prediction result to be consistent with the real result at the level of overall structure and local details; adversarial loss represents the loss term for the difference in overall distribution between the predicted high-resolution frame and the real high-resolution frame, used to constrain the prediction result to conform to the distribution characteristics of the real result in overall performance; temporal loss represents the loss term for the loss term for the continuous difference between the predicted high-resolution frame and the real high-resolution frame at the time level, used to constrain the consistency between the prediction result and the real result in the temporal change relationship at adjacent time positions or adjacent epochs.

[0140] After obtaining the three types of losses for each predicted high-resolution frame, they are combined to obtain the composite loss for that predicted high-resolution frame. Furthermore, the composite losses for all predicted high-resolution frames in the current round are summarized to obtain a composite loss that characterizes the overall difference state of all frames in the current training round. This composite loss serves as the basis for iterative updating of the convolutional neural network parameters, thus ensuring consistency between individual frame quality and overall quality during network training.

[0141] Optionally, when comprehensively processing the perceptual loss, adversarial loss, and temporal loss corresponding to each predicted high-resolution frame, a weighted fusion method can be used to summarize the three types of losses. The weights of each type of loss are pre-set according to their constraint focus in the current training stage, so that the loss used to constrain the structure and detail representation, the loss used to constrain the overall distribution, and the loss used to constrain temporal continuity occupy different proportions in the composite loss, thereby balancing the role of each type of loss in iterative updates through weight allocation.

[0142] Alternatively, the composite loss can be constructed as follows: First, the perceptual loss, adversarial loss, and temporal loss corresponding to all predicted high-resolution frames in the current training round are summarized in a unified manner to obtain the overall loss results of the three types of losses in the current training round. Then, the three types of overall loss results are further integrated to form a composite loss that characterizes the overall difference state of all frames in the current training round, which serves as the basis for the iterative update of the convolutional neural network parameters.

[0143] This technical solution incorporates three constraint dimensions—feature representation, overall distribution, and temporal continuity—into a unified loss calculation framework. This transforms the loss calculation process from merely constraining a single quality indicator or static error, and from simply superimposing multiple constraint terms, into a comprehensive measurement mechanism focused on the quality of prediction results. Different losses are applied to feature representation, overall performance, and temporal variation, respectively, and participate in network updates through a unified aggregation method. This simultaneously constrains the prediction results in terms of structural consistency, distribution rationality, and temporal stability. This loss calculation method aligns with the requirements for detail restoration and temporal coherence in video super-resolution reconstruction, making the network training process more aligned with the quality requirements of real-world application scenarios.

[0144] Furthermore, the loss composite function is mainly used to evaluate the difference between the predicted high-resolution frame and the corresponding real high-resolution frame at the single-frame dimension. At the same time, the loss composite function can also be extended to the video dimension. By summing up and averaging the loss values ​​corresponding to each frame in the frame sequence, the difference between the predicted video and the real video at the overall level can be evaluated, thereby taking into account both frame-level accuracy and sequence-level consistency.

[0145] In this embodiment, in step S901, the composite loss for each round is obtained by calculating the perceptual loss, adversarial loss, and temporal loss of the predicted high-resolution frames and the corresponding real high-resolution frames in each round. This gives the network update a clear basis for measuring differences and covers three constraint dimensions: feature representation, overall distribution, and temporal continuity. Based on this, in the entire technical solution, the iterative mechanism with composite loss as the unified update basis is used to effectively drive and control the training process of the convolutional neural network, so that the training update can be continuously adjusted around the difference between the predicted results and the real results.

[0146] In one exemplary embodiment, Figure 2 The diagram illustrates the architecture of the video reconstruction model during the deployment phase. Based on this, after completing model training and fixing network parameters, the video reconstruction model is deployed in a real-world operating environment to perform stage-by-stage processing on the input low-resolution video. Specifically, the low-resolution video is first received as input and sent to the front-end processing module. Here, a scene classifier analyzes the overall content features of the low-resolution video to determine its corresponding scene type. After scene determination, the corresponding preprocessing strategy is invoked to perform appropriate noise filtering, illumination normalization, and video frame processing on the low-resolution video, ensuring high consistency and stability of the video frames entering the main network.

[0147] After preprocessing, each video frame of the low-resolution video enters the main body of the convolutional neural network and sequentially goes through the feature extraction stage, the inter-frame fusion stage, and the super-resolution enhancement stage. In the feature extraction stage, the video frame is first input to the dense connection layer of the improved RDN network. Through the dense connection structure, the intra-frame features are passed and integrated layer by layer to form intermediate features containing multiple feature levels. Subsequently, the intermediate features of multiple feature levels are input to the attention layer, which assigns weights to different feature levels to highlight the features that are more critical to the content of the current frame, thereby outputting enhanced multi-scale features.

[0148] The multi-scale features output from the feature extraction stage further enter the inter-frame fusion stage. In this stage, the motion relationship between adjacent video frames is first analyzed through a lightweight optical flow motion estimation layer to obtain motion features used to characterize the inter-frame change state. Subsequently, the motion features and multi-scale features are input together into the attention layer, which assigns weights to different motion feature types and different image feature types to perform inter-frame feature fusion of multi-scale features, thereby forming fused features that include temporal dimension information.

[0149] The fused features output from the inter-frame fusion stage further enter the super-resolution enhancement stage. In this stage, the fused features are first fed into a channel-wise spatial convolutional layer and a point-wise convolutional fusion layer to perform channel-level processing and fusion, forming a feature map for spatial reconstruction. Subsequently, this feature map enters a sub-pixel convolutional layer, where spatial expansion is performed, mapping it to a restored frame consistent with the specified high resolution. Based on this, the restored frame is further fed into a detail restoration layer, which processes the texture structure and edge regions in the restored frame to improve the integrity of detail representation. Finally, the result after the super-resolution enhancement stage is output as a high-resolution video.

[0150] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0151] Based on the same inventive concept, this application also provides a model training system for video super-resolution reconstruction to implement the model training method for video super-resolution reconstruction described above. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the model training system for video super-resolution reconstruction provided below can be found in the limitations of the model training method for video super-resolution reconstruction described above, and will not be repeated here.

[0152] In one exemplary embodiment, such as Figure 3 As shown, a model training system for video super-resolution reconstruction is provided, including: an acquisition module 101, a training module 102, and an update module 103, wherein: The acquisition module 101 is used to preprocess the preset video sample set to obtain the training dataset, which includes multiple low-resolution frames and the real high-resolution frames corresponding to each low-resolution frame. Training module 102 is used to perform super-resolution reconstruction on each low-resolution frame based on a convolutional neural network constructed in advance based on three stages: feature extraction, inter-frame fusion, and super-resolution enhancement, so as to obtain the high-resolution frame predicted by the convolutional neural network in the current round. The update module 103 is used to iteratively update the convolutional neural network based on the difference between the predicted high-resolution frames and the corresponding real high-resolution frames in each round, and to use the convolutional neural network that meets the preset evaluation conditions as a trained video reconstruction model for super-resolution reconstruction of any video.

[0153] The modules in the aforementioned model training system for video super-resolution reconstruction can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0154] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above embodiments.

[0155] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above embodiments.

[0156] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A model training method for video super-resolution reconstruction, characterized in that, The method includes: A training dataset is obtained by preprocessing a preset video sample set. The training dataset includes multiple low-resolution frames and the real high-resolution frames corresponding to each low-resolution frame. Based on the convolutional neural network pre-constructed in three stages—feature extraction, inter-frame fusion, and super-resolution enhancement—super-resolution reconstruction is performed on each low-resolution frame to obtain the predicted high-resolution frame of the convolutional neural network in the current round. Based on the difference between the predicted high-resolution frames and the corresponding real high-resolution frames in each round, the convolutional neural network is iteratively updated, and the convolutional neural network that meets the preset evaluation conditions is used as a trained video reconstruction model for super-resolution reconstruction of any video.

2. The method according to claim 1, characterized in that, The preprocessing of the preset video sample set to obtain the training dataset includes: Extract the initial frame sequence corresponding to each video sample in the video sample set, and perform noise filtering on each initial frame sequence to obtain the first frame sequence corresponding to each initial frame sequence. Based on the difference in light intensity between each first frame sequence, the light intensity of each first frame sequence is normalized frame by frame to obtain the second frame sequence corresponding to each first frame sequence. Each frame of a pair of second frame sequences corresponding to the same video content and corresponding to high resolution and low resolution respectively is labeled with the actual high-resolution frame and low-resolution frame to obtain the corresponding high-low resolution frame pair. Data augmentation is performed on each high-low resolution frame pair to obtain multiple extended high-low resolution frame pairs. All high-low resolution frame pairs and all extended high-low resolution frame pairs are then combined to form a training dataset.

3. The method according to claim 2, characterized in that, Before extracting the initial frame sequence corresponding to each video sample in the video sample set, the method further includes: Based on the sampling scene characteristics of the current video sample, a preprocessing strategy matching the current video sample is determined in a preset scene classifier, which is used to perform noise filtering, illumination normalization and data augmentation on the current video sample.

4. The method according to claim 1, characterized in that, The process involves using a convolutional neural network pre-constructed based on three stages—feature extraction, inter-frame fusion, and super-resolution enhancement—to perform super-resolution reconstruction on each low-resolution frame, thereby obtaining the predicted high-resolution frame for the current round. This includes: In the feature extraction stage of the convolutional neural network, the features formed by each low-resolution frame at different feature levels are enhanced and extracted to obtain the multi-scale features corresponding to each low-resolution frame. In the inter-frame fusion stage of the convolutional neural network, based on the motion characteristics between adjacent frames, the multi-scale features corresponding to each low-resolution frame are fused to obtain the fused features corresponding to each low-resolution frame. In the super-resolution enhancement stage of the convolutional neural network, the fusion features corresponding to each low-resolution frame are subjected to convolution-based super-resolution restoration to obtain the predicted high-resolution frames corresponding to each low-resolution frame in the current round.

5. The method according to claim 4, characterized in that, In the feature extraction stage of the convolutional neural network, the features formed at different feature levels of each low-resolution frame are enhanced and extracted to obtain multi-scale features corresponding to each low-resolution frame, including: Based on the pre-defined improved RDN network, the features formed by each low-resolution frame at different feature levels are densely connected to obtain the intermediate features corresponding to each low-resolution frame. Based on the attention weights of each low-resolution frame at different feature levels, the intermediate features corresponding to each low-resolution frame are enhanced and extracted to obtain the multi-scale features corresponding to each low-resolution frame.

6. The method according to claim 4, characterized in that, In the inter-frame fusion stage of the convolutional neural network, based on the motion features between adjacent frames, inter-frame feature fusion is performed on the multi-scale features corresponding to each low-resolution frame to obtain the fused features corresponding to each low-resolution frame, including: Based on the preset motion estimation based on lightweight optical flow, the motion characteristics of each low-resolution frame between frames are obtained; Image analysis is performed on the multi-scale features corresponding to each low-resolution frame to obtain the image features of each low-resolution frame; Based on the attention weights of each low-resolution frame on different motion feature types and different image feature types, inter-frame feature fusion is performed on the multi-scale features corresponding to each low-resolution frame to obtain the fused features corresponding to each low-resolution frame.

7. The method according to claim 4, characterized in that, In the super-resolution enhancement stage of the convolutional neural network, convolution-based super-resolution restoration is performed on the fused features corresponding to each low-resolution frame to obtain the predicted high-resolution frame corresponding to each low-resolution frame in the current round, including: Subpixel convolution processing is performed on the fusion features corresponding to each low-resolution frame to obtain the restored frames corresponding to each low-resolution frame. Based on the preset improved residual block and attention mechanism, the details of the restored frames corresponding to each low-resolution frame are repaired to obtain the predicted high-resolution frames corresponding to each low-resolution frame in the current round.

8. The method according to claim 7, characterized in that, The sub-pixel convolution processing of the fusion features corresponding to each low-resolution frame to obtain the restored frames corresponding to each low-resolution frame includes: Spatial convolution is performed on the fusion features corresponding to each low-resolution frame channel by channel to obtain the channel feature information corresponding to each low-resolution frame. The channel feature information corresponding to each low-resolution frame is convolved and fused point by point to obtain the feature map corresponding to each low-resolution frame. Subpixel convolution is performed on the feature maps corresponding to each low-resolution frame to obtain the restored frames corresponding to each low-resolution frame.

9. The method according to claim 1, characterized in that, The step of iteratively updating the convolutional neural network based on the difference between the predicted high-resolution frames and the corresponding real high-resolution frames in each round includes: Based on the composite loss function constructed from perceptual loss, adversarial loss, and temporal loss, the loss is calculated for the predicted high-resolution frames and the corresponding real high-resolution frames in each round to obtain the composite loss for each round, which is then used to iteratively update the convolutional neural network.

10. A model training system for video super-resolution reconstruction, characterized in that, The system includes: The acquisition module is used to preprocess a preset video sample set to obtain a training dataset, which includes multiple low-resolution frames and the real high-resolution frames corresponding to each low-resolution frame. The training module is used to perform super-resolution reconstruction on each low-resolution frame based on a convolutional neural network constructed in advance based on three stages: feature extraction, inter-frame fusion, and super-resolution enhancement, so as to obtain the predicted high-resolution frame of the convolutional neural network in the current round. The update module is used to iteratively update the convolutional neural network based on the difference between the predicted high-resolution frames and the corresponding real high-resolution frames in each round, and to use the convolutional neural network that meets the preset evaluation conditions as a trained video reconstruction model for super-resolution reconstruction of any video.

Images