Unlock instant, AI-driven research and patent intelligence for your innovation.
A method and system for video blind super-resolution reconstruction based on self-supervised learning
What is Al technical title?
Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
A technology of super-resolution reconstruction and supervised learning, applied in the field of video blind super-resolution reconstruction based on self-supervised learning, can solve the problems of complex blur kernel, complex image degradation process, and reduced visual effect, so as to improve visual effect and improve Generalization ability, improvement of false artifact and effect of erroneous structural information
Active Publication Date: 2021-10-15
NANJING UNIV OF SCI & TECH
View PDF8 Cites 0 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
However, the blur kernel in the actual scene is more complex, so the data set constructed with the hypothetical blur kernel, and then the depth model trained with the data set has poor generalization ability on real videos, but the image degradation in the actual application scene The process is more complicated, so the depth model trained by the above method will have false artifacts and wrong structural information when reconstructing the real video at high resolution. This kind of wrong information will reduce the visual effect, and based on the reconstructed high resolution When the rate video is used for downstream tasks, it will cause a decrease in accuracy
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0068] Such as figure 1 Shown, a kind of video blind super-resolution reconstruction method based on self-supervised learning, described method comprises:
[0069] S1: Determine a first-resolution video sequence based on the first-resolution video.
[0070] S2: A self-supervised learning method is used to determine the blur kernel estimation network, optical flow estimation network, feature extraction network and latent high-resolution intermediate frame reconstruction network.
[0071] S3: Based on the blur kernel estimation network, use the first resolution video sequence to estimate a blur kernel.
[0072] S4: Determine a deformation matrix based on the optical flow estimation network and the first resolution video sequence.
[0073] S5: Use the feature extraction network to extract the features of each video frame in the first resolution video sequence, align the features of each video frame according to the deformation matrix, and obtain the aligned features of each vid...
Embodiment 2
[0129] Such as Figure 5 As shown, the present invention also provides a video blind super-resolution reconstruction system based on self-supervised learning, and the system includes:
[0130] The first-resolution video sequence determining module 501 is configured to determine the first-resolution video sequence based on the first-resolution video.
[0131] The multi-network determination module 502 is configured to determine a blur kernel estimation network, an optical flow estimation network, a feature extraction network and a potential high-resolution intermediate frame reconstruction network by using a self-supervised learning method.
[0132] A blur kernel determination module 503, configured to estimate a blur kernel by using the first resolution video sequence based on the blur kernel estimation network.
[0133] A deformation matrix determining module 504, configured to determine a deformation matrix based on the optical flow estimation network and the first resoluti...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More
PUM
Login to View More
Abstract
The present invention provides a video blind super-resolution reconstruction method and system based on self-supervised learning. The method includes: firstly, using the self-supervised learning method to determine the blur kernel estimation network, optical flow estimation network, feature extraction network and potential high-resolution intermediate Frame reconstruction network; based on the blur kernel estimation network, use the first resolution video sequence to estimate the blur kernel; secondly, determine the deformation matrix based on the optical flow estimation network and the first resolution video sequence; then use the feature extraction network to extract the first resolution video sequence The features of each video frame in , and align the features of each video frame according to the deformation matrix; again use the latent high-resolution intermediate frame reconstruction network and the features of each video frame after alignment to construct the second resolution intermediate video frame; finally based on the second resolution The rate of the middle video frame determines the second resolution video. The invention adopts the self-supervision method, which can effectively improve false artifacts and wrong structure information when reconstructing high-resolution video, and further improve the visual effect.
Description
technical field [0001] The present invention relates to the technical field of video resolution reconstruction, in particular to a video blind super-resolution reconstruction method and system based on self-supervised learning. Background technique [0002] At present, high-resolution display devices are developing rapidly, but blurring and obvious artifacts are inevitable when using these devices to project low-resolution videos, so video super-resolution technology has received more and more attention. [0003] The goal of video super-resolution technology is to reconstruct a high-resolution video from a given low-resolution video. The degradation process of the video super-resolution problem is usually defined as: [0004] the y j =SK j f i→j x i +n,j=i-N,i-N+1,...,i+N (1), [0005] Among them, y j 、x i , n represent the low-resolution image of the j-th video frame, the high-resolution image of the i-th video frame and noise respectively; S and K j Denote the dow...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to View More 
Login to View More