Video super-resolution reconstruction system and method based on deep learning
Through video frame decomposition and pixel measurement sampling, combined with deep learning algorithms, the problem of low video super-resolution processing quality in the prior art is solved, and efficient and high-quality video super-resolution reconstruction is achieved.
Patent Information
- Application Number
- CN202411536012.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing video super-resolution processing methods cannot perform optimal super-resolution video reconstruction based on multiple video samples, resulting in reduced quality and applicability of video reconstruction.
By collecting sample video data, performing video frame decomposition processing and image pixel measurement sampling, constructing optimal pixel measurement sampling image data, realizing numerical measurement and maximum value search of video frame pixel density, combining and reconstructing pixel images with deep learning algorithms. Generate efficient and high-quality super-resolution video data.
It realizes high efficiency, accuracy and high quality of video super-resolution reconstruction, and improves the reliability and applicability of video reconstruction.
Smart Images

Figure CN119067852A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image data processing, and specifically to a video super-resolution reconstruction system and method based on deep learning. Background Art
[0002] Super-resolution is to improve the resolution of the original image through hardware or software methods. The process of obtaining a high-resolution image through a series of low-resolution images is super-resolution reconstruction. The core idea of super-resolution reconstruction is to use temporal bandwidth (obtain a multi-frame image sequence of the same scene) in exchange for spatial resolution, realizing the conversion of temporal resolution to spatial resolution; if existing videos want to obtain super-resolution effects, they can only do so by increasing the shooting pixels of the hardware device or using image data denoising algorithms; existing video super-resolution processing methods cannot be based on the optimal super-resolution video reconstruction processing of multiple video samples, which reduces the quality and applicability of super-resolution video reconstruction.
[0003] The Chinese invention patent application with publication number CN117058002A discloses a video frame super-resolution reconstruction method, device and computer equipment, which obtains streaming media video; extracts the first focus feature of the first video frame through image feature extraction; obtains the first sampling rate of the first video frame through the target mapping relationship according to the first video frame; resamples the first video frame according to the first sampling rate to obtain a second video frame; performs super-resolution processing on the second video frame to obtain a second super-resolution video frame; replaces the second super-resolution video frame with the streaming media video to obtain a streaming media super-resolution video. However, the above technical solution can only perform super-resolution processing on the focus features in the video frame, and cannot perform global image super-resolution reconstruction on the entire video frame, and cannot guarantee the quality of super-resolution reconstruction of the video frame. Summary of the invention
[0004] 1. Technical issues to be resolved In order to solve the problem that the above-mentioned existing video super-resolution processing methods cannot be based on the optimal super-resolution video reconstruction processing of multiple video samples, which reduces the quality and applicability of super-resolution video reconstruction, the above purpose of accurately and autonomously generating sample video frame images, scientifically dividing sample video frame pixel measurement sampling images, intelligently screening the optimal sample video frame pixel measurement sampling images, accurately reconstructing the optimal sample video frame images, and realizing efficient and high-quality super-resolution video reconstruction operations is achieved.
[0005] (II) Technical solution The present invention is implemented by the following technical solution: a video super-resolution reconstruction method, the method comprising the following steps: S1, collect sample video data; S2, performing video frame decomposition processing on the sample video according to the sample video data to generate sample video frame image data; S3, performing a grid division process on the image pixel measurement sampling object of the sample video frame number based on the sample video frame image data to generate sample video frame pixel measurement sampling image data; S4, performing pixel density numerical measurement processing of the sample video frame pixel measurement sampling image according to the sample video frame pixel measurement sampling image data, and generating the sample video frame pixel measurement sampling image pixel density data; S5, performing pixel density maximum value search processing of the sample video frame number pixel measurement sampling image according to the sample video frame pixel measurement sampling image pixel density data, generating sample video frame pixel measurement sampling image pixel density extreme value data and performing optimal pixel measurement sampling image object analysis processing of the sample video frame number in combination with the sample video frame pixel measurement sampling image data, constructing the sample video frame pixel measurement optimal sampling image data; S6, performing optimal pixel image combination reconstruction data processing of the sample video frame number based on the optimal sampling image data of the sample video frame pixel measurement to generate optimal image data of the sample video frame; S7. Construct super-resolution video reconstruction sample image data and perform super-resolution video reconstruction operations to generate super-resolution sample video data.
[0006] Preferably, the steps of collecting sample video data are as follows: S11, repeatedly collecting video data of the video shooting object online through the video lens and generating a sample video data set , ;in Indicates the number of repeated collections Sample video data, Indicates the maximum number of sample videos.
[0007] The present invention collects sample video parameters online efficiently through a video lens, thereby providing data support for decomposing the sample video to generate video frame images.
[0008] Preferably, the steps of performing video frame decomposition processing of the sample video according to the sample video data to generate sample video frame image data are as follows: S21, using video editing software to set the sample video data set according to the time axis of the sample video Sample video data described in Decompose the sample video into individual sample video frames in order according to the sample video number, and generate a sample video frame image data set , ,in Indicates the maximum number of sample video frame images. Represents the sample video data The decomposition generated Sample video frame image data; Represents the sample video data The decomposition generated Sample video frame image data; Represents the sample video data The decomposition generated Sample video frame image data; Represents the sample video data The decomposition generated Sample video frame image data; Represents the sample video data The decomposition generated Sample video frame image data; Represents the sample video data The decomposition generated Sample video frame image data; Represents the sample video data The decomposition generated Sample video frame image data; Represents the sample video data The decomposition generated Sample video frame image data; Represents the sample video data The decomposition generated Sample video frame image data, the video editing software includes any one of Adobe Premiere Pro, Media DreamWorks, and Fotor.
[0009] The present invention achieves the effect of accurately and autonomously generating sample video frame images by efficiently decomposing the video frames of the sample video based on sample video parameters in combination with video editing software.
[0010] Preferably, the steps of performing image pixel measurement sampling object grid division processing of the sample video frame number based on the sample video frame image data to generate the sample video frame pixel measurement sampling image data are as follows: S31, using a square with a side length of L to collect the sample video frame image data set The sample video frame image data described in According to the sample video number and the sample video frame image number, the image grid division processing of the sample video frame image pixel measurement sampling object is carried out in order to generate a sample video frame pixel measurement sampling image data set , ,in Indicates The first The sample video frame image data is divided into image grids to generate a sample video frame pixel measurement sampling image data set, Represents the sample video frame pixel measurement sampled image data set Middle Sample video frame pixel measurement sampled image data, Indicates the maximum number of sampled images for pixel measurement of a sample video frame.
[0011] The present invention achieves the effect of accurate digital acquisition of sample video frame pixel measurement sampling images by performing scientific grid division processing on sample video frame images.
[0012] Preferably, the pixel density value measurement processing of the sample video frame pixel measurement sampling image is performed according to the sample video frame pixel measurement sampling image data, and the operation steps of generating the sample video frame pixel measurement sampling image pixel density data are as follows: S41, using image measurement software to measure the sample video frame pixels and sample image data set The sample video frame pixel measurements sample image data According to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, the pixel density numerical measurement processing of the sample video frame pixel measurement sampling image is performed in order, and a sample video frame pixel measurement sampling image pixel density data set is generated ,in Indicates The first A sample video frame pixel measurement sampling image pixel density data set corresponding to the sample video frame image data, Represents the sample video frame pixel measurement sampled image data The corresponding sample video frame pixel measurement sampled image pixel density data, The unit is PPI, and the image measurement software includes any one of MediaInfo, ImageMeter, and Photoshop.
[0013] The present invention uses image measurement software to accurately perform sample video frame number pixel measurement and sample image pixel density numerical measurement processing, thereby achieving the effect of providing real data support for implementing sample video screening out super-resolution sample images.
[0014] Preferably, according to the sample video frame pixel measurement sampling image pixel density data, the pixel density maximum value search processing of the sample video frame number pixel measurement sampling image is performed, the sample video frame pixel measurement sampling image pixel density extreme value data is generated, and the sample video frame number pixel measurement sampling image object analysis processing is performed in combination with the sample video frame pixel measurement sampling image data, and the operation steps of constructing the sample video frame pixel measurement optimal sampling image data are as follows: S51, using a breadth-first search algorithm to measure the sample video frame pixels and sample the image pixel density data set The sample video frame pixel measurement sample image pixel density data According to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, the pixel density values of the sample video frame pixel measurement sampling image are compared in order, and the sample video frame pixel measurement sampling image pixel density data with the largest value corresponding to different sample video numbers, the same sample video frame image number and the same sample video frame pixel measurement sampling image number is searched out. And generate sample video frame pixel measurement sample image pixel density extreme value data set through data identification ,in Represents the sample video data to All the decomposition generated The sample video frame pixel measurement sample image pixel density extreme value data set corresponding to the sample video frame image data, Represents the sample video data to The decomposition generated The sample video frame image data The sample video frame pixel measurement sample image pixel density extreme value data corresponding to the sample video frame pixel measurement sample image data, wherein The unit is PPI; S52: measuring the sample video frame pixels and sampling the image pixel density extreme value data set The sample video frame pixel measurement sample image pixel density extreme value data A sampled image data set with pixel measurements of the sample video frames The sample video frame pixel measurements sample image data According to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, character matching is performed to search for the sample video frame pixel measurement sampling image pixel density extreme value data The sample video frame pixel measurement sample image data corresponding to And construct the optimal sampling image data set for sample video frame pixel measurement ,in Represents the sample video frame pixel measurement sample image pixel density extreme value data set The corresponding sample video frame pixel measurement optimal sampling image data set, Indicates the sample video frame pixel measurement sample image pixel density extreme value data Corresponding sample video frame pixel measurement optimal sampling image data; executing to construct a sample video frame pixel measurement optimal sampling image data set The specific steps are as follows: S521, initialization, defining the relevant structural parameters as vectors, measuring the sampled image data set in the sample video frame pixels constitute In the dimensional optimization problem, the image recognition sand cat represents the 1× Array, each variable value is a floating point number, each variable value to is a set of sampled image data located at the pixel measurement of the sample video frame Between lower and upper bounds in the search space; S522, Searching for prey, the final and main parameters controlling the transition between exploration and exploitation phases are ,when >1, the image recognition sand cat measures the sampled image data set in the sample video frame pixel The sample video frame pixel measurement sample image pixel density extreme value data is searched in the space Sample image data with pixel measurements of the sample video frame Character matching is performed according to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, and the image recognition sand cat is in the sample video frame pixel measurement sampling image data set The search space relies on the release of low-frequency noise to search for extreme value data of the sampled image pixel density measured by the sample video frame pixel The sampled image data is matched to the sample video frame pixel measurements , assuming the sensitivity range of image recognition sand cat is From 0 to 2kHz, Inspired by the auditory features of sand cats in image recognition, assuming The value is 2, is the current iteration number, is the maximum number of iterations, rand(0,1) represents a random number ranging from 0 to 1; , ,in Represents the sensitivity vector; each image identifies the sand cat according to the best candidate position and the current position and its sensitivity range Update your own position, i.e., measure the sampled image data set in the sample video frame pixels Searching for extreme value data of sampled image pixel density corresponding to the sample video frame pixel measurement in the search space The sampled image data that best matches the sample video frame pixel measurements The position is calculated as follows: ,in Indicates the sampled image data set of the pixel measurement of the sample video frame in the t+1th iteration image recognition sand cat individual in the prey search phase The current position in the search space, It represents the sampling image data set of the t-th iteration image recognition sand cat in the prey search phase, measured in pixels in the sample video frame The best candidate position in the search space; S523, attack prey, when ≤1, the image recognition sand cat measures the sampled image data set in the sample video frame pixel Searching in the search space, using the best candidate position and the current position to generate a random position, that is, measuring the sampled image data set in the sample video frame pixel Randomly search for extreme value data of sampled image pixel density corresponding to the sample video frame pixel measurement in the search space The sampled image data is matched to the sample video frame pixel measurements Assuming that the sensitivity range of the image recognition sand cat is a circle, the roulette method is used to randomly select an angle for each image recognition sand cat. , and calculate the random position according to the formula: Perform pixel measurement of the sample video frame on the sample image data set Search at random positions in the search space, where It represents the sampled image data set of the individual sand cat in the sample video frame pixel measurement in the t+1th iteration image recognition in the attack prey phase Search for a random position in the space, represents the sensitivity vector, Indicates angle The cosine value of Represents the image data set of the sampled image of the sand cat in the attack prey phase by measuring the pixel of the sample video frame The random position in the search space makes the image recognition sand cat approach and attack the prey, that is, the sampled image data set is measured in pixels of the sample video frame. Searching for extreme value data of sampled image pixel density corresponding to the sample video frame pixel measurement in the search space The sampled image data is matched to the sample video frame pixel measurements ; S524: When the algorithm meets the maximum number of iterations, output the sampled image pixel density extreme value data corresponding to the sample video frame pixel measurement The sampled image data that best matches the sample video frame pixel measurements , otherwise continue to iterate until the maximum number of iterations is met; S525, measuring the sample video frame pixel density extreme value data of the sampled image output in step S524 The sampled image data that best matches the sample video frame pixel measurements And construct the optimal sampling image data set for sample video frame pixel measurement .
[0015] The present invention combines the pixel density parameters of the sampled image measured by the pixel number of the sample video frame with the width-first search algorithm to intelligently screen out the pixel density extreme value parameters of the sampled image measured by the pixel number of the sample video frame, and simultaneously adopts the sand cat optimization algorithm to intelligently screen out the optimal sampling image parameters of the sample video frame pixel measurement, so as to achieve the effect of improving the accuracy of video super-resolution reconstruction.
[0016] Preferably, the steps of performing optimal pixel image combination reconstruction data processing of the sample video frame number based on the optimal sampling image data of the sample video frame pixel measurement to generate the optimal image data of the sample video frame are as follows: S61: Obtaining the optimal sampling image data set for pixel measurement of the sample video frame ; S62: Measure the optimal sampling image data set of the sample video frame pixels The sample video frame pixel measurement optimal sampling image data According to the sample video frame image number and the sample video frame pixel measurement sample image number, the sample images are combined and reconstructed in order to generate the optimal image data of the sample video frame ,in Indicates The optimal image data for each sample video frame.
[0017] The present invention achieves the effect of efficiently and accurately generating optimal image parameters of sample video frames by independently performing optimal pixel image combination reconstruction data processing of sample video frames.
[0018] Preferably, the steps of constructing the super-resolution video reconstruction sample image data and performing the super-resolution video reconstruction operation to generate the super-resolution sample video data are as follows: S71, the sample video frame optimal image data The sample video frame optimal image data Perform data identification to construct a super-resolution video reconstruction sample image data set ,in Indicates Super-resolution video reconstruction sample image data; S72: Reconstructing a sample image data set from the super-resolution video The super-resolution video reconstruction sample image data According to the sample video frame pixel measurement, the sample image number and the sample video set time axis parameters are imported into the video editing software to perform the super-resolution video combination and reconstruction operation and generate super-resolution sample video data .
[0019] The present invention constructs super-resolution video reconstruction sample image parameters based on data processing science, and uses video editing software to accurately and efficiently perform super-resolution video combination reconstruction operations and generate super-resolution sample videos, thereby achieving the effect of efficient and high-quality generation of super-resolution videos.
[0020] A video super-resolution reconstruction system based on deep learning, used to implement the video super-resolution reconstruction method, the system includes a video frame decomposition module, a video frame optimization processing module, and a video super-resolution reconstruction module; The video frame decomposition module includes a sample video acquisition unit, a sample video frame image generation unit, and a sample video frame pixel measurement sampling image generation unit; The sample video acquisition unit acquires sample video data through a video lens; the sample video frame image generation unit performs video frame number decomposition processing of the sample video according to the sample video data to generate sample video frame image data; the sample video frame pixel measurement sampling image generation unit performs image pixel measurement sampling object grid division processing of the sample video frame number based on the sample video frame image data to generate sample video frame pixel measurement sampling image data; The video frame optimization processing module includes a sample video frame pixel measurement sampling image pixel density measurement unit, a sample video frame pixel measurement sampling image pixel density maximum value search unit, an optimal sample video frame pixel measurement sampling image analysis unit, and an optimal sample video frame image generation unit; The sample video frame pixel measurement sampling image pixel density measurement unit performs pixel density numerical measurement processing of the sample video frame number pixel measurement sampling image according to the sample video frame pixel measurement sampling image data to generate the sample video frame pixel measurement sampling image pixel density data; the sample video frame pixel measurement sampling image pixel density maximum value search unit performs pixel density maximum value search processing of the sample video frame number pixel measurement sampling image according to the sample video frame pixel measurement sampling image pixel density data based on the deep learning algorithm to generate the sample video frame pixel density extreme value data; the optimal sample video frame pixel measurement sampling image analysis unit performs optimal pixel measurement sampling image object analysis processing of the sample video frame number according to the sample video frame pixel measurement sampling image pixel density extreme value data combined with the sample video frame pixel measurement sampling image data to construct the sample video frame pixel measurement optimal sampling image data; the optimal sample video frame image generation unit performs optimal pixel image combination reconstruction data processing of the sample video frame number based on the sample video frame pixel measurement optimal sampling image data to generate the sample video frame optimal image data; The video super-resolution reconstruction module includes a video super-resolution reconstruction parameter acquisition unit and a video super-resolution reconstruction operation execution unit; The video super-resolution reconstruction parameter acquisition unit is used to construct super-resolution video reconstruction sample image data; the video super-resolution reconstruction operation execution unit executes a super-resolution video reconstruction operation based on the super-resolution video reconstruction sample image data to generate super-resolution sample video data.
[0021] (III) Beneficial effects The present invention provides a video super-resolution reconstruction system and method based on deep learning. It has the following beneficial effects: 1. Through the sample video acquisition unit, a video lens is used to efficiently collect sample video parameters online, providing data support for the decomposition of the sample video to generate video frame images; the sample video frame image generation unit performs efficient decomposition and processing of the video frame number of the sample video based on the sample video parameters in combination with the video editing software, so as to realize the accurate and autonomous generation of the sample video frame image; the sample video frame pixel measurement sampling image generation unit performs scientific grid division processing of the image pixel measurement sampling object on the sample video frame image, realizes the precise digital acquisition of the sample video frame pixel measurement sampling image, and realizes the standardized operation of video data processing for video super-resolution reconstruction.
[0022] 2. Through the cooperation of the sample video frame pixel measurement sampling image pixel density measurement unit and the sample video frame pixel measurement sampling image pixel density maximum search unit, the image measurement software is used to accurately perform the sample video frame pixel measurement sampling image pixel density numerical measurement processing and combine the deep learning algorithm to intelligently screen out the sample video frame pixel measurement sampling image pixel density extreme value parameters, so as to provide data support for the sample video to screen out super-resolution sampling images and improve the accuracy of video super-resolution reconstruction; the optimal sample video frame pixel measurement sampling image analysis unit and the optimal sample video frame image generation unit cooperate with each other to scientifically and efficiently screen out the optimal sampling image parameters of the sample video frame pixel measurement and perform the optimal pixel image combination reconstruction data processing of the sample video frame number, so as to achieve efficient and accurate generation of the optimal image parameters of the sample video frame and improve the quality and efficiency of video super-resolution reconstruction.
[0023] 3. Through the cooperation of the video super-resolution reconstruction parameter acquisition unit and the video super-resolution reconstruction job execution unit, super-resolution video reconstruction sample image parameters are scientifically constructed based on data processing. At the same time, video editing software is used to accurately and efficiently execute super-resolution video combination reconstruction jobs and generate super-resolution sample videos, thereby achieving efficient and high-quality generation of super-resolution videos and improving the reliability and applicability of video super-resolution reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 A schematic diagram of the modules of the deep learning-based video super-resolution reconstruction system provided by the present invention; Figure 2 The present invention provides a flowchart of a video super-resolution reconstruction method. DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0026] The embodiments of the video super-resolution reconstruction system and method based on deep learning are as follows: Embodiment 1:
[0027] See also Figure 1 - Figure 2 , a video super-resolution reconstruction method, the method comprising the following steps: S1, collect sample video data; S2, performing video frame decomposition processing on the sample video according to the sample video data to generate sample video frame image data; S3, performing a grid division process on the image pixel measurement sampling object of the sample video frame number based on the sample video frame image data, and generating the sample video frame pixel measurement sampling image data; S4, performing pixel density numerical measurement processing of the sample video frame pixel measurement sampling image according to the sample video frame pixel measurement sampling image data, and generating the sample video frame pixel measurement sampling image pixel density data; S5, performing pixel density maximum value search processing of the sample video frame number pixel measurement sampling image according to the sample video frame pixel measurement sampling image pixel density data, generating the sample video frame pixel measurement sampling image pixel density extreme value data and performing optimal pixel measurement sampling image object analysis processing of the sample video frame number in combination with the sample video frame pixel measurement sampling image data, constructing the sample video frame pixel measurement optimal sampling image data; S6, performing optimal pixel image combination reconstruction data processing of the sample video frame number based on the optimal sampling image data of the sample video frame pixel measurement to generate optimal image data of the sample video frame; S7. Construct super-resolution video reconstruction sample image data and perform super-resolution video reconstruction operations to generate super-resolution sample video data.
[0028] For further information, see Figure 1 - Figure 2 ,The steps for collecting sample video data are as follows: S11, repeatedly collecting video data of the video shooting object online through the video lens and generating a sample video data set , ;in Indicates the number of repeated collections Sample video data, Indicates the maximum number of sample videos.
[0029] The steps of performing video frame decomposition processing on the sample video according to the sample video data and generating sample video frame image data are as follows: S21, using video editing software to set the sample video data set according to the time axis of the sample video Sample video data Decompose the sample video into individual sample video frames in order according to the sample video number, and generate a sample video frame image data set , ,in Indicates the maximum number of sample video frame images. Represents sample video data The decomposition generated Sample video frame image data; Represents sample video data The decomposition generated Sample video frame image data; Represents sample video data The decomposition generated Sample video frame image data; Represents sample video data The decomposition generated Sample video frame image data; Represents sample video data The decomposition generated Sample video frame image data; Represents sample video data The decomposition generated Sample video frame image data; Represents sample video data The decomposition generated Sample video frame image data; Represents sample video data The decomposition generated Sample video frame image data; Represents sample video data The decomposition generated Sample video frame image data, the video editing software includes any one of Adobe Premiere Pro, Media DreamWorks, and Fotor.
[0030] The steps of performing the grid division processing of the image pixel measurement sampling object of the sample video frame number based on the sample video frame image data and generating the sample video frame pixel measurement sampling image data are as follows: S31, using a square with a side length of L to collect the sample video frame image data Sample video frame image data According to the sample video number and the sample video frame image number, the image grid division processing of the sample video frame image pixel measurement sampling object is carried out in order to generate a sample video frame pixel measurement sampling image data set , ,in Indicates The first The sample video frame image data is divided into image grids to generate a sample video frame pixel measurement sampling image data set, Represents a sample video frame pixel measurement sampled image data set Middle Sample video frame pixel measurement sampled image data, Indicates the maximum number of sampled images for pixel measurement of sample video frames.
[0031] Through the sample video acquisition unit, video lenses are used to efficiently collect sample video parameters online, providing data support for sample video decomposition to generate video frame images; the sample video frame image generation unit, based on the sample video parameters combined with video editing software, performs efficient video frame decomposition and processing of the sample video, to achieve accurate and autonomous generation of sample video frame images; the sample video frame pixel measurement sampling image generation unit performs scientific grid division processing on the sample video frame images, to achieve precise digital acquisition of sample video frame pixel measurement sampling images, and to achieve standardized operations for video data processing for video super-resolution reconstruction.
[0032] For further information, see Figure 1 - Figure 2 , the pixel density value measurement processing of the sample video frame pixel measurement sampling image is performed according to the sample video frame pixel measurement sampling image data, and the operation steps of generating the sample video frame pixel measurement sampling image pixel density data are as follows: S41, using image measurement software to measure the sample video frame pixels and sample image data set Sample video frame pixel measurement sampled image data According to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, the pixel density numerical measurement processing of the sample video frame pixel measurement sampling image is performed in order, and a sample video frame pixel measurement sampling image pixel density data set is generated ,in Indicates The first A sample video frame pixel measurement sampling image pixel density data set corresponding to the sample video frame image data, Represents sample video frame pixel measurement sampled image data The corresponding sample video frame pixel measurement sampled image pixel density data, The unit is PPI. Image measurement software includes any one of MediaInfo, ImageMeter, and Photoshop.
[0033] According to the sample video frame pixel measurement sampling image pixel density data, the pixel density maximum value search processing of the sample video frame number pixel measurement sampling image is performed to generate the sample video frame pixel measurement sampling image pixel density extreme value data and combine the sample video frame pixel measurement sampling image data to perform the optimal pixel measurement sampling image object analysis processing of the sample video frame number, and construct the sample video frame pixel measurement optimal sampling image data. The operation steps are as follows: S51, using a breadth-first search algorithm to measure the sample video frame pixel sample image pixel density data set Sample video frame pixel measurement sampled image pixel density data According to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, the pixel density values of the sample video frame pixel measurement sampling image are compared in order, and the sample video frame pixel measurement sampling image pixel density data with the largest value corresponding to different sample video numbers, the same sample video frame image number and the same sample video frame pixel measurement sampling image number is searched out. And generate sample video frame pixel measurement sample image pixel density extreme value data set through data identification ,in Represents sample video data to All the decomposition generated The sample video frame pixel measurement sample image pixel density extreme value data set corresponding to the sample video frame image data, Represents sample video data to The decomposition generated The sample video frame image data The sample video frame pixel measurement sample image pixel density extreme value data corresponding to the sample video frame pixel measurement sample image data, wherein The unit is PPI; S52: Measure the sample video frame pixels and sample the image pixel density extreme value data set Sample video frame pixel measurement sample image pixel density extreme value data Sampled image data set with sample video frame pixel measurements Sample video frame pixel measurement sampled image data According to the sample video number, sample video frame image number and sample video frame pixel measurement sampling image number, character matching is performed to search for the sample video frame pixel measurement sampling image pixel density extreme value data Corresponding sample video frame pixel measurement sampled image data And construct the optimal sampling image data set for sample video frame pixel measurement ,in Represents the sample video frame pixel measurement sample image pixel density extreme value data set The corresponding sample video frame pixel measurement optimal sampling image data set, Represents the sample video frame pixel measurement sample image pixel density extreme value data Corresponding sample video frame pixel measurement optimal sampling image data; executing to construct a sample video frame pixel measurement optimal sampling image data set The specific steps are as follows: S521, initialization, defining the relevant structural parameters as vectors, measuring the sampled image data set in the sample video frame pixels constitute In the dimensional optimization problem, the image recognition sand cat represents the 1× Array, each variable value is a floating point number, each variable value to is a set of sampled image data measured by pixel in a sample video frame Between lower and upper bounds in the search space; S522, Searching for prey, the final and main parameters controlling the transition between exploration and exploitation phases are ,when >1, the image recognition sand cat measures the sampled image data set in the sample video frame pixel In the search space, the sample video frame pixel measures the sample image pixel density extreme value data Sample image data with sample video frame pixel measurements Character matching is performed according to the sample video number, sample video frame image number and sample video frame pixel measurement sampling image number, and the image recognition sand cat is in the sample video frame pixel measurement sampling image data set The search space relies on the release of low-frequency noise to search for extreme value data of sampled image pixel density measured with sample video frame pixels. Match the sample video frame pixel measurements to the sampled image data , assuming the sensitivity range of image recognition sand cat is From 0 to 2kHz, Inspired by the auditory features of sand cats in image recognition, assuming The value is 2, is the current iteration number, is the maximum number of iterations, rand(0,)) represents a random number ranging from 0 to 1; , ,in Represents the sensitivity vector; each image identifies the sand cat according to the best candidate position and the current position and its sensitivity range Update your own position, that is, measure the sampled image data set in the sample video frame pixel Search the search space to find the extreme value data of the sampled image pixel density measured with the sample video frame pixel Best matching sample video frame pixel measurement sampled image data The position is calculated as follows: ,in It represents the image data set of the pixel measurement sampling of the sand cat individual in the sample video frame in the t+1th iteration of the image recognition in the prey search phase. The current position in the search space, represents the image data set sampled by the pixel measurement of the sample video frame in the t-th iteration of the image recognition sand cat in the prey search phase The best candidate position in the search space; S523, attack prey, when ≤1, the image recognition sand cat is measured in the sample video frame pixel sampling image data set Search in the search space and use the best candidate position and the current position to generate a random position, that is, measure the sampled image data set in the sample video frame pixel Randomly search for the extreme value data of the sampled image pixel density measured with the sample video frame pixel in the search space Match the sample video frame pixel measurements to the sampled image data Assuming that the sensitivity range of the image recognition sand cat is a circle, the roulette method is used to randomly select an angle for each image recognition sand cat. , and follow the random position calculation formula: Perform pixel measurements on sample video frames to sample image data sets Search at random positions in the search space, where It represents the image data set of the sampled image measured by pixel in the sample video frame in the t+1th iteration of image recognition of the sand cat individual in the attack prey phase. Search for a random position in the space, represents the sensitivity vector, Indicates angle The cosine value of Represents the image data set of the image recognition sand cat in the attack prey phase, measured in pixels in the sample video frame. The random position in the search space that makes the image recognition sand cat approach and attack the prey is a random position in the search space that makes the image recognition sand cat approach and attack the prey, that is, the sample image data set is measured in the sample video frame pixel Search the search space to find the extreme value data of the sampled image pixel density measured with the sample video frame pixel Match the sample video frame pixel measurements to the sampled image data ; S524: When the algorithm meets the maximum number of iterations, output the sampled image pixel density extreme value data measured with the sample video frame pixel Best matching sample video frame pixel measurement sampled image data , otherwise continue to iterate until the maximum number of iterations is met; S525, measuring the sample video frame pixel density extreme value data of the sampled image output in step S524 Best matching sample video frame pixel measurement sampled image data And construct the optimal sampling image data set for sample video frame pixel measurement .
[0034] Based on the sample video frame pixel measurement optimal sampling image data, the optimal pixel image combination reconstruction data processing of the sample video frame number is performed to generate the sample video frame optimal image data. The operation steps are as follows: S61. Obtaining an optimal sampling image data set for sample video frame pixel measurement ; S62: Measure the optimal sampling image data set for the sample video frame pixels Optimal sampling of image data for pixel measurement in sample video frames According to the sample video frame image number and the sample video frame pixel measurement sample image number, the sample images are combined and reconstructed in order to generate the optimal image data of the sample video frame ,in Indicates The optimal image data for each sample video frame.
[0035] Through the cooperation of the sample video frame pixel measurement sampling image pixel density measurement unit and the sample video frame pixel measurement sampling image pixel density maximum value search unit, the image measurement software is used to accurately perform the pixel density numerical measurement processing of the sample video frame number pixel measurement sampling image and the deep learning algorithm is combined to intelligently screen out the sample video frame pixel measurement sampling image pixel density extreme value parameters, so as to provide data support for the sample video to screen out super-resolution sampling images and improve the accuracy of video super-resolution reconstruction; the optimal sample video frame pixel measurement sampling image analysis unit and the optimal sample video frame image generation unit cooperate with each other to scientifically and efficiently screen out the optimal sampling image parameters of the sample video frame pixel measurement and perform the optimal pixel image combination reconstruction data processing of the sample video frame number, so as to achieve efficient and accurate generation of the optimal image parameters of the sample video frame and improve the quality and efficiency of video super-resolution reconstruction.
[0036] For further information, see Figure 1 - Figure 2 , construct super-resolution video reconstruction sample image data and perform super-resolution video reconstruction operations. The steps for generating super-resolution sample video data are as follows: S71, the sample video frame optimal image data Optimal image data for sample video frames Perform data identification to construct a super-resolution video reconstruction sample image data set ,in Indicates Super-resolution video reconstruction sample image data; S72: Reconstructing a sample image data set from the super-resolution video Super-resolution video reconstruction sample image data According to the sample video frame pixel measurement, the sample image number and the sample video set time axis parameters are imported into the video editing software to perform the super-resolution video combination and reconstruction operation and generate super-resolution sample video data .
[0037] Through the cooperation of the video super-resolution reconstruction parameter acquisition unit and the video super-resolution reconstruction job execution unit, super-resolution video reconstruction sample image parameters are scientifically constructed based on data processing. At the same time, video editing software is used to accurately and efficiently execute super-resolution video combination reconstruction jobs and generate super-resolution sample videos, thereby achieving efficient and high-quality generation of super-resolution videos and improving the reliability and applicability of video super-resolution reconstruction.
[0038] Embodiment 2:
[0039] See also Figure 1 - Figure 2 , a video super-resolution reconstruction system based on deep learning, used to implement a video super-resolution reconstruction method, the system includes a video frame decomposition module, a video frame optimization processing module, and a video super-resolution reconstruction module; The video frame decomposition module includes a sample video acquisition unit, a sample video frame image generation unit, and a sample video frame pixel measurement sampling image generation unit; The sample video acquisition unit acquires sample video data through a video lens; the sample video frame image generation unit performs video frame number decomposition processing of the sample video according to the sample video data to generate sample video frame image data; the sample video frame pixel measurement sampling image generation unit performs image pixel measurement sampling object grid division processing of the sample video frame number based on the sample video frame image data to generate sample video frame pixel measurement sampling image data; The video frame optimization processing module includes a sample video frame pixel measurement sampling image pixel density measurement unit, a sample video frame pixel measurement sampling image pixel density maximum value search unit, an optimal sample video frame pixel measurement sampling image analysis unit, and an optimal sample video frame image generation unit; A sample video frame pixel measurement sampling image pixel density measurement unit, which performs pixel density numerical measurement processing of the sample video frame number pixel measurement sampling image according to the sample video frame pixel measurement sampling image data, and generates the sample video frame pixel measurement sampling image pixel density data; a sample video frame pixel measurement sampling image pixel density maximum value search unit, which performs pixel density maximum value search processing of the sample video frame number pixel measurement sampling image according to the sample video frame pixel measurement sampling image pixel density data based on the deep learning algorithm, and generates the sample video frame pixel measurement sampling image pixel density extreme value data; an optimal sample video frame pixel measurement sampling image analysis unit, which performs optimal pixel measurement sampling image object analysis processing of the sample video frame number according to the sample video frame pixel measurement sampling image pixel density extreme value data combined with the sample video frame pixel measurement sampling image data, and constructs the sample video frame pixel measurement optimal sampling image data; an optimal sample video frame image generation unit, which performs optimal pixel image combination reconstruction data processing of the sample video frame number based on the sample video frame pixel measurement optimal sampling image data, and generates the sample video frame optimal image data; The video super-resolution reconstruction module includes a video super-resolution reconstruction parameter acquisition unit and a video super-resolution reconstruction job execution unit; The video super-resolution reconstruction parameter acquisition unit is used to construct super-resolution video reconstruction sample image data; the video super-resolution reconstruction operation execution unit executes the super-resolution video reconstruction operation according to the super-resolution video reconstruction sample image data to generate super-resolution sample video data.
[0040] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A video super-resolution reconstruction method, characterized in that: The method comprises the following steps: S1, collect sample video data; S2, performing video frame decomposition processing on the sample video according to the sample video data to generate sample video frame image data; S3, performing a grid division process on the image pixel measurement sampling object of the sample video frame number based on the sample video frame image data to generate sample video frame pixel measurement sampling image data; S4, performing pixel density numerical measurement processing of the sample video frame pixel measurement sampling image according to the sample video frame pixel measurement sampling image data, and generating the sample video frame pixel measurement sampling image pixel density data; S5, performing pixel density maximum value search processing of the sample video frame number pixel measurement sampling image according to the sample video frame pixel measurement sampling image pixel density data, generating sample video frame pixel measurement sampling image pixel density extreme value data and performing optimal pixel measurement sampling image object analysis processing of the sample video frame number in combination with the sample video frame pixel measurement sampling image data, constructing the sample video frame pixel measurement optimal sampling image data; S6, performing optimal pixel image combination reconstruction data processing of the sample video frame number based on the optimal sampling image data of the sample video frame pixel measurement to generate optimal image data of the sample video frame; S7. Construct super-resolution video reconstruction sample image data and perform super-resolution video reconstruction operations to generate super-resolution sample video data.
2. The video super-resolution reconstruction method according to claim 1, characterized in that: The S1 comprises the following steps: S11, repeatedly collecting video data of the video shooting object online through the video lens and generating a sample video data set , ;in Indicates the number of repeated collections Sample video data, Indicates the maximum number of sample videos.
3. The video super-resolution reconstruction method according to claim 2, characterized in that: The S2 comprises the following steps: S21, using video editing software to set the time axis according to the sample video As stated in Decompose the sample video into individual sample video frames in order according to the sample video number, and generate a sample video frame image data set ; Includes sample video frame image data ,in Indicates the The decomposition generated Sample video frame image data.
4. The video super-resolution reconstruction method according to claim 3, characterized in that: The S3 comprises the following steps: S31, using a square with a side length of L to As stated in According to the sample video number and the sample video frame image number, the image grid division processing of the sample video frame image pixel measurement sampling object is carried out in order to generate a sample video frame pixel measurement sampling image data set , Includes sample video frame pixel measurements sampled image data ,in Indicates The first The sample video frame image data is divided into image grids to generate a sample video frame pixel measurement sampling image data set, Indicates the Middle Sample video frame pixel measurements sample image data.
5. The video super-resolution reconstruction method according to claim 4, characterized in that: The S4 comprises the following steps: S41, using image measurement software to As stated in According to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, the pixel density numerical measurement processing of the sample video frame pixel measurement sampling image is performed in order, and a sample video frame pixel measurement sampling image pixel density data set is generated , include ,in Indicates The first A sample video frame pixel measurement sampling image pixel density data set corresponding to the sample video frame image data, Indicates the The corresponding sample video frame pixel measurement sampled image pixel density data, The unit is PPI.
6. The video super-resolution reconstruction method according to claim 5, characterized in that: The S5 comprises the following steps: S51, using a breadth-first search algorithm to As stated in According to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, the pixel density values of the sample video frame pixel measurement sampling image are compared in order, and the largest value corresponding to different sample video numbers, the same sample video frame image number and the same sample video frame pixel measurement sampling image number is searched out. And generate sample video frame pixel measurement sample image pixel density extreme value data set through data identification , include ,in Represents the sample video data to All the decomposition generated The sample video frame pixel measurement sample image pixel density extreme value data set corresponding to the sample video frame image data, Represents the sample video data to The decomposition generated The sample video frame image data The sample video frame pixel measurement sample image pixel density extreme value data corresponding to the sample video frame pixel measurement sample image data, wherein The unit is PPI; S52, the As stated in With the As stated in According to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, character matching is performed to search for the The corresponding And construct the optimal sampling image data set for sample video frame pixel measurement , include ,in Indicates the The corresponding sample video frame pixel measurement optimal sampling image data set, Indicates the Corresponding sample video frame pixel measurement optimal sampling image data; executing to construct a sample video frame pixel measurement optimal sampling image data set The specific steps are as follows: S521, initialization, defining the relevant structural parameters as vectors, in the constitute In the dimensional optimization problem, the image recognition sand cat represents the 1× Array, each variable value is a floating point number, each variable value to is located in the Between lower and upper bounds in the search space; S522, Searching for prey, the final and main parameters controlling the transition between exploration and exploitation phases are ,when >1, the image recognizes the sand cat in the The search space will be described With the Character matching is performed according to the sample video number, the sample video frame image number and the sample video frame pixel measurement sampling image number, and the image recognition sand cat is described in The search space relies on the release of low-frequency noise to search for the Matching the Each image recognizes the sand cat according to the best candidate position and current position and its sensitivity range Update your location, that is, Search the search space for The best match for location; S523, attack prey, when ≤1, the image recognition sand cat is described in Search in the search space and generate a random position using the best candidate position and the current position. Randomly search the search space for Matching the Assuming that the sensitivity range of the image recognition sand cat is a circle, the roulette method is used to randomly select an angle for each image recognition sand cat. , and perform the above according to the random position calculation formula Search at random positions in the search space; S524, when the algorithm meets the maximum number of iterations, output The best match for , otherwise continue to iterate until the maximum number of iterations is met; S525, the output in step S524 The best match for And construct the optimal sampling image data set for sample video frame pixel measurement .
7. The video super-resolution reconstruction method according to claim 6, characterized in that: The S6 comprises the following steps: S61, obtaining the ; S62, the As stated in According to the sample video frame image number and the sample video frame pixel measurement sample image number, the sample images are combined and reconstructed in order to generate the optimal image data set of the sample video frame. , include ,in Indicates The optimal image data for each sample video frame.
8. The video super-resolution reconstruction method according to claim 7, characterized in that: The S7 comprises the following steps: S71, the As stated in Perform data identification to construct a super-resolution video reconstruction sample image data set , include ,in Indicates Super-resolution video reconstruction sample image data; S72, the As stated in According to the sample video frame pixel measurement, the sample image number and the sample video set time axis parameters are imported into the video editing software to perform the super-resolution video combination and reconstruction operation and generate super-resolution sample video data .
9. A video super-resolution reconstruction system based on deep learning, used to implement the video super-resolution reconstruction method according to any one of claims 1 to 8, characterized in that: The system comprises a video frame decomposition module, a video frame optimization processing module and a video super-resolution reconstruction module.
Citation Information
Patent Citations
Video image super-resolution reconstruction method
CN103489173A
A video super-resolution reconstruction method based on depth learning
CN109102462A
Video super-resolution reconstruction method and system, terminal equipment and storage medium
CN115601238A
Video frame super-resolution reconstruction method and device and computer equipment
CN117058002A
High-voltage bushing partial discharge mode identification method and system based on Sodat group algorithm optimization
CN117291224A
Cited By
Intelligent traffic flow statistical analysis method and system
CN120673608A
