A method for repairing damaged videos

By acquiring three consecutive video frames, calculating optical flow data, and performing interpolation and binarization processing, random patches in traditional magnetic tape-stored videos are identified and repaired, thus improving video playback quality.

CN119107260BActive Publication Date: 2026-11-10CHENGDOU HUAQIYUN TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411132731.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2026-11-10
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

Video data stored on traditional magnetic tapes develops random patches after aging or damage, affecting video playback quality.

Method used

By acquiring three consecutive video frames, optical flow data is calculated to estimate a reference image, and interpolation and binarization are performed. Damaged areas are identified using connected component labeling, and the estimated pixel values ​​are used for repair.

Benefits of technology

It effectively fixed the random patch issue in the video, improving the viewing quality.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

This invention discloses a video damage repair method, comprising the following steps: Step 1: Acquire three consecutive video images according to a time sequence, and estimate a second reference image based on the first and third video images; Step 2: Calculate the difference between the second video image and the second reference image and obtain a difference image; Step 3: Binarize the difference image to obtain a binarized image; Step 4: Mark connected components in the binarized image, discarding regions with a connected component area less than 10, to obtain a connected component marked image; Step 5: Use the connected component marked image to determine whether a frame contains random damaged areas; for the found damaged areas, use the corresponding pixel values ​​in the image estimated in Step 1 as the repaired pixel values. This invention can effectively solve the problem of random patches appearing in video images stored on traditional magnetic tapes due to tape aging or damage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of video restoration technology, specifically relating to a method for repairing video damage. Background Technology

[0002] A hard disk is a storage device that uses magnetic recording technology to store data. Its core function is to store large amounts of binary data and retain the data even after power is lost. However, media such as magnetic tapes used to store video data can develop random scratches during storage, resulting in random scratches and damaged patches when the video is played back, which greatly affects the viewing quality and the value of the video. Summary of the Invention

[0003] The purpose of this invention is to provide a video damage repair method that can repair randomly damaged patches in a video.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A method for repairing damaged videos includes the following steps:

[0006] Step 1: Obtain three consecutive video images P1 to P3 according to the time sequence, and estimate the reference image P2G of the second frame based on the first video image P1 and the third video image P3;

[0007] Step 2: Calculate the difference between the second frame video image P2 and the second frame reference image P2G, and obtain the difference image P2D;

[0008] Step 3: Perform P2D binarization on the difference image to obtain a binarized image P2DB, where the pixel values ​​of the regions with large errors are changed to 255, and the color of the regions with small errors is set to 0.

[0009] Step 4: Perform connected component labeling on the binarized image P2DB, discarding regions with a connected component area less than 10, to obtain the connected component labeled image P2DL;

[0010] Step 5: Use the connected component labeling image P2DL to determine whether a frame contains a random damaged region; for the found damaged region, use the corresponding pixel value in the image P2G estimated in Step 1 as the repaired pixel value.

[0011] Furthermore, the specific steps for obtaining the second frame reference image P2G in step 1 are as follows:

[0012] Step 1-1: Calculate the forward optical flow data OF13 from the first video image P1 to the third video image P3, and the backward optical flow data OF31 from the third video image P3 to the first video image P1.

[0013] Steps 1-2: Estimate the first image data of the second frame video image P2 based on the forward optical flow data OF13; and estimate the second image data of the second frame video image P2 based on the forward optical flow data OF31;

[0014] Steps 1-3: Estimate the second frame reference image P2G based on the first image data and the second image data.

[0015] Furthermore, the first image data estimated from the forward optical flow data OF13 for the second frame video image P2 refers to calculating the position to which the second frame video image P2 will move in the next frame by each point (p1x, p1y) on the first frame video image P1.

[0016] of13x=p1x+OF13(p1x,p1y).x×0.5,

[0017] of13y=p1y+OF13(p1x,p1y).y×0.5,

[0018] The image data of the second frame estimated from the forward optical flow data is as follows:

[0019] P2F(of13x,of13y)=P1(p1x,p1y).

[0020] Furthermore, estimating the first image data of the second frame video image P2 based on the backward optical flow data OF13 means calculating the position reached in the previous frame for each point (p3x, p3y) on the third frame video image P3:

[0021] of31x=p3x-OF31(p3x,p3y).x×0.5,

[0022] of31y=p3y-OF31(p3x,p3y).y×0.5,

[0023] The image data of the second frame estimated from the backward optical flow data is as follows:

[0024] P2B(of13x,of13y)=P3(p3x,p3y).

[0025] Furthermore, based on the first image data and the second image data, the second frame reference image P2G is estimated to refer to:

[0026] If the point (x,y) on the second frame of the video image has a calculated value in both the first image data P2F and the second image data P2B, then P2G(x,y) = P2F(x,y) × 0.5 + P2B(x,y) × 0.5;

[0027] If (x,y) has a calculated value only in the first image data P2F, then P2G(x,y) = P2F(x,y);

[0028] If (x,y) has a calculated value only in the second image data P2B, then P2G(x,y) = P2B(x,y);

[0029] If (x,y) has no value in the first image data P2F and the second image data P2B: P2G(x,y) takes the pixel value of the point with a calculated value that is closest to the pixel position (x,y) in the first image data P2F and the second image data P2B.

[0030] Furthermore, in step 5, the connected component labeling image P2DL is used to determine whether a frame contains random damaged regions, as detailed below:

[0031] If only the first frame of video image exists in two adjacent connected component labeled images, it indicates that the corresponding region in the current frame is damaged;

[0032] If there are damaged regions at the same location in two adjacent connected component labeled images, it indicates that there is a damaged region at the corresponding location in the next frame of the current frame.

[0033] If there are no damaged regions in the current connected component labeled image, it means that the two adjacent frames before and after it, as well as the current frame, are all normal.

[0034] This invention can effectively solve the problem of random patches appearing in video footage stored on traditional magnetic tapes due to tape aging or damage. Detailed Implementation

[0035] This embodiment provides a video damage repair method including the following steps:

[0036] Step 1: Obtain three consecutive video images P1 to P3 according to the time sequence, and estimate the reference image P2G of the second frame based on the first video image P1 and the third video image P3;

[0037] Step 2: Calculate the difference between the second frame video image P2 and the second frame reference image P2G, and obtain the difference image P2D;

[0038] Step 3: Perform P2D binarization on the difference image to obtain a binarized image P2DB, where the pixel values ​​of the regions with large errors are changed to 255, and the color of the regions with small errors is set to 0.

[0039] Step 4: Perform connected component labeling on the binarized image P2DB, discarding regions with a connected component area less than 10, to obtain the connected component labeled image P2DL. If there are still large connected components remaining in this labeled image, these connected components are potentially damaged regions.

[0040] Step 5: Use the connected component labeling image P2DL to determine whether a frame contains a random damaged region; for the found damaged region, use the corresponding pixel value in the image P2G estimated in Step 1 as the repaired pixel value.

[0041] The specific steps for obtaining the second frame reference image P2G in step 1 are as follows:

[0042] Step 1-1: Calculate the forward optical flow data OF13 from the first video image P1 to the third video image P3, and the backward optical flow data OF31 from the third video image P3 to the first video image P1.

[0043] Steps 1-2: Estimate the first image data of the second frame video image P2 based on the forward optical flow data OF13; and estimate the second image data of the second frame video image P2 based on the forward optical flow data OF31;

[0044] Steps 1-3: Estimate the second frame reference image P2G based on the first image data and the second image data.

[0045] The first image data of the second frame video image P2 estimated based on the forward optical flow data OF13 refers to calculating the position of each point (p1x, p1y) on the first frame video image P1 in the next frame:

[0046] of13x=p1x+OF13(p1x,p1y).x×0.5,

[0047] of13y=p1y+OF13(p1x,p1y).y×0.5,

[0048] The image data of the second frame estimated from the forward optical flow data is as follows:

[0049] P2F(of13x,of13y)=P1(p1x,p1y).

[0050] In the above formula, OF13(p1x,p1y).x and OF13(p1x,p1y).y are the components of the forward optical flow data in the x and y directions, respectively.

[0051] The first image data estimated based on the backward optical flow data OF13 for the second frame video image P2 refers to calculating the position of each point (p3x, p3y) in the previous frame for the third frame video image P3:

[0052] of31x=p3x-OF31(p3x,p3y).x×0.5,

[0053] of31y=p3y-OF31(p3x,p3y).y×0.5,

[0054] The image data of the second frame estimated from the backward optical flow data is as follows:

[0055] P2B(of13x,of13y)=P3(p3x,p3y).

[0056] In the above formula, OF31(p3x,p3y).x and OF31(p3x,p3y).y are the components of the backward optical flow data in the x and y directions, respectively.

[0057] Based on the first image data and the second image data, the second frame reference image P2G is estimated to refer to:

[0058] If the point (x,y) on the second frame of the video image has a calculated value in both the first image data P2F and the second image data P2B, then P2G(x,y) = P2F(x,y) × 0.5 + P2B(x,y) × 0.5;

[0059] If (x,y) has a calculated value only in the first image data P2F, then P2G(x,y) = P2F(x,y);

[0060] If (x,y) has a calculated value only in the second image data P2B, then P2G(x,y) = P2B(x,y);

[0061] If (x,y) has no value in the first image data P2F and the second image data P2B: P2G(x,y) takes the pixel value of the point with a calculated value that is closest to the pixel position (x,y) in the first image data P2F and the second image data P2B.

[0062] In step 5, the connected component labeling image P2DL is used to determine whether a frame contains random damaged regions, as detailed below:

[0063] If only the first frame of video image is a potentially corrupted region in two adjacent connected component marker graphs, it indicates that the corresponding region in the current frame is corrupted.

[0064] If two adjacent connected component maps both have damaged regions at the same location, it indicates that there is a damaged region at the corresponding location in the next frame of the current frame.

[0065] If there are no damaged regions in the current connected component marker graph, it means that the two adjacent frames before and after it, as well as the current frame, are all normal.

[0066] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications and substitutions based on the technical solutions and inventive concepts provided by the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for repairing video damage, characterized in that, Includes the following steps: Step 1: Obtain three consecutive video images P1~P3 according to the time sequence, and estimate the reference image P2G of the second frame based on the first video image P1 and the third video image P3; The specific steps for obtaining the second frame reference image P2G are as follows: Step 1-1: Calculate the forward optical flow data OF13 from the first video image P1 to the third video image P3, and the backward optical flow data OF31 from the third video image P3 to the first video image P1. Steps 1-2: Estimate the first image data of the second frame video image P2 based on the forward optical flow data OF13; and estimate the second image data of the second frame video image P2 based on the backward optical flow data OF31; Steps 1-3: Estimate the second frame reference image P2G based on the first image data and the second image data; Step 2: Calculate the difference between the second frame video image P2 and the second frame reference image P2G, and obtain the difference image P2D; Step 3: Perform P2D binarization on the difference image to obtain a binarized image P2DB, where the pixel values ​​of the regions with large errors are changed to 255, and the color of the regions with small errors is set to 0. Step 4: Perform connected component labeling on the binarized image P2DB, discarding regions with a connected component area less than 10, to obtain the connected component labeled image P2DL; Step 5: Use the connected component labeling image P2DL to determine whether a frame contains a random damaged region; for the found damaged region, use the corresponding pixel value in the image P2G estimated in Step 1 as the repaired pixel value.

2. The video damage repair method according to claim 1, characterized in that, The first image data estimated from the forward optical flow data OF13 for the second frame video image P2 refers to calculating the position to which the second frame video image P2 will move in the next frame by each point (p1x, p1y) on the first frame video image P1. of13x=p1x+OF13(p1x,p1y).x×0.5, of13y=p1y+OF13(p1x,p1y).y×0.5, The image data of the second frame estimated from the forward optical flow data is P2F(of13x,of13y).

3. The video damage repair method according to claim 2, characterized in that, The second image data estimated based on the backward optical flow data OF31 for the second frame video image P2 refers to calculating the position of each point (p3x, p3y) in the previous frame for each point (p3x, p3y) in the third frame video image P3. of31x=p3x-OF31(p3x,p3y).x×0.5, of31y=p3y-OF31(p3x,p3y).y×0.5, The image data of the second frame estimated from the backward optical flow data is P2B(of31x,of31y).

4. The video damage repair method according to claim 3, characterized in that, Based on the first image data and the second image data, the second frame reference image P2G is estimated to refer to: If the point (x,y) on the second frame of the video image has a calculated value in both the first image data P2F and the second image data P2B, then P2G(x,y) = P2F(x,y) × 0.5 + P2B(x,y) × 0.5; If (x,y) has a calculated value only in the first image data P2F, then P2G(x,y) = P2F(x,y); If (x,y) has a calculated value only in the second image data P2B, then P2G(x,y) = P2B(x,y); If (x,y) has no value in the first image data P2F and the second image data P2B: P2G(x,y) takes the pixel value of the point with a calculated value that is closest to the pixel position (x,y) in the first image data P2F and the second image data P2B.

5. The video damage repair method according to claim 1, characterized in that, In step 5, the connected component labeling image P2DL is used to determine whether a frame contains random damaged regions, as detailed below: If only the first frame of video image exists in two adjacent connected component labeled images, it indicates that the corresponding region in the current frame is damaged; If there are damaged regions at the same location in two adjacent connected component labeled images, it indicates that there is a damaged region at the corresponding location in the next frame of the current frame. If there are no damaged regions in the current connected component labeled image, it means that the two adjacent frames before and after it, as well as the current frame, are all normal.

Citation Information

Patent Citations

  • Digital video file restoration method and device

    CN111127376A

  • End-to-end video completion method and system based on optical flow guidance

    CN114782262A