Video damage repairing method

Abstract

The invention discloses a video damage repairing method, which comprises the following steps of: 1, acquiring three continuous frames of video images according to a time sequence, and estimating a second frame of reference image according to a first frame of video image and a third frame of video image; step 2, calculating a difference value between the second frame of video image and the second frame of reference image and obtaining a difference value image; step 3, carrying out binarization processing on the difference image to obtain a binarized image; step 4, performing connected domain marking on the binarized image, and abandoning areas with connected domain areas smaller than 10 to obtain a connected domain marked image; 5, judging whether a random damaged area appears in a certain frame or not by using the connected domain marked image; and for the found damaged area, using a corresponding pixel value in the image estimated in the step 1 as a restored pixel value. According to the invention, the problem that random patches appear in a video picture due to tape aging or tape damage of video data stored on a traditional tape can be effectively solved.

Description

Technical Field

[0001] The invention belongs to the technical field of video repair, and in particular relates to a method for repairing damaged videos. Background Art

[0002] Disk is a storage device that uses magnetic recording technology to store data. Its core function is to store large amounts of binary data and keep the data intact after a power outage. However, media such as tapes that store video data will be randomly scratched during storage, resulting in random scratches and damaged patches when the video is played, greatly affecting the viewing quality and video value. Summary of the invention

[0003] The purpose of the present invention is to provide a video damage repair method, which can repair random damaged patches in the video.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A video damage repair method comprises the following steps:

[0006] Step 1: Acquire three consecutive video frames P1 to P3 in time sequence, and estimate a second reference image P2G based on the first video frame P1 and the third video frame P3;

[0007] Step 2: Calculate the difference between the second frame video image P2 and the second frame reference image P2G and obtain the difference image P2D;

[0008] Step 3: Binarize the difference image P2D to obtain a binary image P2DB, where the pixel value of the area with large error is changed to 255, and the color of the area with small error is set to 0;

[0009] Step 4: Mark the connected domains of the binary image P2DB, discard the areas with connected domain areas less than 10, and obtain the connected domain marked image P2DL;

[0010] Step 5: Use the connected domain marker image P2DL to determine whether a random damaged area appears in a frame; for the damaged area found, use the corresponding pixel value in the image P2G estimated in step 1 as the pixel value after repair.

[0011] Furthermore, the specific steps of obtaining the second frame reference image P2G in step 1 are as follows:

[0012] Step 1-1: Calculate the forward optical flow data OF13 from the first frame video image P1 to the third frame video image P3, and the backward optical flow data OF31 from the third frame video image P3 to the first frame video image P1;

[0013] Step 1-2: estimating first image data of the second frame video image P2 according to the forward optical flow data OF13; and estimating second image data of the second frame video image P2 according to the forward optical flow data OF31;

[0014] Step 1-3: Estimate a second frame reference image P2G according to the first image data and the second image data.

[0015] Further, estimating the first image data of the second frame video image P2 according to the forward optical flow data OF13 means calculating the position to which each point (p1x, p1y) on the first frame video image P1 moves in the next frame as follows:

[0016] of13x=p1x+OF13(p1x,p1y).x×0.5,

[0017] of13y=p1y+OF13(p1x,p1y).y×0.5,

[0018] The image data of the second frame estimated by the forward optical flow data is:

[0019] P2F(of13x,of13y)=P1(p1x,p1y).

[0020] Further, estimating the first image data of the second frame video image P2 according to the backward optical flow data OF13 means calculating the position to which each point (p3x, p3y) on the third frame video image P3 moved in the previous frame as:

[0021] of31x=p3x-OF31(p3x,p3y).x×0.5,

[0022] of31y=p3y-OF31(p3x,p3y).y×0.5,

[0023] The image data of the second frame estimated by the backward optical flow data is:

[0024] P2B(of13x,of13y)=P3(p3x,p3y).

[0025] Further, estimating the second frame reference image P2G according to the first image data and the second image data means:

[0026] If the point (x, y) on the second frame video image has calculated values ​​in both the first image data P2F and the second image data P2B, then P2G (x, y) = P2F (x, y) × 0.5 + P2B (x, y) × 0.5;

[0027] If (x, y) has a calculated value only in the first image data P2F, then P2G(x, y) = P2F(x, y);

[0028] If (x, y) has a calculated value only in the second image data P2B, then P2G(x, y) = P2B(x, y);

[0029] If (x, y) has no value in the first image data P2F and the second image data P2B: P2G(x, y) takes the pixel value of the point with the shortest distance to the pixel position (x, y) in the first image data P2F and the second image data P2B with a calculated value.

[0030] Furthermore, in step 5, the connected domain marker image P2DL is used to determine whether a random damaged area appears in a frame, as follows:

[0031] If only the first frame of video image is included in two adjacent connected domain marker images, it indicates that the corresponding area on the current frame is damaged;

[0032] If there are damaged areas at the same position in two adjacent connected domain marker images, it means that there is a damaged area at the corresponding position in the next frame of the current frame;

[0033] If there is no damaged area in the current connected domain marker image, it means that the two frames before and after it and the current frame are normal.

[0034] The present invention can effectively solve the problem of random patches appearing in video images of video data stored on traditional magnetic tapes due to tape aging or tape damage. DETAILED DESCRIPTION

[0035] A video damage repair method provided in this embodiment includes the following steps:

[0036] Step 1: Acquire three consecutive video frames P1 to P3 in time sequence, and estimate a second reference image P2G based on the first video frame P1 and the third video frame P3;

[0037] Step 2: Calculate the difference between the second frame video image P2 and the second frame reference image P2G and obtain the difference image P2D;

[0038] Step 3: Binarize the difference image P2D to obtain a binary image P2DB, where the pixel value of the area with large error is changed to 255, and the color of the area with small error is set to 0;

[0039] Step 4: Mark the connected domains of the binary image P2DB, discard the areas with connected domain area less than 10, and obtain the connected domain marked image P2DL. If there are still connected domains with larger areas left in the connected domain marked image, these connected domains are possible damaged areas.

[0040] Step 5: Use the connected domain marker image P2DL to determine whether a random damaged area appears in a frame; for the damaged area found, use the corresponding pixel value in the image P2G estimated in step 1 as the pixel value after repair.

[0041] The specific steps for obtaining the second frame reference image P2G in step 1 are as follows:

[0042] Step 1-1: Calculate the forward optical flow data OF13 from the first frame video image P1 to the third frame video image P3, and the backward optical flow data OF31 from the third frame video image P3 to the first frame video image P1;

[0043] Step 1-2: estimating first image data of the second frame video image P2 according to the forward optical flow data OF13; and estimating second image data of the second frame video image P2 according to the forward optical flow data OF31;

[0044] Step 1-3: Estimate a second frame reference image P2G according to the first image data and the second image data.

[0045] Estimating the first image data of the second frame video image P2 according to the forward optical flow data OF13 means calculating the position to which each point (p1x, p1y) on the first frame video image P1 moves in the next frame as follows:

[0046] of13x=p1x+OF13(p1x,p1y).x×0.5,

[0047] of13y=p1y+OF13(p1x,p1y).y×0.5,

[0048] The image data of the second frame estimated by the forward optical flow data is:

[0049] P2F(of13x,of13y)=P1(p1x,p1y).

[0050] In the above formula, OF13(p1x, p1y).x and OF13(p1x, p1y).y are the components of the forward optical flow data in the x and y directions.

[0051] Estimating the first image data of the second frame video image P2 according to the backward optical flow data OF13 means calculating the position to which each point (p3x, p3y) on the third frame video image P3 moved in the previous frame as:

[0052] of31x=p3x-OF31(p3x,p3y).x×0.5,

[0053] of31y=p3y-OF31(p3x,p3y).y×0.5,

[0054] The image data of the second frame estimated by the backward optical flow data is:

[0055] P2B(of13x,of13y)=P3(p3x,p3y).

[0056] In the above formula, OF31(p3x, p3y).x and OF31(p3x, p3y).y are the components of the backward optical flow data in the x and y directions.

[0057] Estimating the second frame reference image P2G according to the first image data and the second image data means:

[0058] If the point (x, y) on the second frame video image has calculated values ​​in both the first image data P2F and the second image data P2B, then P2G (x, y) = P2F (x, y) × 0.5 + P2B (x, y) × 0.5;

[0059] If (x, y) has a calculated value only in the first image data P2F, then P2G(x, y) = P2F(x, y);

[0060] If (x, y) has a calculated value only in the second image data P2B, then P2G(x, y) = P2B(x, y);

[0061] If (x, y) has no value in the first image data P2F and the second image data P2B: P2G(x, y) takes the pixel value of the point with the shortest distance to the pixel position (x, y) in the first image data P2F and the second image data P2B with a calculated value.

[0062] In step 5, the connected domain marker image P2DL is used to determine whether a random damaged area appears in a frame, as follows:

[0063] If only the first frame of the video image has a possible damaged area in the two adjacent connected domain marker maps, it indicates that the corresponding area on the current frame is damaged;

[0064] If there are damaged areas at the same position in two adjacent connected domain marker maps, it means that there is a damaged area at the corresponding position in the next frame of the current frame;

[0065] If there is no damaged area in the current connected domain marker map, it means that the two frames before and after it and the current frame are normal.

[0066] The above description is only a preferred implementation manner of the present invention, but the protection scope of the present invention is not limited thereto, and any modification and replacement based on the technical solution and inventive concept provided by the present invention should be included in the protection scope of the present invention.

Claims

1. A video damage repair method, characterized in that: The steps include: Step 1: Acquire three consecutive frames of video images P1-P3 in time sequence, and estimate the second frame reference image P2G according to the first frame video image P1 and the third frame video image P3; Step 2: Calculate the difference between the second frame video image P2 and the second frame reference image P2G and obtain the difference image P2D; Step 3: Binarize the difference image P2D to obtain a binary image P2DB, where the pixel value of the area with large error is changed to 255, and the color of the area with small error is set to 0; Step 4: Mark the connected domains of the binary image P2DB, discard the areas with connected domain areas less than 10, and obtain the connected domain marked image P2DL; Step 5: Use the connected domain marker image P2DL to determine whether a random damaged area appears in a frame; for the damaged area found, use the corresponding pixel value in the image P2G estimated in step 1 as the pixel value after repair.

2. A video damage repair method according to claim 1, characterized in that: The specific steps for obtaining the second frame reference image P2G in step 1 are as follows: Step 1-1: Calculate the forward optical flow data OF13 from the first frame video image P1 to the third frame video image P3, and the backward optical flow data OF31 from the third frame video image P3 to the first frame video image P1; Step 1-2: estimating first image data of the second frame video image P2 according to the forward optical flow data OF13; and estimating second image data of the second frame video image P2 according to the forward optical flow data OF31; Step 1-3: Estimate a second frame reference image P2G according to the first image data and the second image data.

3. A video damage repair method according to claim 2, characterized in that: Estimating the first image data of the second frame video image P2 according to the forward optical flow data OF13 means calculating the position to which each point (p1x, p1y) on the first frame video image P1 moves in the next frame as follows: of13x=p1x+OF13(p1x,p1y).x×0.5, of13y=p1y+OF13(p1x,p1y).y×0.5, The image data of the second frame estimated by the forward optical flow data is: P2F(of13x,of13y)=P1(p1x,p1y).

4. A video damage repair method according to claim 3, characterized in that: Estimating the first image data of the second frame video image P2 according to the backward optical flow data OF13 means calculating the position to which each point (p3x, p3y) on the third frame video image P3 moved in the previous frame as: of31x=p3x-OF31(p3x,p3y).x×0.5, of31y=p3y-OF31(p3x,p3y).y×0.5, The image data of the second frame estimated by the backward optical flow data is: P2B(of13x,of13y)=P3(p3x,p3y).

5. A video damage repair method according to claim 4, characterized in that: Estimating the second frame reference image P2G according to the first image data and the second image data means: If the point (x, y) on the second frame video image has calculated values ​​in both the first image data P2F and the second image data P2B, then P2G (x, y) = P2F (x, y) × 0.5 + P2B (x, y) × 0.5; If (x, y) has a calculated value only in the first image data P2F, then P2G(x, y) = P2F(x, y); If (x, y) has a calculated value only in the second image data P2B, then P2G(x, y) = P2B(x, y); If (x, y) has no value in the first image data P2F and the second image data P2B: P2G(x, y) takes the pixel value of the point with the shortest distance to the pixel position (x, y) in the first image data P2F and the second image data P2B with a calculated value.

6. A video damage repair method according to claim 1, characterized in that: In step 5, the connected domain marker image P2DL is used to determine whether a random damaged area appears in a frame, as follows: If only the first frame of video image is included in two adjacent connected domain marker images, it indicates that the corresponding area on the current frame is damaged; If there are damaged areas at the same position in two adjacent connected domain marker images, it means that there is a damaged area at the corresponding position in the next frame of the current frame; If there is no damaged area in the current connected domain marker image, it means that the two frames before and after it and the current frame are normal.