Method for repairing picture quality reduction caused by packet loss in real-time video communication

Through the combination of intra- and inter-frame prediction, the problem of picture quality degradation caused by packet loss in real-time video communication is solved, and a more efficient image repair effect is achieved, avoiding the granularity and distortion problems in the prior art.

CN119094753AActive Publication Date: 2024-12-06CHINA UNICOM WO MUSIC & CULTURE CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411050803.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-12-06
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

The prior art packet loss in real-time video communication leads to a decline in picture quality. The existing filling methods such as the average or median filling of the surrounding pixels within the frame, filling of the same position of adjacent frames, and weighted predictions with graininess, distortion or weighted coefficients difficult to select. The problem.

Method used

By decoding the successfully received data packets, marking the macroblock area of ​​the lost data, performing intra and inter prediction, combining the neighbor area pixel values ​​and motion vectors, the intra and inter prediction pixel values ​​and confidence of the target area are calculated. , and perform fusion repair, and dynamically calculate the weighting coefficients to fill the lost area.

Benefits of technology

Effectively improve intra and inter prediction effects, improve information utilization, ensure the accuracy of repair results and image quality, and avoid graininess and distortion problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094753A_ABST
    Figure CN119094753A_ABST
Patent Text Reader

Abstract

The invention discloses a method for repairing picture quality reduction caused by packet loss in real-time video communication, which comprises the following steps of: S1, decoding a successfully received data packet, marking an image obtained by decoding as a current frame, and marking a macro block region of data loss of the current frame as a target region; s2, marking a neighbor region of the target region on the basis of the target region; s3, performing intra-frame prediction on the current frame to obtain an intra-frame prediction pixel value and an intra-frame certainty degree of the target area; s4, performing inter-frame prediction on the current frame to obtain an inter-frame prediction pixel value and an inter-frame certainty degree of the target area; and S5, fusion repair: calculating to obtain repaired pixels and repair certainty degree of the target area, and correspondingly filling each repaired pixel into the target area to obtain a repaired image. According to the method, packet loss error repair is carried out by utilizing the known macro block information to carry out comprehensive prediction, so that the effects of intra-frame prediction and inter-frame prediction can be effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a method for repairing image quality degradation caused by packet loss in real-time video communication. Background Art

[0002] As we all know, current communication technology is mainly based on data transmission. The speed and quality of data transmission have a direct impact on the actual communication effect between user terminals. For example, in common real-time video communication, packet loss may occur in the data transmission link due to various factors, resulting in a decrease in the quality of the user-side video image, causing a poor user experience for the user-side real-time video communication effect. To solve this problem, the existing conventional processing methods and their corresponding defects are: Solution 1: Fill the pixels of the lost macroblock area by the mean or median of the surrounding pixels in the frame. This method directly adopts the mean or median to fill the pixels, which will cause graininess in the color-rich area, causing the image quality to deteriorate significantly; Solution 2: Fill the pixels of the lost area of ​​the current frame macroblock by the pixel values ​​of the same position in the adjacent frames. This method adopts the solution of filling the pixel values ​​of the same area in the adjacent frames. It has a good effect on the static area or the area with slow changes in the video, but it will cause obvious distortion for the content moving faster, causing the image quality to deteriorate significantly; Solution 3: Fill the pixels of the lost area after weighting the results of intra-frame prediction and inter-frame prediction. This method can make up for their shortcomings to a certain extent by weighting the prediction results of intra-frame and inter-frame, but it is restricted by the prediction results of intra-frame and inter-frame. At the same time, there is no universal and feasible method for selecting the weighting coefficient. Therefore, it is necessary to study and design new systems and methods to improve the phenomenon of picture quality degradation caused by packet loss in real-time video communication. Summary of the invention

[0003] In view of the above problems, the present invention provides a method for repairing the degradation of picture quality caused by packet loss in real-time video communication, comprising the following steps:

[0004] Step S1, decoding the successfully received data packet, recording the decoded image as the current frame, and marking the macroblock area of ​​the current frame where data is lost as the target area;

[0005] Step S2, marking neighboring areas of the target area based on the target area;

[0006] Step S3, performing intra-frame prediction on the current frame, predicting the pixel value and pixel confidence in each target area by using the pixel value and pixel confidence of the neighboring area, and calculating the intra-frame predicted pixel value and intra-frame confidence of the target area;

[0007] Step S4, performing inter-frame prediction on the current frame, taking the previous frame image of the current frame as a reference frame, taking the region in the reference frame having the same position as the target region in the current frame as a reference target region, performing motion prediction on the reference target region to obtain a motion vector of the reference target region, predicting pixel values ​​and pixel confidence in each target region based on information of the reference frame and motion vector information of the reference target region, and calculating to obtain inter-frame predicted pixel values ​​and inter-frame confidence of the target region;

[0008] Step S5, fusion restoration, performs fusion calculation based on the intra-frame predicted pixel value, intra-frame confidence, inter-frame predicted pixel value and inter-frame confidence of the target area to obtain the restored pixels and restoration confidence of the target area, and fills each restored pixel into the target area to obtain the restored image, and uses the restored image as the final output of the restoration algorithm.

[0009] As a further explanation of the present invention, the neighbor area is a number of peripheral macroblocks adjacent to the target area.

[0010] Furthermore, the pixel confidence is the probability that the pixel value is a true value, and the pixel confidence of the pixels in the non-lost macroblock area is 1.

[0011] Furthermore, in the step S3, intra-frame prediction is performed on the target region in a descending order of the average confidence values ​​of neighboring regions.

[0012] Furthermore, if a neighbor region is a non-lost macroblock, the neighbor region confidence of the region is 1;

[0013] If a neighboring region is a lost macroblock, ie, a target region, the neighboring region confidence of the region is the average intra-frame confidence of all pixels in the region.

[0014] Furthermore, during the intra-frame prediction of the region, the prediction calculation method of any point T in the target region is:

[0015] Intra-frame prediction pixel value at point T = (L2*Nb1*C1+L1*Nb2*C2) / ((Nb1+Nb2)(L1+L2));

[0016] Intra-frame confidence at point T = (L2*Nb1+L1*Nb2) / (L1+L2);

[0017] Among them, the four neighboring areas above, below, left and right of the target area are recorded as Q1, Q2, Q3, and Q4, and their neighbor confidences are recorded as N1, N2, N3, and N4 respectively; compare the sizes of N1+N3 and N2+N4, and the neighboring areas corresponding to the larger ones are recorded as Qb1 and Qb2, and the corresponding neighbor confidences are Nb1 and Nb2; the pixels on the edges where Qb1 and Qb2 intersect with the target area are recorded as E1 and E2; the pixel values ​​of the perpendicular line from T to E1 and E2 and the intersection of E1 and E2 are recorded as C1 and C2; the distance from T to C1 is recorded as L1, and the distance from T to C2 is recorded as L2.

[0018] Furthermore, the step of obtaining the motion vector by motion prediction in step S4 includes:

[0019] 1) The area to be motion predicted is recorded as T, the area is a square, and its side length is recorded as L;

[0020] 2) The predicted motion vector is denoted as V, where V is a two-dimensional vector, with the horizontal dimension denoted as Vx and the vertical dimension denoted as Vy;

[0021] 3) Any point P in the area T has its coordinates marked as (Px, Py) and its pixel value as Sp;

[0022] 4) The motion prediction corresponding point of P is Pv in the reference frame, its coordinates are marked as (Px-Vx, Py-Vy), and its pixel value in the reference frame is recorded as Spv;

[0023] 5) The regional prediction error of T is denoted as E. For any point P in T, E = Σ(Sp–Spv)^2;

[0024] 6) For different V values, i.e. different Vx and Vy, different E can be calculated;

[0025] 7) Traverse all Vx and Vy between -L and L, and calculate the minimum E value, which is recorded as Emin;

[0026] 8) Vx and Vy when E is equal to Emin are denoted as Vbx and Vby, and the corresponding motion vector is denoted as Vb. Then Vb is the motion vector output by the motion prediction of region T.

[0027] Furthermore, the step of performing the inter-frame prediction in step S4 includes:

[0028] 1) The target area is denoted as T, and the motion vector obtained by motion prediction of the reference target area corresponding to the target area in the reference frame is denoted as Vb, and the coordinates are denoted as (Vbx, Vby);

[0029] 2) Any point P in the target area T, its coordinates are marked as (Px, Py);

[0030] 3) The point corresponding to the motion of point P in the reference frame is denoted as Pv, then the coordinates of Pv are (Px-Vbx, Py-Vby), the pixel value of Pv is denoted as Svf, and the confidence of Pv is denoted as Nvf;

[0031] If Pv is a point in the lost macroblock, the pixel value of Pv uses its restored pixel value, and the confidence uses its restored confidence

[0032] If Pv is a point in a non-lost macroblock, the pixel value of Pv uses its original pixel value and the confidence level uses 1.

[0033] 4) Use the pixel value and confidence of point Pv as the inter-frame prediction result of point P, that is, the inter-frame prediction result of point P is:

[0034] Point P inter-frame prediction pixel value = Svf

[0035] Confidence between frames at point P = Nvf

[0036] 5) Traverse all pixels in the target area and calculate their inter-frame prediction pixel values ​​and inter-frame confidence.

[0037] Furthermore, the step of performing fusion repair in step S5 includes:

[0038] 1) The target area is denoted as T, and any point P is in it;

[0039] 2) The intra-frame prediction pixel value of point P is recorded as Si, the intra-frame confidence of point P is recorded as Ni, the inter-frame prediction pixel value of point P is recorded as So, and the inter-frame confidence of point P is recorded as No;

[0040] 3) The repair result of P is:

[0041] The restored pixel value of point P = (Si*Ni+So*No) / (Ni+No);

[0042] Restoration confidence of point P = (Ni+No–Ni*No))*(Ni+No-abs(Ni-No)) / (Ni+No);

[0043] 4) Traversing all pixels in the target area, calculating their restored pixel values ​​and restoration confidence;

[0044] 5) Traverse all target areas to obtain the repaired image.

[0045] Beneficial effects of the present invention:

[0046] The present invention can effectively improve the effects of intra-frame prediction and inter-frame prediction by using known macroblock information for comprehensive prediction to repair packet loss errors, and provides a weighted coefficient calculation method that dynamically decides based on packet loss conditions to obtain better weighted effects.

[0047] 1. More effective use of intra-frame information to provide better intra-frame prediction results;

[0048] 2. Inter-frame prediction recursively uses the restored pixel information to further improve information utilization efficiency;

[0049] 3. Each calculation process retains the confidence information, so that the weight of information is more reasonably distributed in the prediction;

[0050] 4. The final restoration fuses the intra-frame results with the inter-frame results according to the confidence level, making the predicted information utilization rate higher, and performs confidence evaluation on the fused restoration results to ensure that more accurate information can be obtained in subsequent inter-frame predictions. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flow chart of a method for repairing image quality degradation caused by packet loss in real-time video communication according to the present invention;

[0052] Figure 2 The figure is a schematic diagram of the principle of performing intra-frame prediction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following is a detailed description of the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0054] In the description of the present invention, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside", "outside", "first", "second", etc., indicating directions or positions or sequential relationships, are based on the directions or positions or sequential relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as a limitation on the present invention.

[0055] A method for repairing the degradation of picture quality caused by packet loss in real-time video communication is invented, which specifically comprises the following steps:

[0056] Step S1, decoding the successfully received data packet, recording the decoded image as the current frame, and marking the macroblock area of ​​the current frame where data is lost as the target area.

[0057] Step S2, marking the neighboring area of ​​the target area based on the target area; the neighboring area here refers to the macroblock area adjacent to the target area. As shown in the accompanying drawings, in this embodiment, the four upper, lower, left and right neighboring areas of the target area are defined as Q1, Q2, Q3, and Q4 respectively.

[0058] Step S3, intra-frame prediction is performed on the current frame, and the pixel value and pixel confidence in each target area are predicted by the pixel value and pixel confidence of the neighboring area, and the intra-frame predicted pixel value and intra-frame confidence of the target area are calculated. The pixel confidence here is used to indicate the probability that the pixel value is a true value. For pixels in the non-lost macroblock area, the pixel confidence is 1. In the above intra-frame prediction process, this embodiment uses the concept of neighbor area confidence. Specifically, if a neighbor area is a non-lost macroblock, the neighbor area confidence of the area is 1; if a neighbor area is a lost macroblock, that is, the target area, the neighbor area confidence of the area is the average intra-frame confidence of all pixels in the area.

[0059] As shown in the accompanying drawings, the specific method steps for performing the above intra-frame prediction in this embodiment are as follows:

[0060] 1) The areas of the four neighboring areas above, below, left and right of the target area are recorded as Q1, Q2, Q3, and Q4 respectively, and their neighbor confidences are recorded as N1, N2, N3, and N4 respectively;

[0061] 2) performing regional intra-frame prediction on the target region in a manner of descending neighbor region confidence averages. In this embodiment, the sizes of N1+N3 and N2+N4 are specifically compared, and the neighbor regions corresponding to the larger ones are recorded as Qb1 and Qb2, and the corresponding neighbor confidences are Nb1 and Nb2;

[0062] 3) The pixels on the edges where Qb1 and Qb2 intersect the target area are recorded as E1 and E2;

[0063] 4) The pixel values ​​of the intersection of the perpendicular line from any point T to E1 and E2 and E1 and E2 are recorded as C1 and C2, the distance from point T to C1 is recorded as L1, and the distance from point T to C2 is recorded as L2;

[0064] 5) The prediction result of point T is:

[0065] Intra-frame prediction pixel value at point T = (L2*Nb1*C1+L1*Nb2*C2) / ((Nb1+Nb2)(L1+L2))

[0066] Confidence within the frame at point T = (L2*Nb1+L1*Nb2) / (L1+L2);

[0067] 6) Traverse all pixels in the target area and calculate their intra-frame predicted pixel values ​​and intra-frame confidence.

[0068] Step S4, performing inter-frame prediction on the current frame, taking the previous frame image of the current frame as a reference frame (there is no reference frame in the first frame), and recording the area in the reference frame that has the same position as the target area in the current frame as a reference target area, performing motion prediction on the reference target area to obtain a motion vector of the reference target area, predicting the pixel value and pixel confidence in each target area based on the information of the reference frame and the motion vector information of the reference target area, and calculating the inter-frame predicted pixel value and inter-frame confidence of the target area.

[0069] In this embodiment, the step of obtaining the motion vector by motion prediction is specifically as follows:

[0070] 1) The area to be motion predicted is recorded as T, the area is a square, and its side length is recorded as L;

[0071] 2) The predicted motion vector is denoted as V, where V is a two-dimensional vector, with the horizontal dimension denoted as Vx and the vertical dimension denoted as Vy;

[0072] 3) Any point P in the area T has its coordinates marked as (Px, Py) and its pixel value as Sp;

[0073] 4) The motion prediction corresponding point of P is Pv in the reference frame, its coordinates are marked as (Px-Vx, Py-Vy), and its pixel value in the reference frame is recorded as Spv;

[0074] 5) The regional prediction error of T is denoted as E. For any point P in T, E = Σ(Sp–Spv)^2;

[0075] 6) For different V values, i.e. different Vx and Vy, different E can be calculated;

[0076] 7) Traverse all Vx and Vy between -L and L, and calculate the minimum E value, which is recorded as Emin;

[0077] 8) Vx and Vy when E is equal to Emin are denoted as Vbx and Vby, and the corresponding motion vector is denoted as Vb. Then Vb is the motion vector output by the motion prediction of region T.

[0078] After the motion vector Vb of the reference target region is predicted, the next step is to perform inter-region prediction, which includes:

[0079] 1) The target area is denoted as T, and the motion vector obtained by motion prediction of the reference target area corresponding to the target area in the reference frame is denoted as Vb, and the coordinates are denoted as (Vbx, Vby);

[0080] 2) Any point P in the target area T, its coordinates are marked as (Px, Py);

[0081] 3) The point corresponding to the motion of point P in the reference frame is denoted as Pv, then the coordinates of Pv are (Px-Vbx, Py-Vby), the pixel value of Pv is denoted as Svf, and the confidence of Pv is denoted as Nvf;

[0082] If Pv is a point in the lost macroblock, the pixel value of Pv uses its restored pixel value, and the confidence uses its restored confidence

[0083] If Pv is a point in a non-lost macroblock, the pixel value of Pv uses its original pixel value and the confidence level uses 1.

[0084] 4) Use the pixel value and confidence of point Pv as the inter-frame prediction result of point P, that is, the inter-frame prediction result of point P is:

[0085] Point P inter-frame prediction pixel value = Svf

[0086] Confidence between frames at point P = Nvf

[0087] 5) Traverse all pixels in the target area and calculate their inter-frame prediction pixel values ​​and inter-frame confidence.

[0088] Step S5, fusion restoration, based on the intra-frame prediction pixel value, intra-frame confidence, inter-frame prediction pixel value and inter-frame confidence of the target area, a fusion calculation is performed to obtain the restored pixels and restoration confidence of the target area. The specific method steps of fusion restoration in this embodiment include:

[0089] 1) The target area is denoted as T, and any point P is in it;

[0090] 2) The intra-frame prediction pixel value of point P is recorded as Si, the intra-frame confidence of point P is recorded as Ni, the inter-frame prediction pixel value of point P is recorded as So, and the inter-frame confidence of point P is recorded as No;

[0091] 3) The repair result of P is:

[0092] The restored pixel value of point P = (Si*Ni+So*No) / (Ni+No);

[0093] Restoration confidence of point P = (Ni+No–Ni*No))*(Ni+No-abs(Ni-No)) / (Ni+No);

[0094] 4) Traversing all pixels in the target area, calculating their restored pixel values ​​and restoration confidence;

[0095] 5) Traverse all target areas, fill each restored pixel into the target area accordingly to obtain a restored image, and use the restored image as the final output of the restoration algorithm.

[0096] The present invention can effectively improve the effects of intra-frame prediction and inter-frame prediction by using known macroblock information for comprehensive prediction to repair packet loss errors, and provides a weighted coefficient calculation method that dynamically decides based on packet loss conditions to obtain better weighted effects.

[0097] The above only describes the preferred embodiments of the present invention, but it cannot be understood as limiting the claims. The present invention is not limited to the above embodiments, and its specific structure is allowed to be changed. In short, all changes made within the protection scope of the independent claims of the present invention are within the protection scope of the present invention.

Claims

1. A method for repairing image quality degradation caused by packet loss in real-time video communication, characterized in that: The following steps are involved: Step S1, decoding the successfully received data packet, recording the decoded image as the current frame, and marking the macroblock area of ​​the current frame where data is lost as the target area; Step S2, marking neighboring areas of the target area based on the target area; Step S3, performing intra-frame prediction on the current frame, predicting the pixel value and pixel confidence in each target area by using the pixel value and pixel confidence of the neighboring area, and calculating the intra-frame predicted pixel value and intra-frame confidence of the target area; Step S4, performing inter-frame prediction on the current frame, taking the previous frame image of the current frame as a reference frame, taking the region in the reference frame having the same position as the target region in the current frame as a reference target region, performing motion prediction on the reference target region to obtain a motion vector of the reference target region, predicting pixel values ​​and pixel confidence in each target region based on information of the reference frame and motion vector information of the reference target region, and calculating to obtain inter-frame predicted pixel values ​​and inter-frame confidence of the target region; Step S5, fusion restoration, performs fusion calculation based on the intra-frame predicted pixel value, intra-frame confidence, inter-frame predicted pixel value and inter-frame confidence of the target area to obtain the restored pixels and restoration confidence of the target area, and fills each restored pixel into the target area to obtain the restored image, and uses the restored image as the final output of the restoration algorithm.

2. The method for repairing the degradation of picture quality caused by packet loss in real-time video communication according to claim 1, characterized in that: The neighboring area is a number of peripheral macroblocks adjacent to the target area.

3. The method for repairing the degradation of picture quality caused by packet loss in real-time video communication according to claim 1, characterized in that: The pixel confidence is the probability that the pixel value is a true value, and the pixel confidence of the pixels in the non-lost macroblock area is 1.

4. The method for repairing the degradation of picture quality caused by packet loss in real-time video communication according to claim 1, characterized in that: In the step S3, intra-frame prediction is performed on the target region according to the average confidence values ​​of neighboring regions from high to low.

5. The method for repairing the degradation of picture quality caused by packet loss in real-time video communication according to claim 4, characterized in that: If a neighboring region is a non-lost macroblock, the neighboring region confidence of the region is 1; If a neighboring region is a lost macroblock, ie, a target region, the neighboring region confidence of the region is the average intra-frame confidence of all pixels in the region.

6. The method for repairing the degradation of picture quality caused by packet loss in real-time video communication according to claim 4, characterized in that: During the intra-frame prediction of the region, the prediction calculation method of any point T in the target region is: Intra-frame prediction pixel value at point T = (L2*Nb1*C1+L1*Nb2*C2) / ((Nb1+Nb2)(L1+L2)); Intra-frame confidence at point T = (L2*Nb1+L1*Nb2) / (L1+L2); Among them, the areas of the four upper, lower, left and right neighboring areas of the target area are recorded as Q1, Q2, Q3, and Q4, and their neighbor confidences are recorded as N1, N2, N3, and N4 respectively; compare the sizes of N1+N3 and N2+N4, and the neighbor areas corresponding to the larger ones are recorded as Qb1 and Qb2, and the corresponding neighbor confidences are Nb1 and Nb2; the pixels on the edges where Qb1 and Qb2 intersect with the target area are recorded as E1 and E2; the pixel values ​​of the perpendicular line from T to E1 and E2 and the intersection of E1 and E2 are recorded as C1 and C2; the distance from T to C1 is recorded as L1, and the distance from T to C2 is recorded as L2.

7. The method for repairing the degradation of picture quality caused by packet loss in real-time video communication according to claim 1, characterized in that: The step S4 of obtaining the motion vector by motion prediction comprises: 1) The area to be motion predicted is recorded as T, the area is a square, and its side length is recorded as L; 2) The predicted motion vector is denoted as V, where V is a two-dimensional vector, with the horizontal dimension denoted as Vx and the vertical dimension denoted as Vy; 3) Any point P in the area T has its coordinates marked as (Px, Py) and its pixel value as Sp; 4) The motion prediction corresponding point of P is Pv in the reference frame, its coordinates are marked as (Px-Vx, Py-Vy), and its pixel value in the reference frame is recorded as Spv; 5) The regional prediction error of T is denoted as E. For any point P in T, E = Σ(Sp–Spv)^2; 6) For different V values, i.e. different Vx and Vy, different E can be calculated; 7) Traverse all Vx and Vy between -L and L, and calculate the minimum E value, which is recorded as Emin; 8) Vx and Vy when E is equal to Emin are denoted as Vbx and Vby, and the corresponding motion vector is denoted as Vb. Then Vb is the motion vector output by the motion prediction of region T.

8. The method for repairing the degradation of picture quality caused by packet loss in real-time video communication according to claim 1, characterized in that: The step S4 of performing the inter-frame prediction comprises: 1) The target area is denoted as T, and the motion vector obtained by motion prediction of the reference target area corresponding to the target area in the reference frame is denoted as Vb, and the coordinates are denoted as (Vbx, Vby); 2) Any point P in the target area T, its coordinates are marked as (Px, Py); 3) The point corresponding to the motion of point P in the reference frame is denoted as Pv, then the coordinates of Pv are (Px-Vbx, Py-Vby), the pixel value of Pv is denoted as Svf, and the confidence of Pv is denoted as Nvf; If Pv is a point in the lost macroblock, the pixel value of Pv uses its restored pixel value, and the confidence uses its restored confidence If Pv is a point in a non-lost macroblock, the pixel value of Pv uses its original pixel value and the confidence level uses 1. 4) Use the pixel value and confidence of point Pv as the inter-frame prediction result of point P, that is, the inter-frame prediction result of point P is: Point P inter-frame prediction pixel value = Svf Confidence between frames at point P = Nvf 5) Traverse all pixels in the target area and calculate their inter-frame prediction pixel values ​​and inter-frame confidence.

9. The method for repairing the degradation of picture quality caused by packet loss in real-time video communication according to claim 1, characterized in that: The step S5 of performing fusion repair comprises: 1) The target area is denoted as T, and any point P is in it; 2) The intra-frame prediction pixel value of point P is recorded as Si, the intra-frame confidence of point P is recorded as Ni, the inter-frame prediction pixel value of point P is recorded as So, and the inter-frame confidence of point P is recorded as No; 3) The repair result of P is: The restored pixel value of point P = (Si*Ni+So*No) / (Ni+No); Restoration confidence of point P = (Ni+No–Ni*No))*(Ni+No-abs(Ni-No)) / (Ni+No); 4) Traversing all pixels in the target area, calculating their restored pixel values ​​and restoration confidence; 5) Traverse all target areas to obtain the repaired image.

Citation Information

Patent Citations

  • Inter prediction apparatus and method for video coding

    CN111758258A

  • Image decoding device, image decoding method, and image decoding program

    US20210211648A1