Video frame prediction method, video bitstream processing method, and related apparatus

Abstract

The present application provides a video frame prediction method, a video bitstream processing method, and a related apparatus. The method comprises: determining a plurality of first angular modes on the basis of angular mode statistical information in a spatial domain (S100); determining, on the basis of indexes of the first angular modes, prediction results corresponding to the first angular modes used by a current CU (S200); and determining a fused prediction result of the current CU on the basis of the prediction results corresponding to the first angular modes and a first weight coefficient (S300).

Description

Video frame prediction method, video bitstream processing method and related device

[0001] Cross-reference to related applications

[0002] This application is based on the Chinese patent application No. 202410890999.5, filed on July 3, 2024, and claims priority to the Chinese patent application No. 202410890999.5, and the entire contents of the Chinese patent application No. 202410890999.5 are hereby incorporated by reference into this application. TECHNICAL FIELD

[0003] Embodiments of the present application relate to the technical field of video coding, and in particular to a video frame prediction method, a video bitstream processing method and related devices. BACKGROUND

[0004] Video frame prediction includes intra prediction and inter prediction. Video frame prediction mainly uses the pixels of the reconstructed region as reference pixels to predict the pixels of the current block, with the purpose of removing the spatial redundancy of the video signal. If the video frame has a video block with complex texture structure, the prediction difficulty will undoubtedly be increased. How to accurately predict the video block with complex texture structure is a technical problem to be solved at present. SUMMARY

[0005] Embodiments of the present application provide a video frame prediction method, a video bitstream processing method and related devices.

[0006] In a first aspect, embodiments of the present application provide a video frame prediction method, which comprises: determining a plurality of first angle modes according to angle mode statistical information of a spatial domain; determining a prediction result corresponding to the first angle mode used by a current CU according to an index of the first angle mode; and determining a fusion prediction result of the current CU according to the prediction result corresponding to each first angle mode and a first weight coefficient.

[0007] In a second aspect, embodiments of the present application provide a video bitstream processing method, which comprises: setting a preset switch flag in encoding parameter information to an opening flag value, the preset switch flag being used to indicate whether to perform fusion prediction, wherein the fusion prediction is realized based on the video frame prediction method as described in the first aspect; and writing the encoding parameter information into a video bitstream.

[0008] In a third aspect, embodiments of the present application provide a video bitstream processing method, which comprises: receiving a video bitstream and obtaining encoding parameter information from the video bitstream; and performing fusion prediction according to a preset switch flag in the encoding parameter information, wherein the fusion prediction is realized based on the video frame prediction method as described in the first aspect.

[0009] In a fourth aspect, an electronic device is provided, which includes at least one processor, at least one memory configured to store at least one program, and the at least one program is configured to, when executed by the at least one processor, implement the video frame prediction method according to the first aspect, or the video bitstream processing method according to the second aspect, or the video bitstream processing method according to the third aspect.

[0010] In a fifth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is configured to, when executed by a processor, implement the video frame prediction method according to the first aspect, or the video bitstream processing method according to the second aspect, or the video bitstream processing method according to the third aspect.

[0011] In a sixth aspect, a program product is provided, which includes a computer program, and the computer program is configured to, when executed by a processor, implement the video frame prediction method according to the first aspect, or the video bitstream processing method according to the second aspect, or the video bitstream processing method according to the third aspect. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the present application and are used to explain the technical solutions of the present application, but do not limit the present application.

[0013] FIG. 1 is a general flowchart of the video frame prediction method according to an embodiment of the present application;

[0014] FIG. 2 is a flowchart of determining the plurality of first angle modes in step S100 in FIG. 1;

[0015] FIG. 3 is a schematic diagram of a plurality of spatial sampling points according to an embodiment of the present application;

[0016] FIG. 4 is a schematic diagram of a frequency histogram according to an embodiment of the present application;

[0017] FIG. 5 is a flowchart of determining the frequency histogram in step S120 in FIG. 2;

[0018] FIG. 6 is a flowchart of determining the first angle mode in step S130 in FIG. 2;

[0019] FIG. 7 is a flowchart of determining the prediction result in the case that the index of the first angle mode belongs to the first index in step S200 in FIG. 1;

[0020] FIG. 8 is a flowchart of determining the prediction result in step S210 in FIG. 7;

[0021] FIG. 9 is a schematic diagram of a reference pixel according to an embodiment of the present application;

[0022] FIG. 10 is a flowchart of determining a prediction result in a case where the index of the first angle mode does not belong to the first index in step S200 of FIG. 1;

[0023] FIG. 11 is a flowchart of obtaining a first weight coefficient corresponding to the first angle mode according to an embodiment of the present application;

[0024] FIG. 12 is a flowchart of obtaining a prediction loss value corresponding to the first angle mode according to an embodiment of the present application;

[0025] FIG. 13 is a schematic diagram of an L-shaped template region according to an embodiment of the present application;

[0026] FIG. 14 is a schematic diagram of an upper template region according to an embodiment of the present application;

[0027] FIG. 15 is a schematic diagram of a left template region according to an embodiment of the present application;

[0028] FIG. 16 is a schematic diagram of a template region and a current CU according to an embodiment of the present application;

[0029] FIG. 17 is a flowchart of determining a prediction pixel value of a template region in step S411 of FIG. 12;

[0030] FIG. 18 is a flowchart of obtaining a first weight coefficient corresponding to the first angle mode according to an embodiment of the present application;

[0031] FIG. 19 is a flowchart of a video stream processing method according to an embodiment of the present application;

[0032] FIG. 20 is a schematic diagram of a framework of video stream processing in an encoding process according to an embodiment of the present application;

[0033] FIG. 21 is a flowchart of setting a preset switch flag in encoding parameter information to an open flag value in step S610 of FIG. 19;

[0034] FIG. 22 is another flowchart of a video stream processing method according to an embodiment of the present application;

[0035] FIG. 23 is a schematic diagram of a framework of video stream processing in a decoding process according to an embodiment of the present application;

[0036] FIG. 24 is a flowchart of performing fusion prediction according to a preset switch flag in encoding parameter information in step S720 of FIG. 22;

[0037] FIG. 25 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0039] It should be understood that, in the description of the embodiments of the present application, if "first", "second" and the like are described, it is only for the purpose of distinguishing technical features, and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or the sequence of the indicated technical features. "At least one" means one or more, and "multiple" means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. Wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" and the like means any group of these items, including any group of single items or multiple items. For example, at least one of a, b and c can mean a, b, c, a and b, a and c, b and c, or a and b and c, wherein a, b and c can be single or multiple.

[0040] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as there is no conflict.

[0041] Video frame prediction uses the pixels of the reconstructed region as reference pixels to predict the pixels of the current block, aiming to remove the spatial redundancy of the video signal and improve the efficiency of video compression. If the video frame has a video block with complex texture structure, the prediction difficulty will undoubtedly be increased. How to accurately predict the video block with complex texture structure is a technical problem to be solved at present.

[0042] Based on this, the embodiments of the present application provide a video frame prediction method, an electronic device, a computer readable storage medium and a computer program product. The video frame prediction method provided by the embodiments of the present application can combine the advantages of multiple first angle mode predictions to accurately predict the video block with complex texture structure, thereby improving the accuracy of frame prediction. The first weight coefficient corresponds to the first angle mode, which can better allocate each prediction result, so that the obtained fusion prediction result is more accurate, and the prediction accuracy of the video block with complex texture structure is improved.

[0043] The present application will be further described below with reference to the accompanying drawings.

[0044] Referring to FIG. 1, FIG. 1 is a general flowchart of a video frame prediction method provided by an embodiment of the present application. The video frame prediction method provided by the embodiments of the present application includes but is not limited to the following steps:

[0045] In step S100, a plurality of first angle modes are determined according to angle mode statistical information of a spatial domain.

[0046] In step S200, a prediction result corresponding to the first angle mode is determined for the current CU according to an index of the first angle mode.

[0047] In step S300, a fusion prediction result of the current CU is determined according to the prediction result corresponding to each first angle mode and a first weight coefficient.

[0048] It should be noted that the spatial domain is also referred to as a pixel domain, which is a space where a pixel position is located. For a video signal, the spatial domain refers to a representation form of the signal in a two-dimensional space.

[0049] It should be noted that the video frame prediction of the embodiments of the present application includes intra prediction and inter prediction. For intra prediction, the spatial domain refers to a spatial domain of a current frame; for inter prediction, the spatial domain refers to a spatial domain of a reference frame.

[0050] It should be noted that the angle mode statistical information refers to statistical information corresponding to angle prediction modes used by already reconstructed coding units in each spatial domain.

[0051] The angle mode, i.e., the angle prediction mode, is used to predict a current pixel by using already coded pixels. In High Efficiency Video Coding (HEVC), i.e., H.265, the number of angle prediction modes is 33. In Versatile Video Coding (VCC), i.e., H.266, the number of angle prediction modes is expanded from 33 to 65.

[0052] It should be noted that the plurality of first angle modes are determined based on the angle mode statistical information of the spatial domain, and the first angle mode is an angle prediction mode used by a plurality of coding units in the spatial domain corresponding to the current CU.

[0053] It should be noted that the index corresponds to the first angle mode, and the index is used to indicate a manner of determining the prediction result corresponding to the first angle mode for the current CU. According to the index of the first angle mode, the prediction result corresponding to the first angle mode for the current CU is determined by a manner corresponding to the index.

[0054] It should be noted that, for each first angle mode, it corresponds to a prediction result and a first weight coefficient. The prediction result refers to the prediction result corresponding to the first angle mode used by the current CU, and the first weight coefficient is used to represent the importance of the prediction result corresponding thereto in the fusion prediction result of the current CU. Based on the prediction result and the first weight coefficient corresponding to each first angle mode, the fusion prediction of the current CU is performed, thereby determining the prediction result of the current CU.

[0055] It should be noted that, the embodiments of the above steps S100 to S300 first determine the angle mode statistical information based on the spatial domain, and determine a plurality of first angle modes. Then, according to the index of the first angle mode, the prediction result corresponding to the first angle mode used by the current CU (Coding Unit, CU) is determined, and then according to the prediction result and the first weight coefficient corresponding to each first angle mode, the fusion prediction result of the current CU is determined. In the embodiments of the present application, the fusion prediction result of the current CU is determined based on the prediction results corresponding to the plurality of first angle modes, so that the video frame prediction method can combine the advantages of the plurality of first angle mode predictions to accurately predict the video block with complex texture structure, thereby improving the accuracy of the intra prediction. In addition, the first weight coefficient corresponds to the first angle mode, which can better allocate each prediction result, so that the obtained fusion prediction result is more accurate, and the prediction accuracy of the video block with complex texture structure is improved.

[0056] It can be understood that, referring to FIG. 2, step S100 includes:

[0057] Step S110, determining a plurality of spatial domain sampling points.

[0058] Step S120, constructing a frequency histogram according to the angle mode information of the CU in which each spatial domain sampling point is located.

[0059] Step S130, determining a plurality of first angle modes according to the frequency histogram.

[0060] It should be noted that, the spatial domain sampling points are determined in the spatial domain corresponding to the video signal, thereby determining a plurality of spatial domain sampling points. In order to improve the efficiency of the intra prediction, more spatial domain sampling points need to be determined in the embodiments of the present application to enhance the search and utilization of the spatial domain information. Referring to FIG. 3, the spatial domain sampling points determined in the spatial domain in the embodiments of the present application are as many as 92.

[0061] It should be noted that, the spatial domain sampling points are located in the coding unit (CU), and the angle mode information of the CU in which each spatial domain sampling point is located is determined to construct a frequency histogram. Referring to FIG. 4, the horizontal axis of the frequency histogram is the angle mode information, and the vertical axis is the amplitude, which can be used to represent the number of CUs using each angle mode information.

[0062] It should be noted that after the frequency histogram is determined, the first angle mode can be determined according to the frequency of appearance of each angle mode information shown in the frequency histogram.

[0063] It should be noted that the embodiment of the above steps S110 to S130 constructs a frequency histogram according to the angle mode information of the CUs where the plurality of spatial domain sampling points are located, and further determines a plurality of first angle modes, the first angle modes being determined based on the frequency of appearance of each angle mode information in the plurality of CUs in the spatial domain, the selection of the first angle modes being more accurate, and thus the accuracy of the intra prediction is improved.

[0064] It can be understood that the spatial domain sampling points include at least one of the neighboring sampling points and the non-neighboring sampling points.

[0065] It should be noted that the CU where the neighboring sampling points are located is adjacent to the CU that needs to be currently predicted. For example, the five neighboring sampling points in FIG. 3 are located in the CUs adjacent to the current CU. The CU where the non-neighboring sampling points are located is not adjacent to the current CU. For example, the spatial domain sampling points in FIG. 3 include 87 non-neighboring sampling points, and the CUs where the 87 non-neighboring sampling points are located are not adjacent to the current CU.

[0066] It should be noted that the spatial domain sampling points include at least one of the neighboring sampling points and the non-neighboring sampling points. The neighboring sampling points and the non-neighboring sampling points can be selected as needed. When the spatial domain sampling points are only the neighboring sampling points or the non-neighboring sampling points, the first angle mode is determined according to the plurality of spatial domain sampling points, and the amount of data of the spatial domain sampling points to be processed is significantly reduced, thereby reducing the time required to determine the first angle mode. When the spatial domain sampling points include the neighboring sampling points and the non-neighboring sampling points, the spatial domain information contained in the spatial domain sampling points is more diverse, thereby improving the matching degree of the first angle mode and the current CU and improving the accuracy of the intra prediction.

[0067] It can be understood that the neighboring sampling points include adjacent pixels located above, above-left, left, below-left, and above-right of the current CU.

[0068] It should be noted that the neighboring sampling points corresponding to the current CU in FIG. 3 include five, and the CUs corresponding to the five neighboring sampling points are located above, above-left, left, below-left, and above-right of the current CU. Therefore, the five neighboring sampling points are adjacent pixels located above, above-left, left, below-left, and above-right of the current CU.

[0069] It can be understood that the non-neighboring sampling points include non-adjacent pixels located above, above-left, left, below-left, and above-right of the current CU.

[0070] It should be noted that the 87 non-adjacent sampling points are distributed in the CUs above, above-left, left, below-left and above-right of the current CU, and the non-adjacent sampling points are non-adjacent pixels located above, above-left, left, below-left and above-right of the current CU.

[0071] The adjacent sampling points and the non-adjacent sampling points are located above, above-left, left, below-left and above-right of the current CU, and correspond to the pixels required for prediction by the angular prediction mode, so as to facilitate determination of the first angular mode to meet the prediction requirement of the current CU.

[0072] It can be understood that, referring to FIG. 5, step S120 comprises:

[0073] Step S121, traversing each CU where the spatial sampling point is located, and determining the second angular mode used by the current traversed CU.

[0074] Step S122, adding the area of the current traversed CU to the second angular mode of the frequency histogram.

[0075] It should be noted that the second angular mode is the second angular mode used by the CU where the spatial sampling point is located. Therefore, for the frequency histogram in FIG. 4, the angular mode information on the horizontal axis can be regarded as the second angular mode. In addition, the frequency histogram can be constructed in the following manner: HoC[idx] = HoC[idx] + CU area,

[0076] wherein HoC is a frequency histogram (Histogram of Occurrence), idx is angular mode information (such as an angular mode index), in VCC, the value range of idx is 1-66, wherein idx of 1 represents a direct current (DC) mode, and idx of 2-66 represents 65 traditional angular prediction modes. After determining the angular mode information of the CU where the spatial sampling point is located, the area of the CU is obtained, and is added to the corresponding second angular mode of the frequency histogram. After the traversal of the multiple spatial sampling points is completed based on the above formula, a complete frequency histogram can be obtained.

[0077] It should be noted that idx of 0 represents a planar (Planar) mode.

[0078] It should be noted that the embodiments of steps S121 and S122 add the area of the current traversed CU to the second angular mode corresponding to the current traversed CU in the frequency histogram by traversing each CU where the spatial sampling point is located, so that the applicable frequency of each second angular mode to the spatial sampling point in the current CU can be accurately determined based on the frequency histogram, and the accuracy of the first angular mode is improved.

[0079] It can be understood that, referring to FIG. 6, step S130 includes:

[0080] Step S131, determining n second angle modes with the highest amplitudes according to the frequency histogram, where n is greater than or equal to 1 and less than or equal to 5.

[0081] Step S132, determining the first angle mode according to the n second angle modes, where the first angle mode at least includes a planar mode.

[0082] It should be noted that n is the number of the second angle modes with the highest amplitudes selected in the frequency histogram. Embodiments of the present application improve the accuracy of the first angle mode, and in order to reduce the amount of calculation, the maximum value of n is set to 5, that is, n is an integer in the interval [1, 5].

[0083] It should be noted that after the frequency histogram is determined, the multiple second angle modes are sorted in descending order of amplitude, and the top 5, that is, the 5 second angle modes with the highest amplitudes are obtained. If there are less than 5 second angle modes with amplitudes not equal to 0 in the frequency histogram, multiple second angle modes with amplitudes not equal to 0 are obtained.

[0084] It should be noted that the maximum value of n can also be set to other values as needed, which can be set according to the intra mode. For example, when the intra mode is occurrence-based intra coding (OBIC), the maximum value of n can be set to 5. When the intra mode is decoder-side intra mode derivation (DIMD), the maximum value of n is set to 5. When the intra mode is template-based intra mode derivation (TIMD), the maximum value of n is 3. The maximum value of n corresponding to the spatial geometric partitioning mode (SGPM) is 2.

[0085] It should be noted that after the n second angle modes are determined, the n second angle modes and the planar mode are determined as the first angle mode. Then the number of the first angle mode is greater than or equal to 2 and less than or equal to 6.

[0086] It should be noted that the embodiments of steps S131 and S132 take the n second angle modes with the highest amplitudes in the frequency histogram and the planar mode as the first angle mode, so that the first angle mode can correspond to the current CU, and the accuracy of intra prediction is improved.

[0087] It can be understood that, referring to FIG. 7, the step S200 comprises:

[0088] The step S210 comprises: determining the prediction result corresponding to the first angle mode by using a matrix-based intra prediction mode in a case where the index of the first angle mode belongs to a first index.

[0089] For example, the first index comprises 1 and (2+2*k), k∈[0,32].

[0090] It should be noted that the first index comprises 1 and (2+2*k), where k is a positive integer in the interval [0,32]. If the first index is 1, the first angle mode is a direct current (DC) mode. If the first index is (2+2*k), the first angle mode is a traditional angle prediction mode.

[0091] It should be noted that, in a case where the index of the first angle mode belongs to a first index, the prediction result corresponding to the first angle mode is determined by using a matrix-based intra prediction mode. The matrix-based intra prediction mode is a non-linear intra prediction mode, and the prediction accuracy of the non-linear intra prediction mode is higher than that of the linear prediction in the related art.

[0092] It should be noted that, in a case where the index of the first angle mode belongs to a first index, the prediction result corresponding to the first angle mode is determined by using a matrix-based intra prediction mode in the step S210, so that the prediction is performed by using the non-linear intra prediction mode, and the accuracy of the prediction result corresponding to the first index is higher.

[0093] It can be understood that, referring to FIG. 8, the step S210 comprises:

[0094] The step S211 comprises: obtaining a second weight coefficient corresponding to a target position in the current CU and the first angle mode from a weight matrix.

[0095] The step S212 comprises: obtaining a reference pixel corresponding to the target position according to the first angle mode.

[0096] The step S213 comprises: obtaining a prediction pixel of the target position according to the second weight coefficient and the reference pixel.

[0097] The step S214 comprises: obtaining the prediction result corresponding to the first angle mode according to the prediction pixels of the target positions in the current CU.

[0098] It should be noted that the weight matrix corresponds to each position in the current CU. The weight matrix is provided with a plurality of weight coefficients corresponding to the first index. Since the first index includes 1 and (2+2*k), k∈[0,32], it is determined that the first index includes 34 values, and 34 weight coefficients are provided in each weight matrix. For a target position in the current CU, the first index is determined based on the first angle mode used by the current CU to obtain the second weight coefficient corresponding to the first index in the weight matrix corresponding to the target position.

[0099] It should be noted that the reference pixel is a neighboring spatial pixel of the current CU. Referring to FIG. 9, P is the current CU, and the reference pixel region of the current CU is the L-shaped shaded part in the figure. The L-shaped reference pixel region is composed of two parts. If the length of the current CU is W and the width is H, then the length of one part of the reference pixel region is 2*W and the width is T1, and the length of the other part of the reference pixel region is T2 and the width is 2*H. In addition, the size and shape of the current CU are not fixed, and it can be a square or a rectangle. To ensure that the reference pixel can correspond to each position in the current CU, the present application embodiment sets corresponding T1 and T2 for different sizes of the current CU.

[0100] Table 1

[0101] Referring to Table 1, Table 1 is a table of the correspondence between the shape of the current CU and the values of T1 and T2. When the size of the current CU is 32x16, 26x32 or 32x32, the values of T1 and T2 are both 1, and in other cases, the values of T1 and T2 are 2.

[0102] It should be noted that the determination of the reference pixel is different for different first angle modes. For example, the reference pixel corresponding to the direct current mode is the pixel above the target position and the pixel on the left side. The reference pixel corresponding to the traditional angle mode prediction needs to be determined according to the angle corresponding to the angle mode prediction. Therefore, the reference pixel corresponding to the target position needs to be obtained according to the first angle mode.

[0103] It should be noted that the prediction pixel of the target position is determined according to the second weight coefficient and the reference pixel. In some embodiments, the prediction pixel is the product of the second weight coefficient and the reference pixel. After determining the prediction pixel of each target position in the current CU, the prediction result corresponding to the first angle mode can be obtained based on each prediction pixel.

[0104] It should be noted that the prediction result corresponding to the first angle mode used by the current CU can be represented as: P(x,y)=∑ k F(x,y,l)*r(l)

[0105] wherein (x, y) is a target position in the current CU, P(x, y) represents a prediction result corresponding to the first angle mode used by the current CU, l is a first index to which the first angle mode belongs, the value of l is 1 and (2+2*k), k∈[0, 32], F(x, y) is a weight matrix of the target position (x, y), F(x, y, l) represents a second weight coefficient corresponding to the target position (x, y) and the first angle mode in the current CU, and r(l) is a reference pixel corresponding to the target position. In addition, the embodiment of the present application assigns an index l to each reference pixel and second weight coefficient, and the setting of the index l represents a calculation order, which facilitates the calculation of the prediction result.

[0106] It should be noted that the embodiment of the above steps S211 to S214 obtains the second weight coefficient corresponding to the target position in the current CU and the first angle mode from the weight matrix in the case where the index of the first angle mode belongs to the first index, and obtains the reference pixel corresponding to the target position according to the first angle mode, so as to obtain the prediction pixel of the target position according to the second weight coefficient and the reference pixel. After the prediction pixels of each target position in the current CU are determined, the prediction result corresponding to the first angle mode is determined. The determination manner of the prediction result gives the detailed steps of the non-linear intra prediction mode, so that the obtained prediction result is more accurate.

[0107] It can be understood that, referring to FIG. 10, the step S200 further includes:

[0108] In the case where the index of the first angle mode does not belong to the first index, the step S220 determines the prediction result corresponding to the first angle mode by using the angle prediction mode.

[0109] It should be noted that the index of the first angle mode does not belong to the first index, that is, the index in this case includes 0 and (1+2*k), wherein k is a positive integer in the interval [0, 32]. If the value corresponding to the index is 0, the first angle mode is the planar mode. If the value corresponding to the index is (1+2*k), the first angle mode is the traditional angle prediction mode.

[0110] It should be noted that in the case where the index of the first angle mode does not belong to the first index, the prediction result corresponding to the first angle mode is determined by using the angle prediction mode corresponding to the first angle mode. For example, in the case where the first angle mode is the planar mode, the prediction result corresponding to the first angle mode used by the current CU is determined by using the planar mode, and the planar mode is a special angle mode. In the case where the index corresponding to the first angle mode is 5, the prediction result corresponding to the first angle mode used by the current CU is determined by using the angle prediction mode.

[0111] It should be noted that the embodiment of the step S220 uses the angle prediction mode to determine the prediction result corresponding to the first angle mode in the case that the index of the first angle mode does not belong to the first index. The angle prediction mode is the linear intra prediction mode. The finally determined fusion prediction result of the current CU contains the characteristics of the linear intra prediction mode and the advantages of the linear intra prediction.

[0112] In addition, the embodiment of the present application uses the matrix-based intra prediction mode, i.e., the non-linear intra prediction mode, to determine the prediction result corresponding to the first angle mode of the current CU in the case that the index of the first angle mode belongs to the first index, and uses the angle prediction mode, i.e., the linear intra prediction mode, to determine the prediction result corresponding to the first angle mode in the case that the index of the first angle mode does not belong to the first index. Since the fusion prediction result of the current CU is determined based on the prediction result corresponding to each first angle mode and the first weight coefficient, the video frame prediction method of the embodiment of the present application is a linear and non-linear intra joint prediction method. The fusion prediction result fuses multiple linear and non-linear prediction results, combines the advantages of the linear intra prediction and the non-linear intra prediction, improves the prediction accuracy of the video block with complex texture, and improves the accuracy of the intra prediction.

[0113] It can be understood that, with reference to FIG. 11, the first weight coefficient corresponding to the first angle mode is obtained by the following steps:

[0114] In step S410, the prediction loss values corresponding to all the first angle modes are obtained.

[0115] In step S420, the total prediction loss value is obtained according to the sum of the prediction loss values corresponding to all the first angle modes.

[0116] In step S430, for each first angle mode, the first weight coefficient corresponding to the first angle mode is obtained according to the prediction loss value corresponding to the first angle mode and the total prediction loss value.

[0117] It should be noted that the intra prediction has the raster scan prediction property. Therefore, for the CU that needs to be intra predicted, the pixels in the upper and left spatial domains of the CU have been reconstructed. Therefore, the prediction loss value corresponding to the first angle mode can be determined according to the prediction result corresponding to the first angle mode and the reconstruction result in the part of the spatial domain.

[0118] It should be noted that after the prediction loss values corresponding to all the first angle modes are determined, the sum of the prediction loss values corresponding to all the first angle modes is obtained to obtain the total prediction loss value.

[0119] It should be noted that the first weight coefficient corresponding to the first angle mode is inversely proportional to the prediction loss value corresponding thereto, that is, the greater the prediction loss value, the smaller the first weight coefficient. Therefore, in the process of calculating the first weight coefficient, firstly, the difference between the total prediction loss value and the prediction loss value corresponding to the first angle mode needs to be calculated, and the first weight coefficient is determined according to the proportion of the difference to the total prediction loss value.

[0120] For example, two available angle mode indexes (i.e. indexes of the first angle mode) are obtained through the frequency histogram, and the first weight coefficient of mode 0 is the first weight coefficient of mode 1 is wherein, cost0 is the prediction loss value of mode 0, cost1 is the prediction loss value of mode 1, and cost0+cost1 is the total prediction loss value. It should be noted that mode 0 is not a planar mode, and mode 1 is not a traditional angle prediction mode, but mode 0 and mode 1 are only the numbering results of the plurality of first angle modes. Since the maximum value of the number of the second angle modes determined through the frequency histogram is 5, the prediction loss value can be expressed as cost x , x∈[0,4], x is the result of renumbering the second angle mode determined through the frequency histogram.

[0121] It should be noted that the weight of the planar mode is not determined by the prediction loss value of other first angle modes. For example, when the first angle mode determined through the frequency histogram is 1, the first weight coefficient of the first angle mode is 43, and the first weight coefficient of the planar mode is 21. When the first angle mode determined through the frequency histogram is 2 to 5, the first weight coefficients of other first angle modes are determined by the proportion of the prediction loss value corresponding to the first angle mode to the total prediction loss value, and the first weight coefficients of the planar mode are all 16.

[0122] The following exemplary describes the corresponding first weight coefficient calculation mode for different available angle mode index numbers.

[0123] Example 1: One available angle mode index is obtained according to the histogram

[0124] The weight of mode 0 is: 43;

[0125] The weight of the planar mode is: 21.

[0126] Example 2: Two available angle mode indexes are obtained according to the histogram

[0127] The weight of mode 0 is:

[0128] The weight of mode 1 is:

[0129] Planar mode weight is: 16.

[0130] Example 3: 3 available angular mode indices are obtained from the histogram

[0131] Mode 0 weight is:

[0132] Mode 1 weight is:

[0133] Mode 2 weight is:

[0134] Planar mode weight is: 16.

[0135] Example 4: 4 available angular mode indices are obtained from the histogram

[0136] Mode 0 weight is:

[0137] Mode 1 weight is:

[0138] Mode 2 weight is:

[0139] Mode 3 weight is:

[0140] Planar mode weight is: 16.

[0141] Example 5: 5 available angular mode indices are obtained from the histogram

[0142] Mode 0 weight is:

[0143] Mode 1 weight is:

[0144] Mode 2 weight is:

[0145] Mode 3 weight is:

[0146] Mode 4 weight is:

[0147] Planar mode weight is: 16.

[0148] It should be noted that the embodiment of the above steps S410 to S430 determines the first weight coefficient by the prediction loss value corresponding to each first angle mode and the total prediction loss value, the greater the prediction loss value, the smaller the first weight coefficient, that is, the smaller the weight of the prediction result corresponding to the first angle mode in the fusion prediction result, thereby reducing the loss value of the fusion prediction result corresponding to the current CU and improving the accuracy of the intra prediction.

[0149] It can be understood that, with reference to FIG. 12, the prediction loss value corresponding to the first angle mode in step S410 is obtained by the following steps:

[0150] Step S411, determining the prediction pixel value of the template region based on the first angle mode.

[0151] Step S412, obtaining the reconstructed pixel value of the template region.

[0152] Step S413, determining the prediction loss value corresponding to the first angle mode according to the prediction pixel value and the reconstructed pixel value of the template region.

[0153] It should be noted that the template region is adjacent to the current CU. No matter which first angle mode the current CU corresponds to, the same template region can be used. The shape of the template region can be set as needed, such as the L-shaped template region shown in FIG. 13, the upper template region shown in FIG. 14, and the left template region shown in FIG. 15. Among them, the L-shaped template region includes two parts of the left template region and the upper template region. In addition, the template region can also be other shapes.

[0154] It should be noted that different template regions can also be set for different first angle modes. For example, in the case where the index of the first angle mode does not belong to the first index, the L-shaped template region in FIG. 13 can be set, and in the case where the index of the first angle mode belongs to the first index, the upper template region shown in FIG. 14 or the left template region shown in FIG. 15 can be set.

[0155] It should be noted that the prediction pixel value of the template region is the prediction result obtained by using the first angle mode for the template region. With reference to FIGS. 13 to 15, the template region is based on the reference pixel and uses the corresponding first angle mode for prediction, thereby obtaining the prediction pixel value.

[0156] It should be noted that the intra prediction has the property of raster scan prediction, so for the CU that needs to be intra predicted, the pixels in the upper and left spatial domain of the CU, that is, the pixels in the template region, have been reconstructed, and the reconstruction result is the reconstructed pixel value. With reference to FIG. 16, the template region is located in the reconstructed region, and the pixels in the template region have been reconstructed. The current CU is located in the unreconstructed region, and therefore, the current CU needs to be intra predicted to obtain the fusion prediction result.

[0157] It should be noted that after the prediction pixel value and the reconstructed pixel value of the template region are determined, the prediction loss value corresponding to the first angle mode can be determined based on the prediction pixel value and the reconstructed pixel value of the template region using the first angle mode. The prediction loss value corresponding to the first angle mode can be expressed as:

[0158] Wherein, the Sum of Absolute Difference is the prediction loss value, xy is each pixel position in the template region, pReco xy is the reconstructed pixel value, and pRred xy is the prediction pixel value. Then, the difference between each reconstructed pixel value and the prediction loss value of the template region corresponding to the first angle mode needs to be determined first, and the sum of the obtained multiple difference values is the prediction loss value.

[0159] It should be noted that the embodiments of the above steps S411 to S413 determine the prediction loss value corresponding to the first angle mode by the reconstructed loss value and the prediction loss value of the template region, and the accuracy of the obtained prediction loss value is higher. Then, the adaptability of the first weight coefficient determined based on the prediction loss value to the video block with different textures is stronger, and the weight distribution can be more accurately performed, thereby improving the prediction accuracy of the video block with complex textures.

[0160] It can be understood that referring to FIG. 17, the step S411 comprises:

[0161] Step S414, in the case that the index of the first angle mode belongs to the first index, the prediction pixel value of the template region is determined using the matrix-based intra prediction mode; for example, the first index includes 1 and (2+2*k), k∈[0,32].

[0162] Or,

[0163] Step S415, in the case that the index of the first angle mode does not belong to the first index, the prediction pixel value of the template region is determined using the angle prediction mode.

[0164] It should be noted that the first index includes 1 and (2+2*k), wherein k is a positive integer in the interval [0,32]. If the first index is 1, the first angle mode is the direct current (DC) mode. If the first index is (2+2*k), the first angle mode is the traditional angle prediction mode.

[0165] It should be noted that in the case that the index of the first angle mode belongs to the first index, the prediction result corresponding to the first angle mode is determined using a matrix-based intra prediction mode. The matrix-based intra prediction mode is a non-linear intra prediction mode, and the prediction accuracy of the non-linear intra prediction mode is higher than that of the linear prediction in the related art.

[0166] It should be noted that the index of the first angle mode does not belong to the first index, that is, the index includes 0 and (1+2*k), where k is a positive integer in the interval [0, 32]. If the value corresponding to the index is 0, the first angle mode is a planar mode. If the value corresponding to the index is (1+2*k), the first angle mode is a traditional angle prediction mode.

[0167] It should be noted that in the case that the index of the first angle mode does not belong to the first index, the prediction result corresponding to the first angle mode is determined using an angle prediction mode corresponding to the first angle mode. For example, in the case that the first angle mode is a planar mode, the prediction result corresponding to the first angle mode is determined using the planar mode, and the planar mode is a special angle mode. In the case that the index corresponding to the first angle mode is 5, the prediction result corresponding to the first angle mode is determined using an angle prediction mode.

[0168] It should be noted that in the case that the index of the first angle mode does not belong to the first index, the prediction result corresponding to the first angle mode is determined using an angle prediction mode corresponding to the first angle mode. For example, in the case that the first angle mode is a planar mode, the prediction result corresponding to the first angle mode is determined using the planar mode, and the planar mode is a special angle mode. In the case that the index corresponding to the first angle mode is 5, the prediction result corresponding to the first angle mode is determined using an angle prediction mode.

[0169] It can be understood that, referring to FIG. 18, the first weight coefficient corresponding to the first angle mode is obtained by the following steps:

[0170] Step S510, obtaining all amplitude values corresponding to the first angle mode from the frequency histogram, wherein the frequency histogram is constructed according to the angle mode statistical information in the spatial domain.

[0171] Step S520, obtaining a total amplitude value according to the sum of all amplitude values corresponding to the first angle mode.

[0172] Step S530, traversing each first angle mode, and obtaining the first weight coefficient corresponding to the currently traversed first angle mode according to the proportion of the amplitude value corresponding to the currently traversed first angle mode to the total amplitude value.

[0173] It should be noted that the frequency histogram is constructed based on the angle mode statistical information of the spatial domain. In some embodiments, a plurality of spatial sampling points corresponding to the current CU are determined, and each CU in which the spatial sampling points are located is traversed, a second angle mode used by the current traversed CU is determined, and the area of the current CU is added to the second angle mode of the frequency histogram. Since the plurality of first angle modes are determined based on the frequency histogram, the amplitude values corresponding to all the first angle modes can be determined from the frequency histogram. Referring to FIG. 4, if n first angle modes in the plurality of first angle modes are determined by the frequency histogram, n largest amplitude values in FIG. 4 are obtained.

[0174] It should be noted that after the amplitude values corresponding to the plurality of first angle modes corresponding to the current CU are determined, the sum of the amplitude values corresponding to all the first angle modes is calculated, and the sum is taken as the total amplitude value. The total amplitude value is determined based on the first angle modes only, and is irrelevant to other amplitude values in the frequency histogram.

[0175] It should be noted that the amplitude value corresponding to the first angle mode is proportional to the first weight coefficient, that is, the larger the amplitude value corresponding to the first angle mode, the larger the first weight coefficient. Therefore, the first weight coefficient can be regarded as the ratio of the amplitude value corresponding to the first angle mode to the total amplitude value. For example, there are three first angle modes determined by the frequency histogram, and the three first angle modes are mode 0, mode 1 and mode 2, which are not direct current modes, planar modes, etc., but are the results of renumbering the three first angle modes determined by the frequency histogram. If the amplitude value of mode 0 is amp0, the amplitude value of mode 1 is amp1, and the amplitude value of mode 2 is amp2, the total amplitude value is amp0+amp1+amp2, the first weight coefficient of mode 0 can be represented as the first weight coefficient of mode 1 can be represented as the first weight coefficient of mode 2 can be represented as

[0176] It should be noted that the plurality of first angle modes include a planar mode, and the planar mode is not determined by the frequency histogram. Therefore, the first weight coefficient of the planar mode is not determined based on the amplitude values of other first angle modes. For example, when there is 1 first angle mode determined by the frequency histogram, the first weight coefficient of the first angle mode is 43, and the first weight coefficient of the planar mode is 21. When there are 2 to 5 first angle modes determined by the frequency histogram, the first weight coefficients of the other first angle modes are determined by the ratio of the amplitude value corresponding to the first angle mode to the total amplitude value, and the first weight coefficients of the planar modes are all 16.

[0177] The following illustrates the calculation of the first weight coefficient corresponding to different available number of angle mode indexes.

[0178] Example 1: 1 available angle mode index is obtained according to the histogram

[0179] The weight of mode 0 is: 43;

[0180] The weight of Planar mode is: 21.

[0181] Example 2: 2 available angle mode indexes are obtained according to the histogram

[0182] The weight of mode 0 is:

[0183] The weight of mode 1 is:

[0184] The weight of Planar mode is: 16.

[0185] Example 3: 3 available angle mode indexes are obtained according to the histogram

[0186] The weight of mode 0 is:

[0187] The weight of mode 1 is:

[0188] The weight of mode 2 is:

[0189] The weight of Planar mode is: 16.

[0190] Example 4: 4 available angle mode indexes are obtained according to the histogram

[0191] The weight of mode 0 is:

[0192] The weight of mode 1 is:

[0193] The weight of mode 2 is:

[0194] The weight of mode 3 is:

[0195] The weight of Planar mode is: 16.

[0196] Example 5: 5 available angle mode indexes are obtained according to the histogram

[0197] The weight of mode 0 is:

[0198] The weight of mode 1 is:

[0199] The mode 2 weight is:

[0200] The mode 3 weight is:

[0201] The mode 4 weight is:

[0202] The planar mode weight is: 16.

[0203] It should be noted that the embodiments of the above steps S510 to S530 determine the first weight coefficient by the amplitude value corresponding to the first angle mode in the frequency histogram, and in some embodiments, the first weight coefficient of the first angle mode is determined based on the proportion of the amplitude value corresponding to the first angle mode to the total amplitude value. The method calculates the first coefficient weight by determining the frequency histogram of the first angle mode in advance, which can shorten the calculation time of the first weight coefficient, and further improve the intra prediction efficiency.

[0204] It should be noted that after obtaining the prediction results and the first weight coefficients corresponding to each first angle mode through the above steps, the fusion prediction result of the current CU is determined based on the prediction results and the first weight coefficients corresponding to each first angle mode. The fusion prediction result of the current CU can be represented as:

[0205] Wherein, pDst is the fusion prediction result of the current CU, x represents the first angle mode, which is determined according to the number of angle modes selected from the frequency histogram, so the value range of x is [2, 6], and the plurality of first angle modes always includes the planar mode. i represents the first angle mode currently being calculated, pPred i is the prediction result corresponding to the first angle mode i used for the current CU, weight i is the first weight coefficient corresponding to the current first angle mode i.

[0206] It should be noted that the video frame prediction method provided in the embodiments of the present application first determines a plurality of spatial domain sampling points corresponding to the current CU, i.e., the CU that needs to be intra-predicted, and constructs a frequency histogram according to the angular mode information of the CU in which the plurality of spatial domain sampling points are located, to determine the first angular mode based on the n second angular modes with the highest amplitudes in the frequency histogram and the planar mode. Then, for each first angular mode, if the index of the first angular mode belongs to the first index, the prediction result corresponding to the first angular mode used by the current CU is determined using the matrix-based intra-prediction mode, wherein the first index includes 1 and (2+2*k), and k is an integer in the interval [0, 32], otherwise the prediction result corresponding to the first angular mode used by the current CU is determined using the angular prediction mode. In addition, the first weight coefficient corresponding to the first angular mode also needs to be determined. There are two methods for calculating the first weight coefficient, one of which is determined based on the ratio of the prediction loss value corresponding to the first angular mode to the total prediction loss value, and the other is determined based on the ratio of the amplitude value of the first angular mode in the frequency histogram to the total amplitude value. Finally, the fusion prediction result of the current CU is determined according to the prediction result corresponding to each first angular mode and the first weight coefficient. The video frame prediction method provided in the embodiments of the present application can combine the advantages of multiple first angular mode predictions to accurately predict video blocks with complex texture structures, thereby improving the accuracy of intra-prediction. In addition, the first weight coefficient corresponds to the first angular mode, which can better allocate each prediction result, making the resulting fusion prediction result more accurate and improving the prediction accuracy of video blocks with complex texture structures.

[0207] In addition, the embodiments of the present application also provide a video bitstream processing method, as shown in FIG. 19, the video bitstream processing method comprises:

[0208] Step S610, set the preset switch flag in the encoding parameter information to an open flag value, the preset switch flag is used to indicate whether to perform fusion prediction, wherein the fusion prediction is realized based on the above-mentioned video frame prediction method.

[0209] Step S620, write the encoding parameter information into the video bitstream.

[0210] It should be noted that video bitstream processing includes video encoding processing, video encoding refers to converting a file in the original video format into another file in another video format through compression technology. The encoding parameter information is the relevant parameter information of the video encoding processing.

[0211] It should be noted that the preset switch flag is used to indicate whether to perform fusion prediction. For example, in the framework diagram of the video code stream processing in the encoding process shown in FIG. 20, the preset switch flag is provided, and the preset switch flag is used to select whether the processing mode of the video code stream is intra prediction or inter prediction, that is, to indicate whether to perform intra prediction.

[0212] It should be noted that after the encoding mode is determined, the encoding parameter information is written into the video code stream to facilitate decoding.

[0213] It should be noted that the fusion prediction in the video code stream processing method provided by the embodiment of the present application is based on the video frame prediction method described above, and the video frame prediction method can improve the prediction accuracy of the video block with complex texture results. Therefore, the video code stream processing method improves the encoding accuracy of the video block with complex texture results and improves the processing efficiency of the video code stream.

[0214] It can be understood that the encoding parameter information includes general constraint information (GCI), sequence parameter set (SPS), picture parameter set (PPS), and coding unit (CU) definition. Referring to FIG. 21, step S610 includes at least one of the following:

[0215] In step S611, a first preset switch flag in the general constraint information GCI is set to an open flag value to indicate that fusion prediction is performed on all video sequences.

[0216] In step S612, a second preset switch flag in the sequence parameter set SPS is set to an open flag value to indicate that fusion prediction is performed on the current video sequence.

[0217] In step S613, a third preset switch flag in the picture parameter set PPS is set to an open flag value to indicate that fusion prediction is performed on the current picture.

[0218] In step S614, a fourth preset switch flag in the CU definition is set to an open flag value to indicate that fusion prediction is performed on the current CU.

[0219] It should be noted that the general constraint information GCI contains common encoding parameters of all video sequences. Therefore, the first preset switch flag in the general constraint information GCI is used to indicate whether to perform fusion prediction on all video sequences. When the first preset switch flag is set to the open flag value, fusion prediction needs to be performed on all video sequences.

[0220] It should be noted that the sequence parameter set SPS contains common coding parameters of the current video sequence. Once the sequence parameter set SPS is referenced, all pictures in the current video sequence are coded using the coding parameters of the sequence parameter set SPS. Therefore, when the second preset switch flag in the sequence parameter set SPS is set to the on flag value, fusion prediction needs to be performed on the current video sequence.

[0221] It should be noted that the picture parameter set PPS contains common coding parameters of the current picture, and therefore the third preset switch flag in the picture parameter set PPS is used to indicate whether fusion prediction is performed on the current picture. In the case where the third preset switch flag is set to the on flag value, fusion prediction is performed on the current picture.

[0222] It should be noted that the CU definition corresponds to the current CU. If the fourth preset switch flag in the CU definition is set to the on flag value, fusion prediction is performed on the current CU.

[0223] It should be noted that when the preset switch flag is set to the off flag value, it indicates that inter-frame prediction is performed on the data corresponding to the preset switch flag.

[0224] It should be noted that the embodiments of the above steps S611 to S614 perform fusion prediction on the related data in the video code stream through the general restriction information GCI, the sequence parameter set SPS, the picture parameter set PPS, and the preset switch flag in the coding unit CU definition, thereby improving the processing accuracy of the video code stream.

[0225] In addition, the embodiments of the present application also provide a video code stream processing method, as shown in FIG. 22, which comprises:

[0226] Step S710, receiving a video code stream and obtaining coding parameter information from the video code stream.

[0227] Step S720, performing fusion prediction according to the preset switch flag in the coding parameter information, wherein the fusion prediction is realized based on the video frame prediction method described above.

[0228] It should be noted that the video code stream is the coding result obtained by the video code stream processing method in the above steps S610 and S610. The video code stream is received and decoded, and the coding parameter information can be obtained. Video code stream processing includes video decoding, that is, converting the compressed digital video signal into playable video content.

[0229] It should be noted that the preset switch flag is set in the coding parameter information, for example, in the frame diagram shown in FIG. 23, the preset switch flag is set, and the preset switch flag is used to select whether the processing mode of the video code stream is intra prediction or inter prediction, that is, whether to perform intra prediction.

[0230] It should be noted that the fusion prediction in the video code stream processing method provided by the embodiments of the present application is based on the video frame prediction method described above, and the video frame prediction method can improve the prediction accuracy of the video block with complex texture results. Therefore, the video code stream processing method improves the decoding accuracy of the video block with complex texture results and improves the processing efficiency of the video code stream.

[0231] It can be understood that the coding parameter information includes general constraint information GCI, sequence parameter set SPS, picture parameter set PPS, and coding unit CU definition. Referring to FIG. 24, step S720 includes one of the following:

[0232] Step S721, when the first preset switch flag in the general constraint information GCI is an opening flag value, performing fusion prediction on all video sequences.

[0233] Step S722, when the second preset switch flag in the sequence parameter set SPS is an opening flag value, performing fusion prediction on the current video sequence.

[0234] Step S723, when the third preset switch flag in the picture parameter set PPS is an opening flag value, performing fusion prediction on the current picture.

[0235] Step S724, when the fourth preset switch flag in the CU definition is an opening flag value, performing fusion prediction on the current CU.

[0236] It should be noted that the respective preset switch flags corresponding to the general constraint information GCI, the sequence parameter set SPS, the picture parameter set PPS, and the coding unit CU definition in the coding parameter information are consistent with the preset switch flags in steps S611 to S614.

[0237] It should be noted that the general constraint information GCI contains common coding parameters of all video sequences. Therefore, the first preset switch flag in the general constraint information GCI is used to indicate whether to perform fusion prediction on all video sequences. When the first preset switch flag is set to an opening flag value, fusion prediction needs to be performed on all video sequences.

[0238] It should be noted that the sequence parameter set SPS contains common coding parameters of the current video sequence. Once the sequence parameter set SPS is referenced, all pictures in the current video sequence are coded using the coding parameters of the sequence parameter set SPS. Therefore, when the second preset switch flag in the sequence parameter set SPS is set to an open flag value, fusion prediction needs to be performed on the current video sequence.

[0239] It should be noted that the picture parameter set PPS contains common coding parameters of the current picture, and therefore, the third preset switch flag in the picture parameter set PPS is used to indicate whether fusion prediction is performed on the current picture. In the case where the third preset switch flag is set to an open flag value, fusion prediction is performed on the current picture.

[0240] It should be noted that the CU definition corresponds to the current CU. If the fourth preset switch flag in the CU definition is set to an open flag value, fusion prediction is performed on the current CU.

[0241] It should be noted that when the preset switch flag is set to a closed flag value, it indicates that inter-frame prediction is performed on the data corresponding to the preset switch flag.

[0242] It should be noted that the embodiments of the above steps S721 to S724 perform fusion prediction on the related data in the video code stream based on the general constraint information GCI, the sequence parameter set SPS, the picture parameter set PPS, and the preset switch flag in the coding unit CU definition, thereby improving the processing accuracy of the video code stream.

[0243] The embodiments of the present application also provide an electronic device, as shown in FIG. 25, the electronic device 1400 includes:

[0244] one or more processors 1410;

[0245] a memory 1420, on which one or more programs are stored, when the one or more programs are executed by the one or more processors 1410, the one or more processors 1410 implement the video frame prediction method provided by any one of the embodiments of the present application, or implement the processing method of the video code stream provided by any one of the embodiments of the present application.

[0246] The memory 1420, as a non-transitory network system, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory 1420 can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory 1420 can include a memory 1420 remotely arranged relative to the processor 1410, which can be connected to the processor 1410 through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0247] The memory 1420 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1420 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1420 and are called and executed by the processor 1410 to implement the method of the embodiments of the present application.

[0248] The processor 1410 can be implemented in the form of a general-purpose CPU (central processing unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., for executing related programs to implement the technical solutions provided by the embodiments of the present application.

[0249] In some embodiments, the electronic device further includes:

[0250] An input / output interface for realizing information input and output;

[0251] A communication interface for realizing communication interaction between the device and other devices, which can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0252] A bus for transmitting information between various components (such as the processor 1410, the memory 1420, the input / output interface, and the communication interface) of the device;

[0253] The processor 1410, the memory 1420, the input / output interface, and the communication interface can realize communication connection between each other inside the device through the bus.

[0254] An embodiment of the present application further provides a computer readable storage medium, which stores computer executable instructions for implementing the video frame prediction method provided by any one of the embodiments of the present application or implementing the video bitstream processing method provided by any one of the embodiments of the present application.

[0255] An embodiment of the present application further provides a computer program product, which comprises a computer program or computer instructions stored in a computer readable storage medium, and a processor of a computer device reads the computer program or computer instructions from the computer readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device implements the video frame prediction method provided by any one of the embodiments of the present application or implements the video bitstream processing method provided by any one of the embodiments of the present application.

[0256] The embodiments of the present application provide a video frame prediction method, a video bitstream processing method and related devices, and aim to improve the prediction accuracy of a video block with a complex texture structure.

[0257] The embodiments of the present application first determine a plurality of first angle modes based on spatial angle mode statistical information. Then, according to the indexes of the first angle modes, the embodiments determine the prediction results corresponding to the first angle modes used by a current CU (Coding Unit, coding unit), and further determine the fusion prediction result of the current CU according to the prediction results corresponding to the first angle modes and first weight coefficients. In the embodiments of the present application, the fusion prediction result of the current CU is determined based on the prediction results corresponding to the plurality of first angle modes, so that the video frame prediction method can combine the advantages of the plurality of first angle modes to accurately predict the video block with a complex texture structure, thereby improving the prediction accuracy. In addition, the first weight coefficients correspond to the first angle modes, which can better distribute the prediction results, so that the obtained fusion prediction result is more accurate, and the prediction accuracy of the video block with a complex texture structure is improved.

[0258] The system architecture and application scenarios described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It can be known by those skilled in the art that, with the evolution of system architecture and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0259] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, databases, or other media in this application is intended to include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0260] Those skilled in the art can understand that all or some steps of the above-mentioned methods and systems can be implemented as software, firmware, hardware and appropriate combinations thereof. Some or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer readable medium, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, those skilled in the art know that communication media generally includes computer readable instructions, data structures, program modules or other data in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium.

[0261] The above describes some embodiments of the present application with reference to the accompanying drawings, and is not limited to the scope of the present application. Any modification, equivalent replacement and improvement made by those skilled in the art without departing from the scope and essence of the present application shall be within the scope of the present application.

Claims

1. A method for video frame prediction, the method comprising: determining a plurality of first angular modes according to angular mode statistics information in a spatial domain; determining a prediction result corresponding to the first angular mode according to an index of the first angular mode for a current CU; and determining a fusion prediction result of the current CU according to the prediction result corresponding to each of the first angular modes and a first weight coefficient. The determining of the plurality of first angular modes according to the angular mode statistics information in the spatial domain comprises: determining a plurality of spatial sampling points; constructing a frequency histogram according to angular mode information of a CU in which each of the spatial sampling points is located; and determining the plurality of first angular modes according to the frequency histogram. The spatial sampling points comprise at least one of a neighboring sampling point and a non-neighboring sampling point. The neighboring sampling point comprises a neighboring pixel located above, above-left, left, below-left or above-right of the current CU.

2. The method of claim 1, wherein, The non-neighboring sampling point comprises a non-neighboring pixel located above, above-left, left, below-left or above-right of the current CU. The constructing of the frequency histogram according to the angular mode information of the CU in which each of the spatial sampling points is located comprises: traversing each of the CUs in which the spatial sampling points are located and determining a second angular mode used by a current traversed CU; and adding an area of the current traversed CU to the second angular mode in the frequency histogram. The determining of the first angular modes according to the frequency histogram comprises: determining n second angular modes with the highest amplitudes according to the frequency histogram, wherein n is greater than or equal to 1 and less than or equal to 5; and determining the first angular modes according to the n second angular modes, wherein the first angular modes at least comprise a planar mode. The determining of the prediction result corresponding to the first angular mode according to the index of the first angular mode for the current CU comprises: in a case where the index of the first angular mode belongs to a first index, determining the prediction result corresponding to the first angular mode using a matrix-based intra prediction mode.

3. The method of claim 2, wherein, The determining of the prediction result corresponding to the first angular mode using the matrix-based intra prediction mode comprises: obtaining a second weight coefficient corresponding to a target position in the current CU and the first angular mode from a weight matrix; obtaining a reference pixel corresponding to the target position according to the first angular mode; obtaining a prediction pixel of the target position according to the second weight coefficient and the reference pixel; and obtaining the prediction result corresponding to the first angular mode according to prediction pixels of each of the target positions in the current CU.

4. The method of claim 3, wherein, The determining of the prediction result corresponding to the first angular mode according to the index of the first angular mode for the current CU further comprises: in a case where the index of the first angular mode does not belong to the first index, determining the prediction result corresponding to the first angular mode using an angular prediction mode.

5. The method of claim 3, wherein, The first weight coefficient corresponding to the first angular mode is obtained by: obtaining prediction loss values corresponding to all of the first angular modes; obtaining a total prediction loss value according to a sum of the prediction loss values corresponding to all of the first angular modes; and obtaining the first weight coefficient corresponding to the first angular mode according to the total prediction loss value.

6. The method of claim 2, wherein, ​ ​ ​ 7. The method of claim 2, wherein, ​ ​ ​ 8. The method of claim 1, wherein, ​ ​ 9. The method of claim 8, wherein, ​ ​ ​ ​ ​ 10. The method of claim 1, wherein, ​ ​ 11. The method of claim 1, wherein, ​ ​ ​ For each of the first angle modes, a first weight coefficient corresponding to the first angle mode is obtained according to a prediction loss value corresponding to the first angle mode and the total prediction loss value.

12. The method of claim 11, wherein, The prediction loss value corresponding to the first angle mode is obtained by the following steps: determining a prediction pixel value of a template region based on the first angle mode; obtaining a reconstructed pixel value of the template region; determining the prediction loss value corresponding to the first angle mode according to the prediction pixel value and the reconstructed pixel value of the template region.

13. The method of claim 12, wherein, The step of determining the prediction pixel value of the template region based on the first angle mode comprises: in a case where the index of the first angle mode belongs to a first index, determining the prediction pixel value of the template region using a matrix-based intra prediction mode; or, in a case where the index of the first angle mode does not belong to the first index, determining the prediction pixel value of the template region using an angle prediction mode.

14. The method of claim 1, wherein, The first weight coefficient corresponding to the first angle mode is obtained by the following steps: obtaining all amplitude values corresponding to the first angle modes from a frequency histogram, wherein the frequency histogram is constructed according to the angle mode statistical information of the spatial domain; obtaining a total amplitude value according to a sum of all the amplitude values corresponding to the first angle modes; iterating through each of the first angle modes, and obtaining a first weight coefficient corresponding to a currently iterated first angle mode according to a ratio of an amplitude value corresponding to the currently iterated first angle mode to the total amplitude value.

15. A video bitstream processing method, the method comprising: setting a preset switch flag in coding parameter information to an on flag value, the preset switch flag being used to indicate whether to perform fusion prediction, wherein the fusion prediction is implemented based on the video frame prediction method according to any one of claims 1-14; writing the coding parameter information into a video bitstream.

16. The method of claim 15, wherein, The coding parameter information comprises general constraint information GCI, sequence parameter set SPS, picture parameter set PPS, and coding unit CU definition; and the step of setting the preset switch flag in the coding parameter information to the on flag value comprises at least one of the following: setting a first preset switch flag in the GCI to the on flag value to indicate that the fusion prediction is performed on all video sequences; or setting a second preset switch flag in the SPS to the on flag value to indicate that the fusion prediction is performed on a current video sequence; or setting a third preset switch flag in the PPS to the on flag value to indicate that the fusion prediction is performed on a current picture; or setting a fourth preset switch flag in the CU definition to the on flag value to indicate that the fusion prediction is performed on a current CU.

17. A video bitstream processing method, the method comprising: receiving a video bitstream and obtaining coding parameter information from the video bitstream; performing fusion prediction according to a preset switch flag in the coding parameter information, wherein the fusion prediction is implemented based on the video frame prediction method according to any one of claims 1-14.

18. The method of claim 17, wherein, The coding parameter information comprises general constraint information (GCI), sequence parameter set (SPS), picture parameter set (PPS) and coding unit (CU) definition; and the performing the fusion prediction according to a preset switch flag in the coding parameter information comprises one of the following: when a first preset switch flag in the GCI is an opening flag value, performing the fusion prediction on all video sequences; or when a second preset switch flag in the SPS is an opening flag value, performing the fusion prediction on a current video sequence; or when a third preset switch flag in the PPS is an opening flag value, performing the fusion prediction on a current picture; or when a fourth preset switch flag in the CU definition is an opening flag value, performing the fusion prediction on a current CU. 19.An electronic device, comprising: one or more processors; a memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the video frame prediction method according to any one of claims 1-14, or the video bitstream processing method according to any one of claims 15-16, or the video bitstream processing method according to any one of claims 17-18. 20.A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the program implements the video frame prediction method according to any one of claims 1-14, or the video bitstream processing method according to any one of claims 15-16, or the video bitstream processing method according to any one of claims 17-18. 21.A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the video frame prediction method according to any one of claims 1-14, or the video bitstream processing method according to any one of claims 15-16, or the video bitstream processing method according to any one of claims 17-18.

Images