Methods and equipment for forecasting decoding, methods for forecasting encoding, encoding systems, and computer-readable storage media.

Abstract

The invention relates to a predictive method and apparatus for decoding, a predictive method for encoding, an encoding system, and a computer-readable storage medium. This method comprises the following steps: obtaining adjacent reference pixels at least one edge of the block to be decoded to obtain the first set of adjacent reference pixels; identifying the reference point from at least one edge, and determining the positions of the reference pixels to be selected corresponding to at least one edge according to a predefined number of samples; based on the positions of the reference pixels to be selected, selecting reference pixels corresponding to the positions of the reference pixels to be selected from the first set of adjacent reference pixels, and generating a subset of adjacent reference pixels from the selected reference pixels; and based on the subset of adjacent reference pixels, performing predictive decoding on the block to be decoded.

Description

Decoding prediction method, device and computer storage medium TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of video coding, and in particular to a decoding prediction method, device and computer storage medium. BACKGROUND

[0002] With the increasing demand for video display quality, new video application forms such as high definition and ultra-high definition video have emerged. H.265 / High Efficiency Video Coding (HEVC) is the latest international video compression standard, and the compression performance of H.265 / HEVC is about 50% higher than that of the previous generation video coding standard H.264 / Advanced Video Coding (AVC), but still cannot meet the needs of the rapid development of video applications, especially new video applications such as ultra-high definition and virtual reality (VR).

[0003] In 2015, the Video Coding Experts Group of ITU-T and the Moving Picture Experts Group of ISO / IEC established the Joint Video Research Group (JVET) to develop the next generation of video coding standards. The Joint Exploration Test Model (JEM) is a general reference software platform, and different coding tools are verified based on this platform. In April 2018, JVET officially named the next generation of video coding standards as Versatile Video Coding (VVC), and its corresponding test model is VTM. In the JEM and VTM reference software, a coding prediction method based on a linear model has been integrated, which predicts the chroma component from the luminance component of the current to-be-decoded block through the linear model. However, when constructing the linear model, the neighboring reference pixel subset constructed by the neighboring reference pixels is not very reasonable, resulting in high search complexity and reducing the decoding prediction performance of the video image.

[0004] SUMMARY

[0005] Therefore, embodiments of the present application expect to provide a decoding prediction method, device and computer storage medium. Since the neighboring reference pixels in the neighboring reference pixel subset not only take into account the importance and dispersion, but also the number of pixels in the neighboring reference pixel subset is small, the complexity of the search is reduced, the prediction performance of the video image decoding is improved, and the bit rate is further reduced.

[0006] The technical solution of the embodiments of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding prediction method, the method comprising:

[0008] obtaining reference pixels adjacent to at least one side of a to-be-decoded block to obtain a first adjacent reference pixel set;

[0009] determining a reference point from the at least one side, and determining a to-be-selected reference pixel position corresponding to the at least one side according to a preset sampling number;

[0010] selecting, based on the to-be-selected reference pixel position, reference pixels corresponding to the to-be-selected reference pixel position from the first adjacent reference pixel set to form an adjacent reference pixel subset;

[0011] performing prediction decoding on the to-be-decoded block based on the adjacent reference pixel subset.

[0012] In a second aspect, an embodiment of the present application provides a decoding prediction device, the decoding prediction device comprising an obtaining unit, a determining unit, a selecting unit and a decoding unit,

[0013] The obtaining unit is configured to obtain reference pixels adjacent to at least one side of a to-be-decoded block to obtain a first adjacent reference pixel set;

[0014] The determining unit is configured to determine a reference point from the at least one side, and determine a to-be-selected reference pixel position corresponding to the at least one side according to a preset sampling number;

[0015] The selecting unit is configured to select, based on the to-be-selected reference pixel position, reference pixels corresponding to the to-be-selected reference pixel position from the first adjacent reference pixel set to form an adjacent reference pixel subset;

[0016] The decoding unit is configured to perform prediction decoding on the to-be-decoded block based on the adjacent reference pixel subset.

[0017] In a third aspect, an embodiment of the present application provides a decoding prediction device, the decoding prediction device comprising a memory and a processor.

[0018] The memory is configured to store a computer program capable of running on the processor;

[0019] The processor is configured to execute steps of the method according to the first aspect when the computer program is running.

[0020] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a decoding prediction program, which, when executed by at least one processor, implements the steps of the method in the first aspect.

[0021] The embodiment of the present application provides a decoding prediction method and device and a computer storage medium. Firstly, reference pixel points adjacent to at least one side of a to-be-decoded block are acquired to obtain a first adjacent reference pixel set. Then, a reference point is determined from the at least one side, and a to-be-selected reference pixel position corresponding to the at least one side is determined according to a preset sampling number. Then, based on the to-be-selected reference pixel position, reference pixel points corresponding to the to-be-selected reference pixel position are selected from the first adjacent reference pixel set to form an adjacent reference pixel subset. Finally, the to-be-decoded block is predicted and decoded based on the adjacent reference pixel subset. Since the selection of the adjacent reference pixel points in the adjacent reference pixel subset takes into account the importance and dispersion, the model parameters constructed based on the adjacent reference pixel subset are relatively accurate, so that the prediction performance of video image decoding can be improved. Moreover, the number of pixels in the adjacent reference pixel subset is relatively small, so that the complexity of search is reduced, the prediction performance of video image decoding is improved, and the bit rate is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0022] FIGS. 1A-1C are structural schematic diagrams of video image sampling formats in related technical solutions;

[0023] FIGS. 2A and 2B are sampling schematic diagrams of adjacent reference values of first image components and adjacent reference values of second image components of a decoding block in related technical solutions;

[0024] FIG. 3 is a structural schematic diagram of constructing a prediction model based on maximum values and minimum values of a decoding block in a related technical solution;

[0025] FIGS. 4A and 4B are structural schematic diagrams of selecting adjacent reference pixel points of a square decoding block and a non-square decoding block respectively provided in a related technical solution;

[0026] FIGS. 5A and 5B are structural schematic diagrams of selecting adjacent reference pixel points respectively provided in a traditional technical solution and an L0138 proposal;

[0027] FIG. 6 is a component block diagram of a video encoding system provided in an embodiment of the present application;

[0028] FIG. 7 is a component block diagram of a video decoding system provided in an embodiment of the present application;

[0029] FIG. 8 is a flowchart of a decoding prediction method provided in an embodiment of the present application;

[0030] FIG. 9 is a structure diagram of selecting a subset of neighboring reference pixels corresponding to an upper side of a to-be-decoded block according to an embodiment of the present application;

[0031] FIG. 10 is a structure diagram of selecting a subset of neighboring reference pixels corresponding to a left side of a to-be-decoded block according to an embodiment of the present application;

[0032] FIG. 11 is a structure diagram of selecting a subset of neighboring reference pixels corresponding to an upper side of a to-be-decoded block according to another embodiment of the present application;

[0033] FIG. 12 is a structure diagram of selecting a subset of neighboring reference pixels corresponding to an upper side of a to-be-decoded block according to yet another embodiment of the present application;

[0034] FIG. 13 is a structure diagram of selecting a subset of neighboring reference pixels corresponding to an upper side of a to-be-decoded block according to still another embodiment of the present application;

[0035] FIG. 14 is a structure diagram of selecting a subset of neighboring reference pixels corresponding to an upper side of a to-be-decoded block according to still another embodiment of the present application;

[0036] FIG. 15 is a structure diagram of a decoding prediction device according to an embodiment of the present application;

[0037] FIG. 16 is a hardware structure diagram of a decoding prediction device according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable a person skilled in the art to more fully understand the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings, which are used only for reference and do not limit the embodiments of the present application.

[0039] In a video image, a first image component, a second image component and a third image component are generally used to represent a decoded block; wherein the three image components are respectively a luminance component, a blue chrominance component and a red chrominance component, specifically, the luminance component is usually represented by a symbol Y, the blue chrominance component is usually represented by a symbol Cb, and the red chrominance component is usually represented by a symbol Cr.

[0040] In the embodiments of the present application, the first image component can be the luminance component Y, the second image component can be the blue chrominance component Cb, and the third image component can be the red chrominance component Cr, but the embodiments of the present application are not limited in this regard. Currently commonly used sampling formats are YCbCr formats, which include the following, as shown in FIGS. 1A to 1C, wherein the cross (X) in the figure represents a first image component sampling point, and the circle (O) represents a second image component or third image component sampling point. The YCbCr formats include:

[0041] 4:4:4 format: as shown in FIG. 1A, indicating that the second image component or the third image component is not down-sampled; it is that 4 sample samples of the first image component, 4 sample samples of the second image component and 4 sample samples of the third image component are taken every 4 consecutive pixels on each scan line;

[0042] 4:2:2 format: as shown in FIG. 1B, indicating that the first image component is horizontally sampled by 2:1 with respect to the second image component or the third image component, and there is no vertical down-sampling; it is that 4 sample samples of the first image component, 2 sample samples of the second image component and 2 sample samples of the third image component are taken every 4 consecutive pixels on each scan line;

[0043] 4:2:0 format: as shown in FIG. 1C, indicating that the first image component is horizontally down-sampled by 2:1 and vertically down-sampled by 2:1 with respect to the second image component or the third image component; it is that 2 sample samples of the first image component, 1 sample sample of the second image component and 1 sample sample of the third image component are taken every 2 consecutive pixels on the horizontal scan line and the vertical scan line.

[0044] In the case where the video image adopts YCbCr in the 4:2:0 format, if the first image component of the video image is a decoding block of 2N*2N size, the corresponding second image component or third image component is a decoding block of N*N size, where N is the side length of the decoding block. In the embodiments of the present application, the following will be described taking the 4:2:0 format as an example, but the technical solutions of the embodiments of the present application are also applicable to other sampling formats.

[0045] In the next-generation video coding standard H.266, in order to further improve the coding performance, the Cross-component Prediction (CCP) is extended and improved, and the Cross-component Linear Model Prediction (CCLM) is proposed. In H.266, the CCLM realizes the prediction between the first image component and the second image component, the first image component and the third image component, and the second image component and the third image component, and the following will be described taking the prediction from the first image component to the second image component as an example, but the technical solutions of the embodiments of the present application are also applicable to the prediction of other image components.

[0046] It can be understood that, in order to reduce the redundancy between the first image component and the second image component, the prediction mode of CCLM is used in VTM, at this time the first image component and the second image component are of the same decoding block, the second image component is predicted based on the first image component reconstruction value of the same decoding block, for example, using the prediction model as shown in formula (1):

[0047] Pred C [i,j] = a Rec L [i,j] + b (1)

[0048] where i, j represent the position coordinates of the pixel points in the decoded block, i represents the horizontal direction, j represents the vertical direction, Pred C [i,j] represents the second image component prediction value corresponding to the pixel point with position coordinates [i, j] in the decoded block, Rec L [i,j] represents the first image component reconstruction value corresponding to the pixel point with position coordinates [i, j] (after downsampling) in the same decoded block, and a and b are model parameters of the above prediction model.

[0049] There are multiple ways to construct the model parameters a and b, which can be a regression construction method based on least squares, a maximum and minimum construction method, or even other construction methods, and the embodiments of the present application do not make specific limitations thereon. The following will be described respectively based on the regression construction method based on least squares and the maximum and minimum construction method.

[0050] In VVC, a derivation method of a regression construction model parameter based on least squares is initially accepted, and specifically, a and b can be derived by minimizing the regression error of the first image component adjacent reference values and the second image component adjacent reference values around the decoded block, as shown in the following formula (2):

[0051]

[0052] L(n) = L(n-1) - L(n+1) (1) C(n) = C(n-1) - C(n+1) (2) wherein, L(n) represents the first image component adjacent reference value corresponding to the down-sampled left side and upper side, C(n) represents the second image component adjacent reference value corresponding to the left side and upper side, N is the side length of the second image component decoding block, n = 1, 2, …, 2N. Referring to FIG. 2A and FIG. 2B, which respectively show the sampling schematic diagram of the first image component adjacent reference value and the second image component adjacent reference value of the decoding block in the related technical solution; wherein, in FIG. 2A, the larger box in bold is used to highlight the indication of the first image component decoding block 21, and the gray solid circle is used to indicate the adjacent reference value L(n) of the first image component decoding block 21; in FIG. 2B, the larger box in bold is used to highlight the indication of the second image component decoding block 22, and the gray solid circle is used to indicate the adjacent reference value C(n) of the second image component decoding block 22. FIG. 2A shows a 2N×2N size first image component decoding block 21, and for a 4:2:0 format video image, a 2N×2N size first image component corresponds to a N×N size second image component, as shown by 22 in FIG. 2B; that is, FIG. 2A and FIG. 2B are the decoding block schematic diagrams obtained by respectively sampling the first image component and the second image component for the same decoding block.

[0053] In VVC, a derivation method of a simplified model parameter is recently accepted, specifically, the model parameters can be derived according to the principle of “two points determine a line” by searching for the maximum first image component adjacent reference value and the minimum first image component adjacent reference value, as shown in the following formula (3) of α and β:

[0054]

[0055] wherein, L max and L min represent the maximum value and the minimum value searched from the down-sampled first image component adjacent reference values corresponding to the left side and the upper side, C max and C min represent the second image component adjacent reference values corresponding to the reference pixel points at the positions of L max and L min . Referring to FIG. 3, which shows the structure schematic diagram of the prediction model constructed based on the maximum value and the minimum value in the related technical solution; wherein, the abscissa represents the first image component adjacent reference value of the decoding block, and the ordinate represents the second image component adjacent reference value of the decoding block, according to L max and L min and C max and C minThe model parameters a and b can be calculated by formula (3), and the constructed prediction model is C = a · L + b; where L represents the reconstructed value of the first image component corresponding to one pixel point in the decoding block, and C represents the prediction value of the second image component corresponding to the pixel point in the decoding block.

[0056] There are various cases in the prior art for the construction of the set of adjacent reference pixel points in the CCLM, which will be described below.

[0057] (a) Differentiation from the shape of the decoding block

[0058] Referring to FIG. 4A, a structure diagram of selecting adjacent reference pixel points for a square decoding block is shown according to the related technical solution; as shown in FIG. 4A, the decoding block is a square decoding block, and all the adjacent pixel points corresponding to the left side and the upper side of the decoding block can be used as reference pixel points; for the first image component, downsampling is first needed, so that the first image component after downsampling has the same resolution as the second image component. In FIG. 4A, the gray solid circles are used to represent the adjacent reference pixel points selected by the square decoding block.

[0059] Referring to FIG. 4B, a structure diagram of selecting adjacent reference pixel points for a non-square decoding block is shown according to the related technical solution; as shown in FIG. 4B, the decoding block is a non-square decoding block, and the width and the height of the decoding block are not equal. On the one hand, for the first image component, downsampling is first needed, so that the first image component after downsampling has the same resolution as the second image component; on the other hand, the adjacent pixel points corresponding to the long side of the decoding block need to be further downsampling, so that the number of adjacent reference pixel points obtained by the long side is equal to the number of adjacent reference pixel points of the short side. In FIG. 4B, the gray solid circles are used to represent the adjacent reference pixel points selected by the non-square decoding block.

[0060] (b) Differentiation from the existence of the adjacent pixel points corresponding to the left side or the upper side of the decoding block

[0061] When the adjacent pixel points corresponding to the left side and the upper side of the decoding block can both be used, the pixel points adjacent to the upper side in a row and the adjacent pixel points adjacent to the left side in a column can both be used as adjacent reference pixel points;

[0062] When the adjacent pixel points corresponding to the left side or the upper side of the decoding block can only be used on one side, the adjacent pixel points corresponding to the effective side are used as adjacent reference pixel points;

[0063] When none of the neighboring pixels corresponding to the left and top sides of the decoded block can be used, i.e. there is no neighboring reference pixel, the model parameter a is set to 0 and the model parameter b is set to the middle value 512 of the second image component, i.e. the prediction value of the second image component corresponding to all the pixels in the decoded block is 512.

[0064] It should be noted that, assuming N is defined as the length of the short side of the second image component decoded block, when the neighboring pixels corresponding to the left and top sides of the decoded block can be used, there are totally 2N neighboring reference pixels that can be used in CCLM. In addition, unless otherwise specified, the following description is based on the case that the neighboring pixels corresponding to the left and top sides of the decoded block can be used.

[0065] (c) Scheme of neighboring reference pixel subset (reduction)

[0066] In the process of calculating the model parameters of CCLM, the required operation complexity is proportional to the number of neighboring reference pixels used in CCLM. Thus, in order to reduce the operation complexity, the L0138 proposal of the 12th meeting of JVET proposes a technical scheme for reducing the number of neighboring reference pixels used in CCLM based on the size of the second image component block corresponding to the decoded block. Referring to Table 1, it shows a relationship table between the size of the second image component block corresponding to the decoded block and the number of neighboring reference pixels provided by the related technical scheme. In Table 1, N1 is the number of neighboring reference pixels used in CCLM in the traditional technical scheme, and N2 is the number of neighboring reference pixels used in CCLM after reduction provided by the L0138 proposal. Specifically, the reduction method in the L0138 proposal is to obtain the neighboring reference pixels in the traditional technical scheme and then downsample them to obtain the neighboring reference pixels.

[0067] Table 1

[0068] Size of second image component block N1N22×2422×n / n×2(n>2)444×4 / 4×n / n×4(n>4)888×8 / 8×n / n×8(n>8)16816×16 / 16×n / n×16(n>16)32832×32648

[0069] (d) Down-sampling scheme of neighboring reference pixels

[0070] For non-square decoded blocks, the VTM has already provided a down-sampling scheme for the long side (as shown in FIG. 4B). As described above, the L0138 proposal further proposes a down-sampling set for further reduction.

[0071] In the L0138 proposal, the selection of adjacent reference pixels in the downsampling process has an impact on the decoding prediction performance. The subset of adjacent reference pixels obtained according to the default downsampling scheme in VTM does not have ideal decoding prediction performance, and therefore the L0138 proposal proposes to use another subset of adjacent reference pixels in the downsampling process. Specifically, in the traditional technical scheme used in VTM, when downsampling the long side of a non-square decoding block, the adjacent row of pixels corresponding to the upper side of the decoding block is selected from the leftmost edge, and the adjacent column of pixels corresponding to the left side is selected from the uppermost edge. In the L0138 proposal, the selection is from the opposite edge, as shown in the point selection structure examples of FIGS. 5A and 5B.

[0072] Referring to FIG. 5A, a structure diagram of a traditional technical scheme for selecting adjacent reference pixels is shown. As shown in FIG. 5A, the adjacent row of pixels corresponding to the long side of the decoding block is selected from the leftmost edge. Referring to FIG. 5B, a structure diagram of an L0138 proposal for selecting adjacent reference pixels is shown. As shown in FIG. 5B, the adjacent row of pixels corresponding to the long side of the decoding block is selected from the rightmost edge. The sampling interval is the same as in the traditional technical scheme, and the embodiments of the present application will not be repeated. Using the point selection scheme of the L0138 proposal, the value range of all adjacent reference points can be covered in the model parameter calculation of CCLM, but the importance of each adjacent reference pixel is not considered, and under the premise of a limited number, the overall characteristics of the long side cannot be represented to the maximum extent.

[0073] In the related technical solution, on the one hand, from the aspect of operation complexity, in VTM, although the maximum first image component adjacent reference value and the minimum first image component adjacent reference value are searched from 2N points, the model parameters are derived according to the principle of "two points determine a line" (see formula (3)). This solution only uses two adjacent reference pixel points, and the operation complexity of constructing model parameters based on the least square method regression is greatly reduced; however, this solution still has considerable complexity, mainly because the maximum value and the minimum value need to be determined in the adjacent reference pixel point set, thereby 4N comparison operation operations are needed to determine the maximum value and the minimum value. Moreover, the larger the length of the decoding block is, the larger the number of adjacent reference pixel points used for CCLM is, resulting in that the number of searches for determining the maximum value and the minimum value is also larger, so the operation complexity brought by this solution is still relatively large. On the other hand, from the aspect of prediction accuracy, since in the process of searching the maximum value and the minimum value, if the searched pixel point has low correlation with the current decoding block, it can be confirmed that the pixel point is a bad point. In this way, if the two searched pixel points contain a bad point, a large model deviation will be caused to the prediction model. Therefore, the method of constructing model parameters based on the maximum value and the minimum value has poor robustness, and the decoding prediction performance is reduced.

[0074] Therefore, in the current latest L0138 proposal, the concept of adjacent reference pixel subset is proposed, which improves the above deficiencies, reduces the number of adjacent reference pixel points, further reduces the operation complexity, and slightly improves the decoding prediction performance by selecting appropriate pixel points (as shown in FIG. 5B), but this solution still has room for improvement.

[0075] In order to better improve the performance of coding and decoding prediction, the embodiment of the present application provides a decoding prediction method, which determines the position of the reference pixel point to be selected according to a preset sampling number based on the midpoint of at least one side of the to-be-decoded block, so as to put the adjacent reference pixel points selected by taking into account the importance and dispersion into the adjacent reference pixel subset; and the to-be-decoded block is predicted and decoded based on the adjacent reference pixel subset. In this way, since the model parameters constructed based on the adjacent reference pixel subset are relatively accurate, the decoding prediction performance can be improved; moreover, the number of pixels in the adjacent reference pixel subset is small, thereby reducing the complexity of the search, and further reducing the bit rate. The embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0076] Referring to FIG. 6, it shows a constituent block diagram example of a video coding system provided by the embodiments of the present application; as shown in FIG. 6, the video coding system 600 comprises components such as transform and quantization 601, intra estimation 602, intra prediction 603, motion compensation 604, motion estimation 605, inverse transform and inverse quantization 606, filter control analysis 607, deblocking filtering and sample adaptive offset (SAO) filtering 608, header information coding and context-based adaptive binary arithmatic coding (CABAC) 609, and decoded picture buffer 610; for an input original video signal, a video coding block can be obtained through division of a coding tree unit (CTU), then the residual pixel information obtained after intra or inter prediction is transformed on the video coding block through transform and quantization 601, including transforming the residual information from a pixel domain to a transform domain, and quantizing the obtained transform coefficients, to further reduce the bit rate; intra estimation 602 and intra prediction 603 are used for intra prediction of the video coding block; in particular, intra estimation 602 and intra prediction 603 are used to determine an intra prediction mode to be used to encode the video coding block; motion compensation 604 and motion estimation 605 are used to perform inter prediction encoding of the received video coding block relative to one or more blocks in one or more reference frames to provide temporal prediction information; the motion estimation performed by motion estimation 605 is a process of generating a motion vector that can estimate the motion of the video coding block, then motion compensation is performed by motion compensation 604 based on the motion vector determined by motion estimation 605; after the intra prediction mode is determined, intra prediction 603 is also used to provide the selected intra prediction data to header information coding and CABAC 609, and motion estimation 605 also sends the calculated determined motion vector data to header information coding and CABAC 609; in addition, inverse transform and inverse quantization 606 are used for reconstruction of the video coding block, to reconstruct a residual block in the pixel domain, the reconstructed residual block is used to remove block effect artifacts through filter control analysis 607 and deblocking filtering and SAO filtering 608, then the reconstructed residual block is added to a predictive block in one of the frames of decoded picture buffer 610, to generate a reconstructed video coding block; header information coding and CABAC 609 are used to encode various coding parameters and quantized transform coefficients, in a CABAC-based encoding algorithm, the context content can be based on neighboring coding blocks, which can be used to encode information indicating the determined intra prediction mode, to output a bitstream of the video signal; and decoded picture buffer 610 is used to store the reconstructed video coding block, for predictive reference.As the video image encoding proceeds, new reconstructed video coding blocks are continuously generated, which are stored in the decoded image buffer 610.

[0077] Referring to FIG. 7, an example of a block diagram of a video decoding system is shown according to an embodiment of the present application. As shown in FIG. 7, the video decoding system 700 includes a header information decoding and CABAC decoding 701, an inverse transform and inverse quantization 702, an intra prediction 703, a motion compensation 704, a deblocking filter and SAO filter 705, and a decoded image buffer 706. After the input video signal is processed by the encoding system shown in FIG. 6, a bitstream of the video signal is output. The bitstream is input into the video decoding system 700, and first passes through the header information decoding and CABAC decoding 701 to obtain decoded transform coefficients. The transform coefficients are processed by the inverse transform and inverse quantization 702 to generate residual blocks in the pixel domain. The intra prediction 703 is used to generate prediction data of a current video decoding block based on a determined intra prediction mode and data from previously decoded blocks of the current frame or picture. The motion compensation 704 is used to determine prediction information for a video decoding block by parsing motion vectors and other associated syntax elements, and use the prediction information to generate a predictive block of the video decoding block being decoded. The decoded video block is formed by summing the residual block from the inverse transform and inverse quantization 702 and the corresponding predictive block generated by the intra prediction 703 or the motion compensation 704. The decoded video signal is processed by the deblocking filter and SAO filter 705 to remove blocking artifacts, which can improve the video quality. The decoded video block is then stored in the decoded image buffer 706, which stores reference pictures for subsequent intra prediction or motion compensation, and also outputs the video signal, i.e., the original video signal is recovered.

[0078] The embodiments of the present application are mainly applied to the intra prediction 603 shown in FIG. 6 and the intra prediction 703 shown in FIG. 7. That is, the embodiments of the present application can be applied to the encoding system or the decoding system, but the embodiments of the present application do not make a specific limitation in this regard.

[0079] Based on the above-mentioned application scenario examples of FIG. 6 or FIG. 7, referring to FIG. 8, a flowchart of a decoding prediction method is shown according to an embodiment of the present application. The method can include the following steps:

[0080] S801: Obtain reference pixel points adjacent to at least one side of a to-be-decoded block to obtain a first set of adjacent reference pixels;

[0081] S802: Determine a reference point from the at least one side, and determine the to-be-selected reference pixel point positions corresponding to the at least one side according to a preset sampling number;

[0082] S803: selecting, based on the to-be-selected reference pixel point position, a reference pixel point corresponding to the to-be-selected reference pixel point position from the first neighboring reference pixel set, and composing a neighboring reference pixel subset by using the selected reference pixel points;

[0083] S804: performing prediction decoding on the to-be-decoded block based on the neighboring reference pixel subset.

[0084] It should be noted that the to-be-decoded block is a decoding block that is currently to be subjected to second image component prediction or third image component prediction. The at least one side of the to-be-decoded block can refer to the upper side of the to-be-decoded block, or can refer to the left side of the to-be-decoded block, or can even refer to the upper side and the left side of the to-be-decoded block, and the embodiments of the present application are not limited in this regard.

[0085] It should be further noted that the reference point can be the midpoint of the at least one side, or can be the first reference pixel point position left of the midpoint of the at least one side, or can be the first reference pixel point position right of the midpoint of the at least one side, or can even be other reference pixel point positions of the at least one side, and the embodiments of the present application are not limited in this regard.

[0086] In the embodiments of the present application, the decoding prediction method of the embodiments of the present application can also be applied to an encoding system. By constructing a neighboring reference pixel subset in the encoding system, the encoding prediction performance of a video image can be improved, and the encoding compression efficiency is improved to save the encoding code rate. In the following, only the construction of a neighboring reference pixel subset in a decoding system will be described.

[0087] In the embodiments of the present application, first, reference pixel points adjacent to the at least one side of the to-be-decoded block are acquired to obtain a first neighboring reference pixel set; then, a reference point is determined from the at least one side, and a to-be-selected reference pixel point position corresponding to the at least one side is determined according to a preset sampling number; then, based on the to-be-selected reference pixel point position, a reference pixel point corresponding to the to-be-selected reference pixel point position is selected from the first neighboring reference pixel set, and a neighboring reference pixel subset is composed by using the selected reference pixel points; finally, the to-be-decoded block is subjected to prediction decoding based on the neighboring reference pixel subset. In the embodiments of the present application, all reference pixel points adjacent to the upper side or the left side of the to-be-decoded block are no longer involved in the search operation of decoding prediction, but on the premise of taking into account the importance and dispersion, a neighboring reference pixel subset is composed by selecting appropriate neighboring reference pixels, so that the number of pixels in the neighboring reference pixel subset is small, which not only reduces the complexity of the search, but also improves the decoding prediction performance, and thus reduces the bit rate.

[0088] In some embodiments, the determining the reference point from the at least one edge comprises:

[0089] determining a midpoint of the at least one edge based on a length of the at least one edge, and taking the midpoint of the at least one edge as the reference point.

[0090] In some embodiments, when the reference point is the midpoint of the at least one edge, the determining the reference point from the at least one edge comprises:

[0091] if the midpoint of the at least one edge is located at a middle position between two reference pixels, taking a first reference pixel position to the right of the middle position as the reference point of the at least one edge, or taking a first reference pixel position to the left of the middle position as the reference point of the at least one edge, based on the length of the at least one edge.

[0092] It should be noted that, considering that the importance of the reference pixels adjacent to the at least one edge of the to-be-decoded block is related to the positions thereof, in order to enable the reference pixels in the adjacent reference pixel subset to represent the characteristics of the entire adjacent edge, the reference pixel located at the center position of the edge should be selected as much as possible, so as to eliminate the reference pixels with lower importance, such as the reference pixels at the two edges of the edge. In the embodiments of the present application, if the upper edge of the to-be-decoded block is taken as an example for illustration, the first reference pixel position to the right or left of the middle position can be taken as the reference point of the edge.

[0093] Exemplarily, referring to FIG. 9, a structure diagram of selection of an adjacent reference pixel subset corresponding to an upper edge of a to-be-decoded block is shown according to an embodiment of the present application. As shown in FIG. 9, for all the reference pixels distributed on the upper edge of the to-be-decoded block, the midpoint of the edge is taken as the center (as shown by the dashed line in FIG. 9), and the selection of the reference pixels is performed based on the center as the reference point. Assuming that the length of the upper edge of the to-be-decoded block is 16, and the preset sampling number is 4, the sampling interval Δ is 16 / 4=4. Thus, since the length of the upper edge is 16, it can be determined that the midpoint is between 7 and 8, that is, the midpoint can be selected as 7 or 8. In FIG. 9, the selection of 8 as the reference point is taken as an example, and since the preset sampling number is 4, it can be determined that the to-be-selected reference pixel positions (as shown by the gray points in FIG. 9) are 2, 6, 10 and 14. According to the reference pixel positions, the reference pixels corresponding thereto can be selected to form the adjacent reference pixel subset.

[0094] In some embodiments, the determining the reference point from the at least one edge comprises:

[0095] If the midpoint of the at least one side is located at a middle position between two reference pixel points, a first reference pixel point position below the middle position is taken as the reference point of the at least one side, or a first reference pixel point position above the middle position is taken as the reference point of the at least one side, based on the length of the at least one side.

[0096] It should be noted that, considering that the importance of the reference pixel points adjacent to the at least one side of the to-be-decoded block is related to the corresponding positions, in order to enable the reference pixel points in the adjacent reference pixel subset to represent the characteristics of the entire adjacent side, the reference pixel point at the center position of the side should be selected as much as possible, so as to eliminate the reference pixel points with low importance, such as the reference pixel points at the two edges of the side. In the embodiment of the present application, if the left side of the to-be-decoded block is taken as an example for illustration, the first reference pixel point position below or above the middle position can be taken as the reference point of the side.

[0097] Exemplarily, referring to FIG. 10, a structure diagram of selection of an adjacent reference pixel subset corresponding to the left side of a to-be-decoded block is shown. As shown in FIG. 10, for all the reference pixel points distributed on the left side of the to-be-decoded block, the midpoint of the side is taken as the center (as shown by the dashed line position in FIG. 10), and the selection of the reference pixel points is performed based on the center as the reference point. Assuming that the length of the upper side of the to-be-decoded block is 8, and the preset sampling number is 2, the sampling interval Δ is 8 / 2=4. Thus, since the length of the left side is 8, it can be determined that the midpoint is between 3 and 4, that is, the midpoint can be selected as 3 or 4. In FIG. 10, the position 4 is taken as the reference point as an example, and since the preset sampling number is 2, the to-be-selected reference pixel point positions (as shown by the gray points in FIG. 10) are 2 and 6. According to the reference pixel point positions, the reference pixel points corresponding thereto can be selected to form the adjacent reference pixel subset.

[0098] In practical application, since the length of the left side or the upper side of the to-be-decoded block is an integer multiple of 2, the middle position of the left side or the upper side of the to-be-decoded block is between two points. In the example of FIG. 9, the first pixel point on the right of the middle position is taken as the midpoint of the side; however, the first pixel point on the left of the middle position can also be taken as the midpoint of the side, as shown in the structural example of FIG. 11. In FIG. 11, the first pixel point on the left of the middle position (for example, 3 in FIG. 11) is taken as the midpoint of the side, and since the preset sampling number is 2, the positions of the reference pixel points (for example, the gray points in FIG. 11) can be determined as 1 and 5, and the reference pixel points corresponding to the positions can also be selected to form the adjacent reference pixel subset. Therefore, in the embodiment of the present application, for the upper side of the to-be-decoded block, the first pixel point on the right of the middle position can be taken as the midpoint of the side, or the first pixel point on the left of the middle position can be taken as the midpoint of the side, which is not limited in the embodiment of the present application. In addition, for the left side of the to-be-decoded block, the first pixel point on the bottom of the middle position can be taken as the midpoint of the side, or the first pixel point on the top of the middle position can be taken as the midpoint of the side, which is also not limited in the embodiment of the present application.

[0099] Except for special description, the upper side of the to-be-decoded block will be taken as an example in the following description, but the prediction method of the embodiment of the present application is also applicable to the left side of the to-be-decoded block, or even other sides of the to-be-decoded block.

[0100] It can be understood that if the existence of the reference pixel points adjacent to the left side or the upper side of the to-be-decoded block is not considered, the adjacent reference pixel subset can also be constructed according to formula (4) and formula (5),

[0101] Δ=length / (N2 / 2) (4)

[0102] shift=Δ / 2 (5)

[0103] wherein, Δ represents a sampling interval, length represents a number of reference pixel points of a line adjacent to an upper side of the block to be decoded or a number of reference pixel points of a column adjacent to a left side of the block to be decoded, N2 represents a number of adjacent reference pixel points in the adjacent reference pixel subset expected by the block to be decoded (generally, the left side and the upper side are each half, but the embodiments of the present application do not make specific limitation), and shift represents a starting point position of the selected reference pixel point. Here, when the middle position of the left side or the upper side of the block to be decoded is between two points, if the first pixel point to the right of the middle position is taken as the middle point of the side, the starting point position shift = Δ / 2; if the first pixel point to the left of the middle position is taken as the middle point of the side, the starting point position shift = Δ / 2-1.

[0104] Exemplarily, taking the upper side shown in FIG. 9 as an example, length is equal to 16, N2 is equal to 8, and it is assumed that the left side and the upper side are each half, i.e., the preset sampling number of the upper side is 4, then according to the formula (4) and the formula (5), Δ = length / (N2 / 2) = 4 and shift = Δ / 2 = 2 are calculated respectively, i.e., taking 2 as the starting point position and 4 as the sampling interval, the position of the reference pixel point to be selected can be determined first, such as 2, 6, 10 and 14, and then the corresponding reference pixel points can be selected to form the adjacent reference pixel subset. Here, it should be noted that the preset sampling number corresponding to the left side and the preset sampling number corresponding to the upper side can be the same or different, and the embodiments of the present application do not make specific limitation.

[0105] In addition, the embodiments of the present application also provide a scheme for determining the preset sampling number, as shown in Table 2. In this way, the embodiments of the present application can also replace N2 with N2' in Table 2, and bring it into the above-mentioned formula (4) and formula (5) for calculation, so as to make the adjacent reference pixel subset more accurate. Thus, the decoding prediction performance is improved.

[0106] Referring to Table 2, which shows a relationship table between the size of the second image component block corresponding to the block to be decoded and the number of adjacent reference pixel points provided by the embodiments of the present application. In Table 2, N represents the length of the short side of the block to be decoded, N1 represents the number of adjacent reference pixel points provided in the traditional technical scheme, N2 represents the number of adjacent reference pixel points provided in the L0138 proposal, and N2' represents the number of adjacent reference pixel points provided in the embodiments of the present application. According to Table 2, it can be seen that when the length of the short side of the block to be decoded is less than or equal to 4, the adjacent reference pixel subset contains 4 reference pixel points; when the length of the short side of the block to be decoded is greater than 4, the adjacent reference pixel subset can contain 8 reference pixel points.

[0107] Table 2

[0108] Length of short side of size of second image component block N1N2N2'2 x n / n x 2 (n≥2) 2424 4 x n / n x 4 (n≥4) 4848 8 x n / n x 8 (n≥8) 8168 16 x n / n x 16 (n≥16) 1632 88 32 x 32 3264 88

[0109] In some embodiments, the determining a reference point from the at least one edge, and determining the to-be-selected reference pixel position corresponding to the at least one edge according to a preset sampling number, comprises:

[0110] Based on the preset sampling number and the length of the at least one edge, a first sampling interval is calculated;

[0111] Taking the midpoint of the at least one edge as the reference point, the to-be-selected reference pixel position corresponding to the at least one edge is determined according to the first sampling interval.

[0112] Optionally, the taking the midpoint of the at least one edge as the reference point, and determining the to-be-selected reference pixel position corresponding to the at least one edge according to the first sampling interval, comprises:

[0113] Based on the length of the at least one edge, a midpoint value of the at least one edge is calculated;

[0114] According to the midpoint value and the first sampling interval, a reference pixel position is calculated;

[0115] When the midpoint value is a non-integer, the reference pixel position on the left side of the midpoint value is rounded down, and the rounded reference pixel position is determined as the to-be-selected reference pixel position; the reference pixel position on the right side of the midpoint value is rounded up, and the rounded reference pixel position is determined as the to-be-selected reference pixel position.

[0116] Optionally, the taking the midpoint of the at least one edge as the reference point, and determining the to-be-selected reference pixel position corresponding to the at least one edge according to the first sampling interval, comprises:

[0117] Based on the length of the at least one edge, a midpoint value of the at least one edge is calculated;

[0118] According to the midpoint value and the first sampling interval, a reference pixel position is calculated;

[0119] When the midpoint value is a non-integer, the reference pixel position on the left side of the midpoint value is rounded up, and the rounded reference pixel position is determined as the to-be-selected reference pixel position; the reference pixel position on the right side of the midpoint value is rounded down, and the rounded reference pixel position is determined as the to-be-selected reference pixel position.

[0120] It should be noted that according to the preset sampling number and the length of one side of the to-be-decoded block, the first sampling interval corresponding to the side can be calculated. In addition, since the length of the left side or the upper side of the to-be-decoded block is an integer multiple of 2, the middle position of the left side or the upper side of the to-be-decoded block is between two points, at this time, the calculated midpoint value is a non-integer, and the calculated reference pixel point position is also a non-integer; however, if the length of the left side or the upper side of the to-be-decoded block is not an integer multiple of 2, the middle position of the left side or the upper side of the to-be-decoded block will not be between two points, at this time, the calculated midpoint value is an integer, and the calculated reference pixel point position is also an integer; that is, the calculated midpoint value can be an integer or a non-integer; correspondingly, the calculated reference pixel point position can also be an integer or a non-integer; the embodiments of the present application are not limited specifically.

[0121] In this way, when the calculated midpoint value is an integer, correspondingly, the calculated reference pixel point position is also an integer, at this time, the calculated reference pixel point position can be directly taken as the to-be-selected reference pixel point position; when the calculated midpoint value is a non-integer, correspondingly, the calculated reference pixel point position is also a non-integer, at this time, the to-be-selected reference pixel point position can be determined by rounding up or rounding down.

[0122] For example, taking the upper side shown in FIG. 11 as an example, assuming that the preset sampling number is 2 and the length of the upper side is 8, the first sampling interval can be determined to be 4. Since the length of the upper side is 8, that is, the middle position of the upper side is between two points. Since the reference pixel arrangement starts from 0 and ends at 7, the actual position of the midpoint of the upper side can be calculated to be 3.5; since the first sampling interval is 4, the reference point positions of 1.5 and 5.5 can be calculated by shifting 4 / 2 points to the left and right of the midpoint, respectively; at this time, the reference pixel point position on the left side of the midpoint can be rounded down to obtain one of the to-be-selected reference pixel point positions as 1, and the reference pixel point position on the right side of the midpoint can be rounded up to obtain another to-be-selected reference pixel point position as 6; this method can also be called an outward rounding scheme, as shown in FIG. 12; in this way, the reference pixel points corresponding to positions 1 and 6 can form an adjacent reference pixel subset. In addition, when the calculated reference point positions are 1.5 and 5.5, at this time, the reference pixel point position on the left side of the midpoint can be rounded up to obtain one of the to-be-selected reference pixel point positions as 2, and the reference pixel point position on the right side of the midpoint can be rounded down to obtain another to-be-selected reference pixel point position as 5; this method can also be called an inward rounding scheme, as shown in FIG. 13; in this way, the reference pixel points corresponding to positions 2 and 5 can form an adjacent reference pixel subset.

[0123] In some embodiments, after the first sampling interval is obtained, the method further comprises:

[0124] adjusting the first sampling interval to obtain a second sampling interval;

[0125] determining the to-be-selected reference pixel positions corresponding to the at least one edge according to the second sampling interval, with the midpoint of the at least one edge as a reference point.

[0126] In some embodiments, after the second sampling interval is obtained, the method further comprises:

[0127] determining the to-be-selected reference pixel positions corresponding to the at least one edge according to the first sampling interval, with the midpoint of the at least one edge as a reference point; and determining the to-be-selected reference pixel positions corresponding to the at least one edge according to the second sampling interval, with the midpoint of the at least one edge as the reference point.

[0128] It should be noted that, after the first sampling interval is obtained, the first sampling interval can be fine-tuned, such as being increased by 1 or decreased by 1, to obtain a second sampling interval. For example, if the first sampling interval is 4, the adjusted second sampling interval can be 3 or 5. In the embodiments of the present application, the adjustment of the first sampling interval can be a small adjustment (such as being increased by 1 or decreased by 1), but the specific adjustment range is not limited in the embodiments of the present application.

[0129] In addition, after the reference point of the at least one edge of the to-be-decoded block is determined, the first sampling interval or the second sampling interval can be used for uniform sampling, or the first sampling interval and the second sampling interval can be used for non-uniform sampling, and the to-be-selected reference pixel positions determined after sampling can be symmetrically distributed on both sides of the reference point or asymmetrically distributed on both sides of the reference point, which is not limited in the embodiments of the present application.

[0130] In some embodiments, the method further comprises:

[0131] determining the to-be-selected reference pixel positions corresponding to the at least one edge according to the preset sampling number, with the midpoint of the at least one edge as a reference point; and determining the to-be-selected reference pixel positions corresponding to the at least one edge according to the preset sampling number, with the midpoint of the at least one edge as the reference point.

[0132] It should be noted that, since the reference pixel points adjacent to the first image component reconstruction value of the to-be-decoded block are the reference pixel points in the middle position, the positions of the reference pixel points adjacent to the middle position and of a preset sampling number can be used as the positions of the to-be-selected reference pixel points, and this method can be referred to as a middle position continuous point selection scheme, as shown in FIG. 14. In this way, the reference pixel points at positions 2, 3, and 4 can form the adjacent reference pixel subset.

[0133] It can be understood that, assuming that the positions of the reference pixel points on a row / column adjacent to the upper side or the left side of the to-be-decoded block are numbered from 0, the number of the adjacent reference pixel points in the adjacent reference pixel subset formed in this embodiment and the corresponding positions of the to-be-selected reference pixel points are shown in Table 3.

[0134] Table 3

[0135]

[0136] In this embodiment, the positions of the reference pixel points adjacent to the middle position and of a preset sampling number are used as the positions of the to-be-selected reference pixel points to form the adjacent reference pixel subset. According to the adjacent reference pixel subset, the prediction decoding is performed, and on the basis of the L0138 proposal, the bit rate can still be saved, the decoding gain can be improved, and thus the decoding prediction performance is improved.

[0137] In some embodiments, the to-be-decoded block includes a square decoding block or a non-square decoding block.

[0138] Further, in some embodiments, when the to-be-decoded block is a non-square decoding block, the method further includes:

[0139] determining a long side of the to-be-decoded block and a third sampling interval corresponding to the long side based on the length of at least one side of the to-be-decoded block;

[0140] deleting the reference pixel points at the end positions of the long side, initially offsetting the long side from which the reference pixel points are deleted by a preset offset, using the positions of the reference pixel points after the offset as starting points, and sampling the long side from which the reference pixel points are deleted according to the third sampling interval to determine the positions of the to-be-selected reference pixel points corresponding to the long side.

[0141] It should be noted that the embodiments of the present application can be applied to square decoding blocks or non-square decoding blocks, and the embodiments of the present application are not limited in this regard.

[0142] It should be noted that when the to-be-decoded block is a non-square decoding block, there is necessarily one long side and one short side in the left side or the upper side of the to-be-decoded block; according to the ratio between the long side and the short side, a third sampling interval corresponding to the long side can be obtained. Before sampling the long side, the reference pixel points corresponding to the end positions of the long side can be deleted first, and then the long side with the deleted reference pixel points is initially offset by a preset offset, and the positions of the reference pixel points after the offset are taken as starting points, and then the long side with the deleted reference pixel points is sampled to determine the positions of the to-be-selected reference pixel points corresponding to the long side. In the embodiments of the present application, the preset offset can be one half of the third sampling interval, or can be other values, which are not limited in the embodiments of the present application.

[0143] Exemplarily, in the related technical solutions, the sampling of the reference pixel points for the long side of the non-square decoding block is shown in FIG. 4B. In FIG. 4B, according to the ratio between the long side and the short side, it can be determined that the third sampling interval is 4; the long side is taken as the starting point from the first reference pixel point position on the left side, and then the same number of reference pixel points as the adjacent reference pixel points of the short side are selected according to the third sampling interval. Since this will cause the sampled reference pixel points to be left-biased and not to cover the entire long side, in the embodiments, the long side of the non-square decoding block is first initially offset so that the sampled reference pixel points can cover the entire long side. For example, the preset offset is one half of the third sampling interval, i.e., the preset offset is 2, that is, the sampling in the embodiments is taken as the starting point of 2, so that the adjacent reference pixel subset formed in this way can better cover the entire long side.

[0144] In some embodiments, the determining a reference point from the at least one side, and determining the to-be-selected reference pixel point positions corresponding to the at least one side according to a preset sampling number, comprises:

[0145] deleting the reference pixel points corresponding to the starting positions and the end positions of the at least one side to obtain a second adjacent reference pixel set;

[0146] taking the midpoint of the at least one side as a reference point, and determining the to-be-selected reference pixel point positions according to a preset sampling number based on the second adjacent reference pixel set.

[0147] It should be noted that the embodiments of the present application can directly select reference pixel points based on the midpoint of the left side or the upper side of the to-be-decoded block as a reference point to form the adjacent reference pixel subset; the embodiments of the present application can also first delete reference pixel points corresponding to the starting position and the end position of the left side / upper side of the to-be-decoded block (for example, for the upper side, the starting position is the left edge position, and the end position is the right edge position; for the left side, the starting position is the upper edge position, and the end position is the lower edge position), and then retain reference pixel points in the middle part, and then select the reference pixel points to form the adjacent reference pixel subset.

[0148] It should be further noted that the preset sampling number can be arbitrarily valued; generally, the preset sampling number is less than N1 shown in Table 2; however, the embodiments of the present application are not limited to the preset sampling number being equal to N2 or N2'. Since the preset sampling number is less than N1 shown in Table 2, the search complexity of the reference pixel points can be reduced to improve the decoding prediction performance. Specifically, by changing the values of Δ and shift in the above formula (4) and formula (5), different starting points and sampling intervals can be used to determine the positions of the to-be-selected reference pixel points, and the corresponding reference pixel points can be selected to form the adjacent reference pixel subset.

[0149] In some embodiments, the prediction decoding of the to-be-decoded block based on the adjacent reference pixel subset comprises:

[0150] determining model parameters based on the adjacent reference pixel subset;

[0151] establishing a prediction model according to the model parameters; wherein the prediction model is used to represent the prediction relationship between the first image component and the second image component corresponding to each pixel point in the to-be-decoded block;

[0152] performing prediction decoding on the to-be-decoded block based on the prediction model.

[0153] It should be noted that after obtaining the adjacent reference pixel subset, the model parameters α and β can be constructed; in this way, the prediction model can be established according to the above formula (1); and the to-be-decoded block can be prediction decoded according to the prediction model. Since the adjacent reference pixel points included in the adjacent reference pixel subset take into account the importance and dispersion, the constructed model parameters are more accurate, thereby better improving the decoding prediction performance and reducing the bit rate.

[0154] The above embodiment provides a decoding prediction method, comprising: obtaining reference pixels adjacent to at least one side of a to-be-decoded block to obtain a first adjacent reference pixel set; determining a reference point from the at least one side, determining a to-be-selected reference pixel position corresponding to the at least one side according to a preset sampling number; selecting reference pixels corresponding to the to-be-selected reference pixel position from the first adjacent reference pixel set based on the to-be-selected reference pixel position, and forming an adjacent reference pixel subset by using the selected reference pixels; and performing prediction decoding on the to-be-decoded block based on the adjacent reference pixel subset. Since the adjacent reference pixels in the adjacent reference pixel subset not only consider importance and dispersion, but also have a small number of pixel points, the complexity of search is reduced, the prediction performance of video image decoding is improved, and the bit rate is reduced.

[0155] Based on the same inventive concept as the technical solution shown in FIG. 8, referring to FIG. 15, a component structure diagram of a decoding prediction device 150 provided by an embodiment of the present application is shown. The decoding prediction device 150 can include an obtaining unit 1501, a determining unit 1502, a selecting unit 1503, and a decoding unit 1504, wherein,

[0156] The obtaining unit 1501 is configured to obtain reference pixels adjacent to at least one side of a to-be-decoded block to obtain a first adjacent reference pixel set.

[0157] The determining unit 1502 is configured to determine a reference point from the at least one side, and determine a to-be-selected reference pixel position corresponding to the at least one side according to a preset sampling number.

[0158] The selecting unit 1503 is configured to select reference pixels corresponding to the to-be-selected reference pixel position from the first adjacent reference pixel set based on the to-be-selected reference pixel position, and form an adjacent reference pixel subset by using the selected reference pixels.

[0159] The decoding unit 1504 is configured to perform prediction decoding on the to-be-decoded block based on the adjacent reference pixel subset.

[0160] In the above scheme, the determining unit 1502 is specifically configured to determine the reference point based on the length of the at least one side, and if a midpoint of the at least one side is located at a middle position between two reference pixels, take the first reference pixel position to the right of the middle position as the reference point of the at least one side, or take the first reference pixel position to the left of the middle position as the reference point of the at least one side.

[0161] In the above scheme, the determining unit 1502 is specifically configured to, based on the length of the at least one side, if a midpoint of the at least one side is located at a middle position between two reference pixel points, take a first reference pixel point position below the middle position as a reference point of the at least one side, or take a first reference pixel point position above the middle position as the reference point of the at least one side.

[0162] In the above scheme, referring to FIG. 15, the decoding prediction apparatus 150 further includes a calculating unit 1505 configured to calculate a first sampling interval based on the preset sampling number and the length of the at least one side.

[0163] The determining unit 1502 is specifically configured to take the midpoint of the at least one side as a reference point, and determine the to-be-selected reference pixel point position corresponding to the at least one side according to the first sampling interval.

[0164] In the above scheme, the calculating unit 1505 is further configured to calculate a midpoint value of the at least one side based on the length of the at least one side, and calculate a reference pixel point position according to the midpoint value and the first sampling interval.

[0165] The determining unit 1502 is specifically configured to, when the midpoint value is a non-integer, take a reference pixel point position left of the midpoint value to a small integer, determine the integer reference pixel point position as the to-be-selected reference pixel point position, and take a reference pixel point position right of the midpoint value to a large integer, determine the integer reference pixel point position as the to-be-selected reference pixel point position.

[0166] In the above scheme, the calculating unit 1505 is further configured to calculate a midpoint value of the at least one side based on the length of the at least one side, and calculate a reference pixel point position according to the midpoint value and the first sampling interval.

[0167] The determining unit 1502 is specifically configured to, when the midpoint value is a non-integer, take a reference pixel point position left of the midpoint value to a large integer, determine the integer reference pixel point position as the to-be-selected reference pixel point position, and take a reference pixel point position right of the midpoint value to a small integer, determine the integer reference pixel point position as the to-be-selected reference pixel point position.

[0168] In the above scheme, referring to FIG. 15, the decoding prediction apparatus 150 further includes an adjusting unit 1506 configured to adjust the first sampling interval to obtain a second sampling interval.

[0169] The determining unit 1502 is further configured to determine the to-be-selected reference pixel position corresponding to the at least one side according to the second sampling interval with the midpoint of the at least one side as a reference point.

[0170] In the foregoing solution, the determining unit 1502 is further configured to determine the to-be-selected reference pixel position corresponding to the left side of the reference point according to the first sampling interval and the to-be-selected reference pixel position corresponding to the right side of the reference point according to the second sampling interval with the midpoint of the at least one side as the reference point.

[0171] In the foregoing solution, the determining unit 1502 is further configured to determine the continuous reference pixel positions near the reference point as the to-be-selected reference pixel positions according to the preset sampling number with the midpoint of the at least one side as the reference point, and the reference point is located at the middle position of the to-be-selected reference pixel positions.

[0172] In the foregoing solution, the to-be-decoded block includes a square decoding block or a non-square decoding block.

[0173] In the foregoing solution, the obtaining unit 1501 is further configured to delete the reference pixels corresponding to the start position and the end position of the at least one side respectively to obtain a second adjacent reference pixel set.

[0174] The determining unit 1502 is further configured to determine the to-be-selected reference pixel position according to a preset sampling number based on the second adjacent reference pixel set with the midpoint of the at least one side as the reference point.

[0175] In the foregoing solution, referring to FIG. 15, the decoding prediction device 150 further includes an establishing unit 1507 configured to determine model parameters based on the adjacent reference pixel subset, and establish a prediction model according to the model parameters, where the prediction model is used to represent the prediction relationship between the first image component and the second image component corresponding to each pixel point in the to-be-decoded block.

[0176] The decoding unit 1504 is specifically configured to perform prediction decoding on the to-be-decoded block based on the prediction model.

[0177] It can be understood that, in the embodiment, the “unit” can be a part of circuit, a part of processor, a part of program or software, and the like, and of course can be a module, and can also be non-modular. Moreover, the constituent parts in the embodiment can be integrated in a processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function module.

[0178] The integrated unit, if implemented in the form of a software function module and not sold or used as an independent product, can be stored in a computer readable storage medium based on such understanding. The technical solutions of the embodiments essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiments. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0179] Therefore, the embodiments provide a computer storage medium storing a decoding prediction program, which, when executed by at least one processor, implements the steps of the method described in the technical solution shown in FIG. 8.

[0180] Based on the composition of the decoding prediction device 150 and the computer storage medium, referring to FIG. 16, a specific hardware structure example of the decoding prediction device 150 provided by the embodiments of the present application is shown, which can include a network interface 1601, a memory 1602, and a processor 1603; and various components are coupled together through a bus system 1604. It can be understood that the bus system 1604 is used to realize the connection and communication between the components. In addition to the data bus, the bus system 1604 also includes a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, various buses are marked as the bus system 1604 in FIG. 16. The network interface 1601 is used for receiving and sending signals in the information transmission process between other external network elements;

[0181] The memory 1602 is used to store computer programs capable of running on the processor 1603;

[0182] The processor 1603 is used to execute the following when running the computer programs:

[0183] Obtain reference pixel points adjacent to at least one side of the to-be-decoded block to obtain a first adjacent reference pixel set;

[0184] Determine a reference point from the at least one side, and determine the to-be-selected reference pixel point position corresponding to the at least one side according to a preset sampling number;

[0185] Based on the to-be-selected reference pixel position, a reference pixel corresponding to the to-be-selected reference pixel position is selected from the first set of adjacent reference pixels, and the selected reference pixel forms a subset of adjacent reference pixels;

[0186] Based on the subset of adjacent reference pixels, the to-be-decoded block is predicted and decoded.

[0187] It can be understood that the memory 1602 in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) or a flash memory. The volatile memory can be a random access memory (Random Access Memory, RAM) used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synchlink DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM). The memory 1602 of the system and method described herein is intended to include, but not limited to, these and any other suitable types of memory.

[0188] The processor 1603 can be an integrated circuit chip logic circuit, or an instruction in software form. The processor 1603 can be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. The methods and steps disclosed in the embodiments of the present application can be implemented or executed by the processor 1603. The general purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the methods disclosed in the embodiments of the present application can be directly embodied to be executed by a hardware decoding processor or be executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in the random access memory, the flash memory, the read only memory, the programmable read only memory, the electrically erasable programmable memory, the register, or other mature storage medium in the art. The storage medium is located in the storage memory 1602, and the processor 1603 reads information in the storage memory 1602 and combines the hardware to complete the steps of the above method.

[0189] It can be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For a hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP Devices), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.

[0190] For a software implementation, the techniques described herein can be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes can be stored in memory and executed by processors. The memory can be implemented within the processors or external to the processors.

[0191] Optionally, as another embodiment, the processor 1603 is further configured to execute the steps of the method in the technical solutions shown in Figure 8 when running the computer program.

[0192] It should be noted that the technical solutions disclosed in the embodiments of the present application can be combined arbitrarily without conflict.

[0193] The above describes only specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims. Industrial applicability

[0194] In the embodiments of the present application, first, reference pixel points adjacent to at least one side of a to-be-decoded block are acquired to obtain a first adjacent reference pixel set; then a reference point is determined from the at least one side, and a to-be-selected reference pixel point position corresponding to the at least one side is determined according to a preset sampling number; then, based on the to-be-selected reference pixel point position, reference pixel points corresponding to the to-be-selected reference pixel point position are selected from the first adjacent reference pixel set to form an adjacent reference pixel subset; finally, based on the adjacent reference pixel subset, the to-be-decoded block is predicted and decoded; since the selection of the adjacent reference pixel points in the adjacent reference pixel subset takes into account the importance and dispersion, the model parameters constructed based on the adjacent reference pixel subset are relatively accurate, which can improve the prediction performance of video image decoding; moreover, the number of pixels in the adjacent reference pixel subset is small, thereby reducing the complexity of the search and improving the prediction performance of video image decoding, and thus reducing the bit rate.

Claims

1. A decoding prediction method, wherein, The method includes: Obtain reference pixels that are adjacent to at least one edge of the block to be decoded, and obtain the first set of adjacent reference pixels; A reference point is determined from the at least one edge, and the position of the reference pixel point to be selected corresponding to the at least one edge is determined according to a preset number of samples; Based on the position of the reference pixel to be selected, a reference pixel corresponding to the position of the reference pixel to be selected is selected from the first set of adjacent reference pixels, and the selected reference pixels are used to form a subset of adjacent reference pixels. Based on the adjacent reference pixel subset, predictive decoding is performed on the block to be decoded.

2. The method according to claim 1, wherein, When the reference point is the midpoint of at least one edge, determining a reference point from the at least one edge includes: Based on the length of the at least one side, if the midpoint of the at least one side is located in the middle position between two reference pixels, then the position of the first reference pixel slightly to the right of the middle position is taken as the reference point of the at least one side, or the position of the first reference pixel slightly to the left of the middle position is taken as the reference point of the at least one side.

3. The method according to claim 1, wherein, When the reference point is the midpoint of at least one edge, determining a reference point from the at least one edge includes: Based on the length of the at least one side, if the midpoint of the at least one side is located in the middle position between two reference pixels, then the position of the first reference pixel slightly below the middle position is taken as the reference point of the at least one side, or the position of the first reference pixel slightly above the middle position is taken as the reference point of the at least one side.

4. The method according to claim 1, wherein, The step of determining a reference point from the at least one edge and determining the position of the candidate reference pixel corresponding to the at least one edge according to a preset number of samples includes: The first sampling interval is calculated based on the preset number of samples and the length of the at least one edge; Using the midpoint of the at least one edge as a reference point, the position of the selected reference pixel point corresponding to the at least one edge is determined according to the first sampling interval.

5. The method according to claim 4, wherein, The step of determining the position of the candidate reference pixel corresponding to the at least one edge according to the first sampling interval, using the midpoint of the at least one edge as the reference point, includes: Based on the length of the at least one edge, calculate the midpoint value of the at least one edge; The position of the reference pixel is calculated based on the midpoint value and the first sampling interval. When the midpoint value is not an integer, the reference pixel position to the left of the midpoint value is rounded down to the nearest integer, and the rounded reference pixel position is determined as the reference pixel position to be selected; the reference pixel position to the right of the midpoint value is rounded up to the nearest integer, and the rounded reference pixel position is determined as the reference pixel position to be selected.

6. The method according to claim 4, wherein, The step of determining the position of the candidate reference pixel corresponding to the at least one edge according to the first sampling interval, using the midpoint of the at least one edge as the reference point, includes: Based on the length of the at least one edge, calculate the midpoint value of the at least one edge; The position of the reference pixel is calculated based on the midpoint value and the first sampling interval. When the midpoint value is not an integer, the reference pixel position to the left of the midpoint value is rounded up, and the rounded reference pixel position is determined as the candidate reference pixel position; the reference pixel position to the right of the midpoint value is rounded down, and the rounded reference pixel position is determined as the candidate reference pixel position.

7. The method according to claim 4, wherein, After the first sampling interval is calculated, the method further includes: The first sampling interval is adjusted to obtain the second sampling interval; Using the midpoint of the at least one edge as a reference point, the position of the selected reference pixel point corresponding to the at least one edge is determined according to the second sampling interval.

8. The method according to claim 7, wherein, After obtaining the second sampling interval, the method further includes: Using the midpoint of at least one edge as a reference point, the position of the candidate reference pixel to the left of the reference point is determined according to the first sampling interval, and the position of the candidate reference pixel to the right of the reference point is determined according to the second sampling interval.

9. The method according to claim 1, wherein, The step of determining a reference point from the at least one edge and determining the position of the candidate reference pixel corresponding to the at least one edge according to a preset number of samples includes: Using the midpoint of at least one edge as a reference point, the positions of consecutive reference pixels near the reference point are determined as the positions of reference pixels to be selected according to the preset number of samples, and the reference point is located at the middle position of the positions of the reference pixels to be selected.

10. The method according to any one of claims 1 to 9, wherein, The block to be decoded includes a square decoding block or a non-square decoding block.

11. The method according to claim 1, wherein, The step of determining a reference point from the at least one edge and determining the position of the candidate reference pixel corresponding to the at least one edge according to a preset number of samples includes: Delete the reference pixels corresponding to the start and end positions of at least one edge to obtain a second set of adjacent reference pixels; Using the midpoint of at least one edge as a reference point, the position of the reference pixel to be selected is determined based on the second set of adjacent reference pixels according to a preset number of samples.

12. The method according to any one of claims 1 to 11, wherein, The step of predictive decoding of the block to be decoded based on the adjacent reference pixel subset includes: Based on the adjacent reference pixel subset, determine the model parameters; A prediction model is established based on the model parameters; wherein the prediction model is used to characterize the prediction relationship between the first image component and the second image component corresponding to each pixel in the block to be decoded; Based on the prediction model, predictive decoding is performed on the block to be decoded.

13. A decoding prediction apparatus, wherein, The decoding prediction device includes an acquisition unit, a determination unit, a selection unit, and a decoding unit. The acquisition unit is configured to acquire reference pixel points adjacent to at least one edge of the block to be decoded, thereby obtaining a first set of adjacent reference pixels; The determining unit is configured to determine a reference point from the at least one edge, and determine the position of the candidate reference pixel point corresponding to the at least one edge according to a preset number of samples. The selection unit is configured to select a reference pixel corresponding to the position of the reference pixel to be selected from the first set of adjacent reference pixels based on the position of the reference pixel to be selected, and to form an adjacent reference pixel subset by selecting the reference pixels. The decoding unit is configured to perform predictive decoding on the block to be decoded based on the adjacent reference pixel subset.

14. The decoding prediction apparatus according to claim 13, wherein, The determining unit is specifically configured to, based on the length of the at least one side, if the midpoint of the at least one side is located in the middle position between two reference pixels, take the position of the first reference pixel slightly to the right of the middle position as the reference point of the at least one side, or take the position of the first reference pixel slightly to the left of the middle position as the reference point of the at least one side.

15. The decoding prediction apparatus according to claim 13, wherein, The determining unit is specifically configured to, based on the length of the at least one side, if the midpoint of the at least one side is located in the middle position between two reference pixels, take the position of the first reference pixel slightly below the middle position as the reference point of the at least one side, or take the position of the first reference pixel slightly above the middle position as the reference point of the at least one side.

16. The decoding prediction apparatus according to claim 13, wherein, The decoding prediction device further includes a calculation unit configured to calculate a first sampling interval based on the preset number of samples and the length of the at least one edge; The determining unit is specifically configured to determine the position of the selected reference pixel point corresponding to the at least one edge, based on the midpoint of the at least one edge as the reference point and according to the first sampling interval.

17. The decoding prediction apparatus according to claim 16, wherein, The calculation unit is further configured to calculate the midpoint value of the at least one edge based on the length of the at least one edge; and to calculate the reference pixel position based on the midpoint value and the first sampling interval; the determining unit is specifically configured to, when the midpoint value is not an integer, round down the reference pixel position to the left of the midpoint value and determine the rounded reference pixel position as the candidate reference pixel position; and round up the reference pixel position to the right of the midpoint value and determine the rounded reference pixel position as the candidate reference pixel position.

18. The decoding prediction apparatus according to claim 16, wherein, The calculation unit is further configured to calculate the midpoint value of the at least one edge based on the length of the at least one edge; and to calculate the reference pixel position based on the midpoint value and the first sampling interval; the determining unit is specifically configured to, when the midpoint value is not an integer, round up the reference pixel position to the left of the midpoint value and determine the rounded reference pixel position as the candidate reference pixel position; and round down the reference pixel position to the right of the midpoint value and determine the rounded reference pixel position as the candidate reference pixel position.

19. The decoding prediction apparatus according to claim 16, wherein, The decoding prediction device further includes an adjustment unit configured to adjust the first sampling interval to obtain a second sampling interval; The determining unit is further configured to determine the position of the selected reference pixel point corresponding to the at least one edge, using the midpoint of the at least one edge as a reference point and according to the second sampling interval.

20. The decoding prediction apparatus according to claim 19, wherein, The determining unit is further configured to use the midpoint of the at least one edge as a reference point, determine the position of the candidate reference pixel corresponding to the left of the reference point according to the first sampling interval, and determine the position of the candidate reference pixel corresponding to the right of the reference point according to the second sampling interval.

21. The decoding prediction apparatus according to claim 13, wherein, The determining unit is further configured to use the midpoint of the at least one edge as a reference point, and determine the positions of consecutive reference pixels near the reference point as the positions of reference pixels to be selected according to the preset number of samples, wherein the reference point is located at the middle position of the positions of the reference pixels to be selected.

22. The decoding prediction apparatus according to any one of claims 13 to 21, wherein, The block to be decoded includes a square decoding block or a non-square decoding block.

23. The decoding prediction apparatus according to claim 13, wherein, The acquisition unit is further configured to delete the reference pixel points corresponding to the start position and end position of the at least one edge, respectively, to obtain a second set of adjacent reference pixels; The determining unit is further configured to determine the position of the reference pixel to be selected based on the midpoint of the at least one edge as a reference point and the second set of adjacent reference pixels, according to a preset number of samples.

24. The decoding prediction apparatus according to any one of claims 13 to 23, wherein, The decoding prediction device further includes an establishment unit configured to determine model parameters based on the adjacent reference pixel subset; and to establish a prediction model based on the model parameters; wherein the prediction model is used to characterize the prediction relationship between the first image component and the second image component corresponding to each pixel in the block to be decoded; The decoding unit is specifically configured to perform predictive decoding on the block to be decoded based on the prediction model.

25. A decoding prediction apparatus, wherein, The decoding prediction device includes: a memory and a processor; The memory is used to store computer programs that can run on the processor; The processor is configured to perform the steps of the method as described in any one of claims 1 to 12 when running the computer program.

26. A computer storage medium, wherein, The computer storage medium stores a decoding prediction program, which, when executed by at least one processor, implements the steps of the method as described in any one of claims 1 to 12.

Images