Video decoding method, video encoding method, device, equipment and storage medium

Abstract

The present application provides a video decoding method, a video encoding method, a device, an apparatus, and a storage medium, which relate to the technical field of video encoding and decoding processing. The decoding method includes the steps of: obtaining a first parameter set corresponding to a video frame to be decoded; determining an effective QM based on syntax elements included in the first parameter set, where the effective QM refers to a QM actually used when performing inverse quantization on transform coefficients quantized in the decoding process of the video frame to be decoded; and performing decoding using the effective QM. By adopting the technical means of the present application, the decoder side only needs to perform decoding using the effective QM, thereby reducing the computational complexity on the decoder side.

Description

[Technical Field]

[0001] The embodiments of the present application relate to the technical field of video encoding and decoding, and in particular to a video decoding method, a video encoding method, an apparatus, a device, and a storage medium.

[0002] This application claims priority from a Chinese patent application filed on December 18, 2019, bearing application number 201911309768.6, and entitled "Video decoding method, device, equipment, and storage medium," the entire contents of which are incorporated herein by reference. [Background technology]

[0003] H.266 is a next-generation video coding technology that has been improved based on H.265 / HEVC (High Efficient Video Coding). It has already been officially named VVC (Versatile Video Coding) and is being continuously updated and improved under the guidance of the JVET (Joint Video Experts Team) organization.

[0004] At the 14th JVET meeting, it was decided that frequency-related scaling can be supported in VVC by using two types of quantization matrices: a default quantization matrix and a user-defined quantization matrix. When a quantization matrix is ​​enabled, independent quantization can be performed on the transform coefficients in a TB (Transform Block) based on the quantization coefficients (i.e., integer weighting values) contained in the quantization matrix.

[0005] Currently, the quantization matrix decoding method employed in VVC has a relatively high degree of computational complexity on the decoder side. Summary of the Invention [Problem to be solved by the invention]

[0006] The embodiments of the present application provide a video decoding method, a video encoding method, a device, an apparatus and a storage medium, which can reduce the computational complexity at the decoder side. [Means for solving the problem]

[0007] In one aspect, embodiments of the present application provide a video decoding method, the method comprising: obtaining a first parameter set corresponding to a video frame to be decoded, the first parameter set including a parameter set used to define a syntax element related to a quantization matrix (QM); determining an effective QM based on syntax elements included in the first parameter set, where the effective QM refers to a QM actually used when performing inverse quantization on quantized transform coefficients in a decoding process of the video frame to be decoded; and performing decoding on the valid QM.

[0008] In another aspect, embodiments of the present application provide a video encoding method, the method comprising: determining an effective QM corresponding to a video frame to be coded, where the effective QM refers to a QM actually used when quantizing transform coefficients in a coding process of the video frame to be coded; and performing encoding on the syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set, the first parameter set including a parameter set used to define syntax elements associated with the QM.

[0009] In yet another aspect, an embodiment of the present application provides a video decoding device, the device including: a parameter obtaining module; a QM determining module; and a QM decoding module; The parameter acquisition module is used to acquire a first parameter set corresponding to a video frame to be decoded, the first parameter set including a parameter set used to define a syntax element associated with QM; the QM determination module is used to determine an effective QM based on syntax elements included in the first parameter set, and the effective QM refers to a QM actually used when performing inverse quantization on quantized transform coefficients in a decoding process of the video frame to be decoded; The QM decoding module is used to perform decoding on the valid QM.

[0010] In yet another aspect, an embodiment of the present application provides a video encoding device, the device including: a QM determination module; and a QM encoding module; The QM determination module is used to determine an effective QM corresponding to a video frame to be coded, where the effective QM refers to a QM actually used when quantizing transform coefficients in a coding process of the video frame to be coded; The QM encoding module is used to perform encoding on syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set, where the first parameter set includes a parameter set used to define syntax elements related to the QM.

[0011] In yet another aspect, an embodiment of the present application provides a computer device, the computer device including a processor and a memory, wherein at least one instruction, at least one program, code set or instruction set is stored in the memory, the at least one instruction, the at least one program, code set or instruction set being loaded and executed by the processor to implement the video decoding method or to implement the video encoding method.

[0012] In yet another aspect, an embodiment of the present application provides a computer-readable storage medium having stored thereon at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, code set or instruction set being loaded and executed by a processor to implement the video decoding method or to implement the video encoding method.

[0013] In yet another aspect, an embodiment of the present application provides a computer program product, which, when executed by a processor, implements the above video decoding method or is used to implement the above video encoding method. [Effects of the Invention]

[0014] The technical means provided by the embodiments of the present application may include the following beneficial effects.

[0015] By obtaining a first parameter set corresponding to the video frame to be decoded, an effective QM is determined based on the syntax elements included in the first parameter set, where the effective QM refers to the QM actually used when quantizing the transform coefficients in the process of encoding the video frame to be decoded, and then decoding is performed using the effective QM. In this way, the decoder only needs to perform decoding using the effective QM, thereby reducing the computational complexity on the decoder side. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a schematic diagram of video encoding as exemplarily shown in the present application; [Figure 2] 1 is a simplified block diagram of a communication system provided by one embodiment of the present application; [Figure 3] 1 is a schematic diagram of an arrangement of a video encoder and a video decoder in a streaming environment, as an example shown in the present application; [Figure 4]FIG. 1 is a schematic diagram of an encoding process under an inter-frame prediction mode provided by an embodiment of the present application; [Figure 5] FIG. 1 is a schematic diagram of an encoding process under an intra-frame prediction mode provided by an embodiment of the present application. [Figure 6] 1 is a schematic diagram of functional modules of a video encoder provided by one embodiment of the present application; [Figure 7] FIG. 2 is a schematic diagram of functional modules of a video decoder provided by one embodiment of the present application. [Figure 8] FIG. 1 is a schematic diagram of generating a QM by downsampling copying provided by one embodiment of the present application. [Figure 9] FIG. 2 is a schematic diagram of a diagonal scan order provided by one embodiment of the present application. [Figure 10] 1 is a flowchart of a video decoding method provided by one embodiment of the present application. [Figure 11] 1 is a flowchart of a video encoding method provided by one embodiment of the present application. [Figure 12] 1 is a block diagram of a video decoding device provided by one embodiment of the present application; [Figure 13] FIG. 2 is a block diagram of a video decoding device provided by another embodiment of the present application; [Figure 14] 1 is a block diagram of a video encoding device provided by one embodiment of the present application; [Figure 15] FIG. 1 is a structural block diagram of a computer device provided by one embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0017] In order to make the objectives, technical means and advantages of the present application clearer, the embodiments of the present application will be described in more detail below with reference to the accompanying drawings.

[0018] 1, the current block 101 contains samples already found by the encoder during the motion estimation process, and these samples can be predicted based on a previous block of the same size with a spatial offset. Alternatively, the motion vectors (MVs) can be derived from metadata associated with one or more reference pictures, rather than directly encoding the MVs. For example, the MVs associated with any of five surrounding samples A0, A1, B0, B1, and B2 (corresponding to 102 to 106, respectively) can be used, and the MVs can be derived from the metadata of the nearest reference picture (according to the decoding order).

[0019] FIG. 2 illustrates a simplified block diagram of a communication system according to one embodiment of the present application. The communication system 200 includes multiple devices, which can communicate with each other, for example, via a network 250. For example, the communication system 200 includes a first device 210 and a second device 220 connected to each other via the network 250. In the embodiment of FIG. 2, the first device 210 and the second device 220 perform unidirectional data transmission. For example, the first device 210 can encode video data, such as a video picture stream collected by the first device 210, and transmit the encoded video data to the second device 220 via the network 250. The encoded video data is transmitted in the form of one or more encoded video code streams. The second device 220 can receive the encoded video data from the network 250, decode the encoded video data to recover the video data, and display video pictures based on the recovered video data. Unidirectional data transmission is common in applications such as media services.

[0020] In another embodiment, the communication system 200 includes a third device 230 and a fourth device 240 that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For the bidirectional data transmission, each of the third device 230 and the fourth device 240 may perform encoding on video data (e.g., a video picture stream collected by the device) and transmit it to the other of the third device 230 and the fourth device 240 via the network 250. Each of the third device 230 and the fourth device 240 may further receive the encoded video data transmitted by the other of the third device 230 and the fourth device 240, perform decoding on the encoded video data to recover the video data, and display video pictures on an accessible display device based on the recovered video data.

[0021] In the embodiment of FIG. 2 , the first device 210, the second device 220, the third device 230, and the fourth device 240 may be computing devices such as servers, personal computers, and smartphones, although the principles disclosed herein are not limited thereto. The embodiments of the present application apply to personal computers (PCs), mobile phones, tablet computers, media players, and / or dedicated videoconferencing equipment. The network 250 represents any number of networks transmitting encoded video data between the first device 210, the second device 220, the third device 230, and the fourth device 240, including, for example, wired and / or wireless communication networks. The communication network 250 can exchange data over circuit-switched and / or packet-switched channels. The network may include an electronic communication network, a local area network, a wide area network, and / or the Internet. For purposes of the present application, unless otherwise interpreted below, the architecture and topology of the network 250 may be irrelevant to the operation disclosed herein.

[0022] As an example, Figure 3 illustrates a configuration of a video encoder and a video decoder in a streaming environment. The subject matter disclosed herein is equally applicable to other applications that support video, including video conferencing, digital TV, and storing compressed video on digital media, including compact discs (CDs), digital versatile discs (DVDs), memory sticks, etc.

[0023] The streaming system may include an acquisition subsystem 313, which may include a video source 301, such as a digital camera, that creates an uncompressed video picture stream 302. In an embodiment, the video picture stream 302 includes samples captured by the digital camera. Compared to the encoded video data 304 (or encoded video code stream), the video picture stream 302 is depicted as a thick line to emphasize the high data volume of the video picture stream, which may be processed by an electronic device 320. The electronic device 320 includes a video encoder 303 coupled to the video source 301. The video encoder 303 may include hardware, software, or a combination of software and hardware to realize or implement aspects of the disclosed subject matter, which are described in more detail below. Compared to the video picture stream 302, the encoded video data 304 (or encoded video code stream 304) is depicted as a thin line to emphasize the relatively low data volume of the encoded video data 304 (or encoded video code stream 304), which may be stored in a streaming server 305 for future use. One or more streaming client terminal subsystems, such as client terminal subsystem 306 and client terminal subsystem 308 in FIG. 3, can access streaming server 305 to retrieve duplicate copies 307 and 309 of encoded video data 304. Client terminal subsystem 306 may include a video decoder 310, for example, in electronic device 330. Video decoder 310 decodes the derived duplicate copies 307 of encoded video data and produces an output video picture stream 311 that can be presented on display 312 (e.g., a display screen) or another presentation device (not shown). In some streaming systems, the encoded video data 304, video data 307, and video data 309 (e.g., a video code stream) can be encoded according to some video encoding / compression standard.

[0024] It should be noted that electronic device 320 and electronic device 330 may include other components (not shown). For example, electronic device 320 may include a video decoder (not shown), and electronic device 330 may further include a video encoder (not shown). The video decoder is used to decode received encoded video data, and the video encoder is used to encode the video data.

[0025] When encoding an image block in a video frame, an inter-frame prediction mode or an intra-frame prediction mode can be used, and a prediction block is generated based on one or more previously coded reference blocks. The prediction block may be an estimated version of the original block. A residual block can be generated by subtracting the original block from the prediction block, or vice versa, and the residual block can be used to represent a prediction residual (also called a prediction error). Because the amount of data required to represent the prediction residual can typically be less than the amount of data required to represent the original block, coding the residual block can achieve a relatively high compression ratio. For example, as shown in Figure 4, for the inter-frame prediction mode, the previously coded reference block 41 and the block to be coded 42 are located in two different video frames. As shown in Figure 5, for the intra-frame prediction mode, the previously coded reference block 51 and the block to be coded 52 are located in the same video frame.

[0026] Subsequently, the residual values ​​of the residual block in the spatial domain can be transformed into transform coefficients in the frequency domain. The transform can be realized by a two-dimensional transform similar to, for example, a discrete cosine transform (DCT). In the transform matrix, low-index transform coefficients (e.g., located in the upper-left region) can correspond to large spatial features and have relatively large magnitude values, while high-index transform coefficients (e.g., located in the lower-right region) can correspond to small spatial features and have relatively small magnitude values. Furthermore, a quantization matrix including quantization coefficients can be applied to the transform matrix, thereby quantizing all transform coefficients into quantized transform coefficients. As a result of quantization, the scale or magnitude values ​​of the transform coefficients may be reduced. Some high-index transform coefficients can be reduced to zero and then skipped in subsequent scanning and encoding steps.

[0027] FIG. 6 illustrates a portion of an exemplary video encoder 60, including a transform module 62, a quantization module 64, and an entropy coding module 66. Although not illustrated in FIG. 6, it should be understood that other modules, such as a prediction module, a dequantization module, and a reconstruction module, may be included in the video encoder 60. In operation, the video encoder 60 may obtain a video frame, which may include multiple image blocks. For simplicity, encoding one image block may be considered as an example here. To encode an image block, a prediction block may be first generated as an estimate of the image block. In conjunction with the above, the prediction block may be generated by the prediction module using an inter-frame prediction or intra-frame prediction mode. Subsequently, a difference value between the image block and the prediction block may be calculated to generate a residual block. The residual block may be transformed into transform coefficients by the transform module 62. During the transformation, the residual value in the spatial domain, which includes large and small features, is transformed into transform coefficients in the frequency domain, which includes high-frequency and low-frequency bands. Quantization module 64 may then quantize the transform coefficients using QM, thereby generating quantized transform coefficients, which may then be coded by entropy coding module 66 and ultimately transmitted from video encoder 60 as part of a bitstream.

[0028] FIG. 7 illustrates a portion of an exemplary video decoder 70, including an entropy decoding module 72, an inverse quantization (dequantization) module 74, and an inverse transform module 76. Although not illustrated in FIG. 7, it should be understood that the video decoder 70 may include other modules, such as a prediction module, a transform module, and a quantization module. In operation, the video decoder 70 receives a bitstream output from the video encoder 60, performs decoding on the bitstream according to an inter-frame prediction or intra-frame prediction mode, and outputs a reconstructed video frame. Here, the entropy decoding module 72 may generate quantized transform coefficients by performing entropy decoding on the input bitstream. The inverse quantization module 74 may perform inverse quantization on the quantized transform coefficients based on QM to obtain inversely quantized transform coefficients. The inverse transform module 76 performs inverse transform on the inversely quantized transform coefficients to generate a reconstructed residual block. Then, a reconstructed image block is generated based on the reconstructed residual block and the prediction block.

[0029] As can be seen from the above, QM is an essential part of the video encoding and decoding process. The QM setting can determine how much information of the transform coefficients is retained or filtered, and therefore, QM can affect the encoding performance and quality. In fact, both the encoder and the decoder require QM. Specifically, to accurately decode an image, the encoder must encode information about the quantized coefficients in the QM and transmit this information from the encoder to the decoder. In video encoding and decoding technologies and standards, QM may also be referred to as a scaling matrix or a weighting matrix. Therefore, the term "QM" used herein may be a general term that covers a quantization matrix, a scaling matrix, a weighting matrix, and other equivalent terms.

[0030] Below, some basic concepts related to the embodiments of the present application will be introduced and explained.

[0031] 1. Quantization matrix

[0032] The latest version of VTM (VVC Test Model, VVC Test Model) (i.e., VTM7) not only allows square TBs but also non-square TBs, so the number of QMs is relatively large. To reduce the number of digits and memory needs for QM signaling, VVC adopts an upsampling and copying design for non-square TBs and large square TBs.

[0033] Non-square QMs do not exist in VVC bitstreams, and they are obtained by copying the corresponding square QMs at the decoder side. More specifically, a 32x4 QM is obtained by copying rows 0, 8, 16, and 24 of a 32x32 QM. As shown in Figure 8, a 32x4 QM is obtained by downsampling a 32x32 QM. The shaded rows 0, 8, 16, and 24 are copied from the 32x32 QM to the 32x4 QM.

[0034] When the size of the square TB is larger than 8x8, the size of the corresponding QM in VTM7 is constrained to 8x8. An upsampling method is applied to these 8x8 QMs to create 16x16, 32x32, and 64x64 QMs. More specifically, to create a 16x16 QM, each element in the corresponding 8x8 QM is upsampled and copied to a 2x2 area, and to create a 32x32 QM, each element in the corresponding 8x8 QM is upsampled and copied to a 4x4 area.

[0035] In VTM7, as many as 28 QMs need to be coded. Table 1 determines the identifier variable (id) of the QM based on the variables sizeId and matrixId specified in Tables 2 and 3, respectively. Here, sizeId represents the size of the quantization matrix, and matrixId is an identifier of the type of QM based on the prediction mode (predMode) and color components (cIdx).

[0036] [Table 1]

[0037] [Table 2]

[0038] [Table 3]

[0039] In Table 2, when sizeId is greater than 3, it has a DC (Direct Current) coefficient, which is the element value at the (0,0) position in the QM. In VVC, when the DC value is 0, the QM may use the default QM, but the QM can still be transmitted. The main reason is that an uncoded QM may need to refer to the QM. When the DC value is not 0, the QM uses a user-defined QM, and is transmitted after encoding using the encoding method introduced below.

[0040] In Table 3, MODE_INTRA represents an intra-frame prediction mode, MODE_INTER represents an inter-frame prediction mode, and MODE_IBC represents an IBC (Intra Block Copy) prediction mode, Y represents luminance, and Cb and Cr represent chrominance.

[0041] 2. Quantization matrix coding method

[0042] To reduce bit overhead, VTM7 employs intra-frame and inter-frame predictive coding to encode 28 QMs.

[0043] In intraframe prediction mode, DPCM (Differential Pulse Code Modulation) coding is applied to the QM in a diagonal scan order. The DPCM intraframe residual also needs to be transmitted to the bitstream. For example, as shown in Figure 9, for a 4x4 QM, the diagonal scan order is (0,0), (1,0), (0,1), (2,0), (1,1), ..., (2,3), (3,3).

[0044] There are two interframe prediction modes: copy mode and predict mode. In copy mode, the current QM to be coded is exactly the same as a QM available for decoding, called the reference QM. This also means that copy mode has zero interframe residual, and of course, there is no need to signal the residual. The encoder needs to transmit an incremental ID between the current QM and its reference QM, allowing the decoder to reconstruct the current QM by directly copying the reference QM. Predict mode is similar to copy mode, but has an additional interframe residual. DPCM coding is applied to the interframe residual in diagonal scan order, and the encoder needs to transmit the DPCM interframe residual in the bitstream.

[0045] As described above, when the sizeId of a QM is greater than 3, an upsampling algorithm is applied to copy each element in the QM to a large square region. Because the DC coefficient at (0,0) is the most important for video reconstruction, the VTM 7 performs coding on it directly, rather than copying from the corresponding element of another QM. For each QM, a mode decision is used to calculate the bit costs of the three candidate modes of the QM (i.e., copy mode for inter-frame prediction mode, prediction mode for inter-frame prediction mode, and intra-frame prediction mode), and the one with the smallest bit cost is selected as the final optimal mode. The optimal mode is then adopted to perform coding on the QM.

[0046] 3. Quantization Matrix Signaling

[0047] By using QM, VVC supports frequency-related quantization of transform blocks. Suppose QM is W, and W[x][y] represents the QM weight of the transform coefficient at position (x, y) in the TB. For a transform coefficient coeff[x][y], the following Equation 1 is adopted to calculate the quantized transform coefficient level[x][y]:

[0048]

number

[0049] Here, QP is the quantization parameter (also called the quantization stride), and offset is the offset value. W[x][y]=16 indicates that no weighting is applied to the transform coefficient at position (x, y). When all elements in QM have the same value as 16, the effect is the same as not using QM.

[0050] The SPS (Sequence Parameter Set) syntax element sps_scaling_list_enable_flag is used to indicate whether to enable QM for those images whose picture header (PH) already references the SPS. When this flag is enabled, i.e., when sps_scaling_list_enable_flag is enabled, an additional flag in the PH is used to control whether to use the default QM (all elements equal to 16) or a user-defined QM. In VTM7, the user-defined QM is signaled in the APS (Adaptive Parameter Set). If the user-defined QM is enabled in the SPS and PH, an APS index can be sent in the PH to specify the QM set for the image that references this PH.

[0051] In one APS, as many as 28 groups of QM coding modes, Δid (increment id), AC, and DC coefficients should be reported. In each APS, the 28 groups of QMs are coded and decoded according to the increasing order of id.

[0052] In VVC Draft 7, the definitions of the QM coding mode, Δid (increment id), syntax and semantics of AC and DC coefficients are shown in Table 4 below.

[0053] [Table 4]

[0054] scaling_list_copy_mode_flag[ id ] equal to 1 indicates that the element values ​​of the current QM and its reference QM are the same. The reference QM is represented by scaling_list_pred_id_delta[ id ]. scaling_list_copy_mode_flag[ id ] equal to 0 indicates that scaling_list_pred_mode_flag is present.

[0055] scaling_list_pred_mode_flag[ id ] equal to 1 indicates that the current QM can be predicted from the reference QM. The reference QM is represented by scaling_list_pred_id_delta[ id ]. scaling_list_pred_mode_flag[ id ] equal to 0 indicates that the element value of the current QM is explicitly signaled. When not present, the value of scaling_list_pred_mode_flag[ id ] is inferred to be equal to 0.

[0056] scaling_list_pred_id_delta[id] represents the reference QM used to infer the predicted QM, i.e., ScalingMatrixPred[id]. When not present, the value of scaling_list_pred_id_delta[id] is inferred to be equal to 0. The value of scaling_list_pred_id_delta[id] should be in the range of 0 to maxIdDelta, where maxIdDelta is inferred based on id, as shown in Equation 2 below.

[0057]

number

[0058] That is, if id<2, then maxIdDelta=id; if id≧2 and<8, then maxIdDelta=id−2; if id≧8, then maxIdDelta=id−8.

[0059] The variables refId and matrixSize are calculated using the following formulas:

[0060] refId=id- scaling_list_pred_id_delta[ id ] Formula 3

number

[0061] That is, if id<2, then matrixSize=2; if id≧2 and <8, then matrixSize=4; if id≧8, then matrixSize=8.

[0062] The QM prediction matrix of matrixSize×matrixSize is denoted as ScalingMatrixPred[x][y], where x∈[0,matrixSize−1], y∈[0,matrixSize−1], and the variable ScalingMatrixDCPred represents the predicted value of DC, which is specifically calculated as follows:

[0063] When scaling_list_copy_mode_flag[id] and scaling_list_pred_mode_flag[id] are both equal to 0, all elements of ScalingMatrixPred are set to be equal to 8, and the value of ScalingMatrixDCPred is set to be equal to 8.

[0064] Otherwise, when scaling_list_pred_id_delta[id] is equal to 0, set all elements of ScalingMatrixPred to be equal to 16, and set the value of ScalingMatrixDCPred to be equal to 16.

[0065] Otherwise, when scaling_list_copy_mode_flag[ id ] or scaling_list_pred_mode_flag[ id ] is equal to 1 and scaling_list_pred_id_delta[ id ] is greater than 0, ScalingMatrixPred is set equal to ScalingMatrixPred[refId], and the calculation of the value of ScalingMatrixDCPred is as follows: if refId is greater than 13, the value of ScalingMatrixDCPred is set equal to ScalingMatrixDCRec[ refId - 14 ], otherwise (i.e., refId is less than or equal to 13), the value of ScalingMatrixDCPred is set equal to ScalingMatrixPred[0][0].

[0066] scaling_list_dc_coef[id-14] is used to calculate the value of the variable ScalingMatrixDC[id-14] when id is greater than 13, and is shown in Equation 5 below.

[0067] ScalingMatrixDCRec[ id - 14 ] = ( ScalingMatrixDCPred + scaling_list_dc_coef[ id - 14 ] + 256 ) % 256 ) Formula 5

[0068] Here, % indicates that the remainder is to be found.

[0069] When not present, the value of scaling_list_dc_coef[ id - 14 ] is inferred to be equal to 0. The value of scaling_list_dc_coef[ id - 14 ] must be in the range of -128 to 127, inclusive. The value of ScalingMatrixDCRec[ id - 14 ] must be greater than 0.

[0070] scaling_list_delta_coef[id][i] represents the difference between the current matrix coefficient ScalingList[id][i] and the previous matrix coefficient ScalingList[id][i-1] when scaling_list_copy_mode_flag[id] is equal to 0. The value of scaling_list_delta_coef[id][i] must be within the range of -128 to 127 (inclusive). When scaling_list_copy_mode_flag[id] is equal to 1, all elements of ScalingList[id] are set to 0.

[0071] The ScalingMatrixRec[id] of the QM of matrixSize×matrixSize can be calculated using the following formula 6.

[0072] ScalingMatrixRec[ id ][ x ][ y ] = ( ScalingMatrixPred[ x ][ y ] + ScalingList[ id ][ k ] + 256 ) % 256 ) Formula 6

[0073] Here, % indicates finding the complement, and k∈[0,(matrixSize×matrixSize-1)].

[0074] x= DiagScanOrder[ Log2( matrixSize ) ][ Log2( matrixSize ) ][ k ]

[0000] , and y = DiagScanOrder[ Log2( matrixSize ) ][ Log2( matrixSize ) ][ k ]

[0001] .

[0075] The value of ScalingMatrixRec[ id ][ x ][ y ] must be greater than 0.

[0076] Consider the decoding process of one QM, that is, the process of decoding based on the above syntax elements to obtain ScalingMatrixRec[id][x][y] and ScalingMatrixDCRec.

[0077] 4. Limiting TB size with SPS

[0078] In VVC Draft 7, the definition of SPS syntax and semantics related to TB size constraints is shown in Table 5 below.

[0079] [Table 5]

[0080] sps_max_luma_transform_size_64_flag equal to 1 indicates that the maximum transform block size for luma sampling is equal to 64. sps_max_luma_transform_size_64_flag equal to 0 indicates that the maximum transform block size for luma sampling is equal to 32.

[0081] chroma_format_idc represents the chrominance sampling corresponding to the luma sampling, as shown in Table 6.

[0082] [Table 6]

[0083] In Table 6 above, SubWidthC and SubHeightC respectively represent the width and height of a CTU (Coding Tree Unit) corresponding to the chrominance component, and Monochrome represents the absence of a chrominance component.

[0084] separate_colour_plane_flag equal to 1 indicates that each of the three color components of the 4:4:4 chroma format is coded. separate_colour_plane_flag equal to 0 indicates that no color component is coded separately. When separate_colour_plane_flag is not present, its value is inferred to be equal to 0.

[0085] When separate_colour_plane_flag is equal to 1, the coded image consists of three separate components, each consisting of coded samples of one colour plane (Y, Cb or Cr), and uses a monochromatic coding syntax, where each colour plane is associated with a specific colour_plane_id value.

[0086] colour_plane_id specifies the colour plane associated with the slice associated with PH, and when separate_colour_plane_flag is equal to 1, the value of colour_plane_id should be in the range 0 to 2 (inclusive). The colour_plane_id values ​​0, 1 and 2 correspond to the Y, Cb and Cr planes respectively. It should be noted that there is no dependency between the decoding process of images with different colour_plane_id values.

[0087] sps_log2_ctu_size_minus5+5 represents the size of the luma coding treeblock for each CTU. The value of sps_log2_ctu_size_minus5 must be 2 or less to ensure bitstream consistency.

[0088] Based on sps_log2_ctu_size_minus5, the maximum luma coding block size can be calculated.

[0089] CtbLog2SizeY = sps_log2_ctu_size_minus5 + 5 CtbSizeY = 1 << CtbLog2SizeY

[0090] where CtbSizeY represents the maximum luma coding block size, CtbLog2SizeY represents the logarithm of CtbSizeY to the base 2, and << is the left shift operator.

[0091] log2_min_luma_coding_block_size_minus2+2 represents the minimum luma coding block size. The value range of log2_min_luma_coding_block_size_minus2 must be within the range of 0 to sps_log2_ctu_size_minus5+3 (inclusive).

[0092] The variables MinCbLog2SizeY, MinCbSizeY, and VSize are calculated as follows:

[0093] MinCbLog2SizeY = log2_min_luma_coding_block_size_minus2 + 2 Formula 7 MinCbSizeY = 1 << MinCbLog2SizeY Equation 8 VSize = Min( 64, CtbSizeY ) Formula 9

[0094] where MinCbSizeY represents the minimum luma coding block size, MinCbLog2SizeY represents the logarithm of MinCbSizeY to the base 2, VSize represents the maximum luma coding block size, and << is the left shift operator. The value of MinCbSizeY must be less than or equal to VSize.

[0095] The width and height of each chrominance CTB (Coding Tree Block), that is, the variables CtbWidthC and CtbHeightC, are determined using the following method.

[0096] If chroma_format_idc is equal to 0 (monochrome) or Separate_color_Plane_flag is equal to 1, then CtbWidthC and CtbHeightC are both equal to 0.

[0097] Otherwise, CtbWidthC and CtbHeightC are calculated using the following formulas:

[0098] CtbWidthC = CtbSizeY / SubWidthC Formula 10 CtbHeightC = CtbSizeY / SubHeightC Formula 11

[0099] Here, CtbSizeY represents the size of the luminance CTB.

[0100] Currently, the encoding method for quantization matrices adopted by VVC requires all 28 QMs to be coded and transmitted in APS, which requires the QM signaling to occupy a relatively large number of codewords, resulting in large bit overhead and increased computational complexity on the decoder side. In the technical solution provided by the embodiments of the present application, a first parameter set corresponding to a video frame to be decoded is obtained, and an effective QM is determined based on the syntax elements included in the first parameter set. The effective QM refers to the QM actually used to quantize the transform coefficients in the process of encoding and generating the video frame to be decoded, and then decoding is performed using the effective QM. In this way, the encoder only codes and transmits the effective QM, thereby saving the codewords required for QM signaling and reducing bit overhead. The decoder only needs to decode using the effective QM, thereby reducing computational complexity on the decoder side.

[0101] It should be noted that the technical solutions provided by the embodiments of the present application can be applied to the H.266 / VCC standard or next generation video encoding and decoding standards, but the embodiments of the present application are not limited thereto.

[0102] It should be further noted that in the video decoding method provided by the embodiments of the present application, each step is performed by a decoding device, while in the video encoding method provided by the embodiments of the present application, each step is performed by an encoding device, and both the decoding device and the encoding device may be computer devices, which refer to electronic devices with data calculation, processing and storage capabilities, such as PCs, mobile phones, tablet computers, media players, dedicated video conferencing devices, or servers.

[0103] In addition, the methods provided herein may be used alone or in combination with other methods in any order. The encoder and decoder based on the methods provided herein may be implemented by one or more processors or one or more integrated circuits. The technical solutions of the present application are introduced and described below through several embodiments.

[0104] 10, which shows a flowchart of a video decoding method provided by one embodiment of the present application. In this embodiment, the method is mainly applied to the decoding device introduced above as an example. The method may include the following steps (1001 to 1003):

[0105] Step 1001: Obtain a first parameter set corresponding to a video frame to be decoded.

[0106] The video frame to be decoded may be any one video frame (also referred to as an image frame) to be decoded in the video to be decoded. The first parameter set includes a parameter set used to define syntax elements related to QM, and for example, the decoding device can obtain QM by decoding based on the syntax elements in the first parameter set.

[0107] Optionally, the first parameter set is APS. Of course, in some other embodiments, the first parameter set may not be APS, and may be SPS, etc., and the embodiments of the present application are not limited thereto.

[0108] Step 1002: Determine an effective QM based on the syntax elements included in the first parameter set, where the effective QM refers to the QM actually used when performing inverse quantization on the quantized transform coefficients during the decoding process of the video frame to be decoded.

[0109] Assuming that the number of QMs that may be used when performing inverse quantization on the quantized transform coefficients is n, the number of effective QMs may be less than or equal to n, where n is a positive integer. For example, if all n QMs are actually used when performing inverse quantization on the quantized transform coefficients, the number of effective QMs is n, and if some QMs (e.g., m QMs, where m is a positive integer less than n) of the total n QMs are actually used when performing inverse quantization on the quantized transform coefficients, the number of effective QMs is m.

[0110] By defining syntax elements used to determine valid QMs in the first parameter set, the decoding device can determine which QMs are valid and which are not valid QMs by reading the syntax elements. For QMs that are not valid QMs (which may also be referred to as invalid QMs), i.e., QMs that are not actually used when quantizing transform coefficients in the process of encoding and generating a video frame to be decoded, the decoding device may not need to decode them.

[0111] Optionally, for other QMs that do not belong to the valid QMs, all elements thereof are predefined to have a default value, which is optionally 16, as shown in Equation 1. In this case, the scaling coefficients of all transform coefficients in the TB are all 1, which has the same effect as not using a QM.

[0112] Step 1003: Decoding is performed for the valid QM.

[0113] After determining the valid QM, the number of valid QMs may be one or more, so the decoding device needs to perform decoding for each valid QM. Taking any valid QM as an example, when decoding the valid QM, it can determine the coding mode corresponding to the valid QM, and then decode the valid QM based on the coding mode.

[0114] For example, referring to Table 1 above, if it is determined that there are 28 possible QMs that can be used when quantizing transform coefficients, and 12 of these are valid QMs, then the decoding device only needs to decode the 12 valid QMs and does not need to decode the remaining 16 invalid QMs.

[0115] As described above, the technical solutions provided by the embodiments of the present application obtain a first parameter set corresponding to a video frame to be decoded, and then determine an effective QM based on the syntax elements included in the first parameter set. The effective QM refers to the QM actually used when quantizing transform coefficients in the process of encoding and generating the video frame to be decoded, and then decoding is performed using the effective QM. In this way, the decoder only needs to perform decoding using the effective QM, thereby reducing the computational complexity on the decoder side.

[0116] In an exemplary embodiment, the step of determining a valid QM based on the syntax elements included in the first parameter set includes the following sub-steps:

[0117] 1: Determine the valid size range of the QM based on the syntax elements included in the first parameter set.

[0118] The valid size range of the QM defines the minimum and maximum sizes of the QM that are actually used when performing inverse quantization on the quantized transform coefficients during the decoding process, and the QM size values ​​are powers of 2, such as 2, 4, 8, 16, 32, and 64.

[0119] 2: QMs that fall within the valid size range are determined as valid QMs.

[0120] For example, when the valid size range of a QM is [4,32], valid QMs include a 4x4 QM, an 8x8 QM, a 16x16 QM, and a 32x32 QM. Also, when the valid size range of a QM is [8,16], valid QMs include an 8x8 QM and a 16x16 QM.

[0121] For example, assuming that the valid size range of QMs is determined to be [8,16], referring to Tables 1 and 2 above, the sizeId corresponding to an 8x8 size QM is 3, and the sizeId corresponding to a 16x16 size QM is 4. The decoding device determines that a total of 12 QMs with ids from 8 to 19 are valid QMs, and the remaining 16 QMs with ids from 0 to 7 and 20 to 27 are invalid QMs.

[0122] In one example, the following method is adopted to determine the valid size range of the QM based on the syntax elements included in the first parameter set.

[0123] 1.1: Determine the minimum luma coding block size, the luma coding tree block size and the maximum luma TB size based on the syntax elements included in the first parameter set.

[0124] Optionally, the first parameter set defines a first syntax element, which is used to indicate a minimum luma coding block size; the first parameter set defines a second syntax element, which is used to indicate a block size of a luma coding tree; and the first parameter set defines a third syntax element, which is used to indicate a maximum luma TB size. The decoding device reads the first, second, and third syntax elements from the first parameter set to determine the minimum luma coding block size, the block size of a luma coding tree, and the maximum luma TB size.

[0125] 1.2: Determine the valid size range of the luma QM based on the minimum luma coding block size, the luma coding tree block size, and the maximum luma TB size, where the valid size range of the luma QM includes the minimum size and the maximum size of the luma QM.

[0126] Optionally, the decoding device determines the minimum size of the luma QM based on the smallest luma coding block size. For example, the smallest luma coding block size is determined as the minimum size of the luma QM. The decoding device determines the larger of the block size of the luma coding tree and the maximum luma TB size as the maximum size of the luma QM. For example, when the block size of the luma coding tree is larger than the maximum luma TB size, the block size of the luma coding tree is determined as the maximum size of the luma QM. When the block size of the luma coding tree is smaller than the maximum luma TB size, the maximum luma TB size is determined as the maximum size of the luma QM. When the block size of the luma coding tree is equal to the maximum luma TB size, the two are equal, so either the block size of the luma coding tree or the maximum luma TB size is determined as the maximum size of the luma QM, and the result is the same.

[0127] 1.3: Determine the effective size range of the chrominance QM based on the effective size range of the luma QM and the sampling rate of the chrominance component relative to the luma component, where the effective size range of the chrominance QM includes the minimum size and maximum size of the chrominance QM.

[0128] Optionally, a fourth syntax element is defined in the first parameter set, and the fourth syntax element is used to indicate a sampling rate of the chrominance component relative to the luma component.

[0129] Optionally, the decoding device calculates the minimum size of the chrominance QM based on the minimum size of the luma QM and the sampling rate of the chrominance component relative to the luma component, and calculates the maximum size of the chrominance QM based on the maximum size of the luma QM and the sampling rate of the chrominance component relative to the luma component.

[0130] In an exemplary embodiment, taking the first parameter set as an APS as an example, the syntax elements and syntax structure table included in the APS are shown in Table 7 below.

[0131] [Table 7]

[0132] aps_qm_size_info_present_flag indicates whether syntax elements related to QM size are present in the bitstream. A value of 1 indicates that syntax elements related to QM size are present in the bitstream, and based on this, the valid size range of QM can be determined, thereby determining which size of QM needs to be decoded. A value of 0 indicates that syntax elements related to QM size are not present in the bitstream, and all sizes of QM need to be decoded.

[0133] For aps_log2_ctu_size_minus5, its value +5 indicates the block size of the luma coding tree. Its value is specified to be the same as the numeric value of the syntax element sps_log2_ctu_size_minus5.

[0134] For aps_log2_min_luma_coding_block_size_minus2, its value +2 indicates the minimum luma coding block size. Its value is specified to be the same as the numeric value of the syntax element sps_log2_min_luma_coding_block_size_minus2.

[0135] Regarding aps_max_luma_transform_size_64_flag, a value of 1 indicates that the maximum luminance TB size is 64, and a value of 0 indicates that the maximum luminance TB size is 32. The value is specified to be the same as the numerical value of the syntax element sps_max_luma_transform_size_64_flag.

[0136] aps_chroma_format_idc indicates the sampling rate of the chrominance component relative to the luminance component, and is specifically shown in Table 6. Its value is specified to be the same as the numerical value of the syntax element chroma_format_idc.

[0137] Based on the above syntax elements, the derivation process of the variables minQMSizeY (representing the minimum size of the luminance QM) and maxQMSizeY (representing the maximum size of the luminance QM) is as follows:

[0138] When the value of the syntax element aps_qm_size_info_present_flag is 1, the following applies:

[0139] minQMSizeY = 1 << (aps_log2_min_luma_coding_block_size_minus2+2) Equation 12

number

[0140] where << is the left shift operator,

number

[0141] When the value of the syntax element aps_qm_size_info_present_flag is 0:

[0142] minQMSizeY = 4, maxQMSizeY = 64.

[0143] The derivation process of the variables minQMSizeUV (representing the minimum size of the chrominance QM) and maxQMSizeUV (representing the maximum size of the chrominance QM) is as follows.

[0144] When the value of the syntax element aps_qm_size_info_present_flag is 1, the following applies:

[0145]

number

[0146] where ! represents the logical negation operation,

number

[0147] The above formulas 14 and 15 are interpreted as follows.

[0148] If aps_chroma_format_idc does not exist, then minQMSizeUV=0, and conversely, minQMSizeUV = minQMSizeY / SubWidthC, If aps_chroma_format_idc does not exist, maxQMSizeUV=0 and conversely, maxQMSizeUV = maxQMSizeY / SubHeightC.

[0149] When the value of the syntax element aps_qm_size_info_present_flag is 0:

[0150] minQMSizeUV = 2, maxQMSizeUV = 32.

[0151] In the syntax structure table shown in Table 7, the variable cIdx represents the color component corresponding to the current QM. For the luma component Y, its value is 0, for the chrominance component Cb, its value is 1, and for the chrominance component Cr, its value is 2. The variable matrixSize represents the actual encoding size of the current QM and is pointed to by the third column of Table 2. The variable matrixQMSize represents the TB size corresponding to the current QM and is pointed to by Tables 1 and 2.

[0152] In the syntax structure table shown in Table 7, the decoding device first determines whether to decode the current QM based on the two conditions proposed by the present application. For example, to determine whether the first QM is a valid QM (the first QM can be any one of the 28 available QMs), if the first QM satisfies either the first or second condition, the first QM is determined to be a valid QM.

[0153] Here, the first condition is cIdx==0 && (matrixQMSize >= minQMSizeY && matrixQMSize <= maxQMSizeY), which indicates that the first QM belongs to the luma component used in the quantization process of luma TB, and the first QM is within the valid size range of luma QM [MinQMSizeY, MaxQMSizeY], where MinQMSizeY represents the minimum size of luma QM and MaxQMSizeY represents the maximum size of luma QM. The second condition is cIdx!=0 && (matrixQMSize >= minQMSizeUV && matrixQMSize <= maxQMSizeUV), which indicates that the first QM belongs to the chrominance components used in the quantization process of the chrominance TB, and that the first QM is within the valid size range of the chrominance QM [MinQMSizeUV, MaxQMSizeUV], where MinQMSizeUV is the minimum size of the chrominance QM and MaxQMSizeUV is the maximum size of the chrominance QM.

[0154] In the above example, the decoding device needs to calculate the valid size range of the QM based on the syntax elements included in the first parameter set, and then determine the valid QM based on the valid size range. In the example introduced below, the syntax element for the valid size range of the luma QM may be directly defined in the first parameter set, and the decoding device can directly obtain the valid size range of the luma QM after reading the syntax element, and then determine the valid size range of the chroma QM based on the valid size range of the luma QM in conjunction with the chroma format. Specifically, as follows:

[0155] In another example, the following method is adopted to determine the valid size range of the QM based on the syntax elements included in the first parameter set.

[0156] 1.1: Determine a valid size range of the luminance QM based on syntax elements included in the first parameter set, where the valid size range of the luminance QM includes a minimum size and a maximum size of the luminance QM.

[0157] Optionally, a fifth syntax element is defined in the first parameter set, and the fifth syntax element is used to indicate the minimum size of the luma QM. A sixth syntax element is defined in the APS, and the fifth syntax element is used to indicate the maximum size of the luma QM. The decoding device reads the fifth and sixth syntax elements from the first parameter set and determines the minimum and maximum sizes of the luma QM.

[0158] 1.2: Determine the effective size range of the chrominance QM based on the effective size range of the luma QM and the sampling rate of the chrominance component relative to the luma component, where the effective size range of the chrominance QM includes the minimum and maximum sizes of the chrominance QM.

[0159] Optionally, a fourth syntax element is defined in the first parameter set, and the fourth syntax element is used to indicate a sampling rate of the chrominance component relative to the luma component.

[0160] Optionally, the decoding device calculates the minimum size of the chrominance QM based on the minimum size of the luma QM and the sampling rate of the chrominance component relative to the luma component, and calculates the maximum size of the chrominance QM based on the maximum size of the luma QM and the sampling rate of the chrominance component relative to the luma component.

[0161] In an exemplary embodiment, taking the first parameter set as an APS as an example, the syntax elements and syntax structure table included in the APS are shown in Table 8 below.

[0162] [Table 8]

[0163] aps_qm_size_info_present_flag indicates whether syntax elements related to QM size are present in the bitstream. A value of 1 indicates that syntax elements related to QM size are present in the bitstream, and based on this, the valid size range of QM can be determined, thereby determining which size of QM needs to be decoded. A value of 0 indicates that syntax elements related to QM size are not present in the bitstream, and all sizes of QM need to be decoded.

[0164] For aps_log2_min_luma_qm_size_minus2, the value +2 indicates the minimum size of the luma QM.

[0165] For aps_log2_max_luma_qm_size_minus5, the value +5 indicates the maximum size of the luma QM.

[0166] Based on the above syntax elements, the derivation process of the variables minQMSizeY (representing the minimum size of the luminance QM) and maxQMSizeY (representing the maximum size of the luminance QM) is as follows:

[0167] When the value of the syntax element aps_qm_size_info_present_flag is 1, the following applies:

[0168] minQMSizeY = 1 << (aps_log2_min_luma_qm_size_minus2 + 2) Equation 16 maxQMSizeY = 1 << (aps_log2_max_luma_qm_size_minus5 + 5) Equation 17

[0169] where << is the left shift operator.

[0170] When the value of aps_qm_size_info_present_flag is 1, minQMSizeY and maxQMSizeY are defined to be equal to the numerical values ​​of the TB size variables MinCbSizeY and VSize, respectively, calculated and obtained by the SPS syntax element.

[0171] When the value of the syntax element aps_qm_size_info_present_flag is 0:

[0172] minQMSizeY = 4, maxQMSizeY = 64.

[0173] aps_chroma_format_idc indicates the sampling rate of the chrominance component relative to the luminance component, and is specifically shown in Table 6. Its value is specified to be the same as the numerical value of the syntax element chroma_format_idc.

[0174] The derivation process of the variables minQMSizeUV (representing the minimum size of the chrominance QM) and maxQMSizeUV (representing the maximum size of the chrominance QM) is as follows.

[0175] When the value of the syntax element aps_qm_size_info_present_flag is 1, the following applies:

[0176]

number

[0177] where ! represents the logical negation operation,

number

[0178] When the value of the syntax element aps_qm_size_info_present_flag is 0:

[0179] minQMSizeUV = 2, maxQMSizeUV = 32.

[0180] In some other examples, the decoding device can also determine valid QMs based on syntax elements included in the SPS. Specifically, the decoding device can calculate a valid size range [MinQMSizeY, MaxQMSizeY] for the luma QM and a valid size range [MinQMSizeUV, MaxQMSizeUV] for the chroma QM based on the syntax elements included in the SPS. Here, the variable MinQMSizeY represents the minimum size of the luma QM, the variable MaxQMSizeY represents the maximum size of the luma QM, the variable MinQMSizeUV represents the minimum size of the chroma QM, and the variable MaxQMSizeUV represents the maximum size of the chroma QM.

[0181] As seen in conjunction with the SPS syntax structure table shown in Table 5 above, the above variables can be calculated and obtained using the following formulas:

[0182]

number

[0183] where << is the left shift operator and ! represents the logical negation operation.

number

[0184] Compared to determining a valid QM based on syntax elements contained in an SPS, by defining relevant syntax elements in an APS and determining a valid QM based on the relevant syntax elements defined in the APS, the parsing dependency between the APS and SPS codestreams can be eliminated, and therefore decoding of the APS does not need to depend on the syntax elements of the SPS.

[0185] In an exemplary embodiment, the step of determining a valid QM based on the syntax elements included in the first parameter set includes the following sub-steps:

[0186] 1: Read the value of the flags syntax element corresponding to the first QM from the first parameter set.

[0187] 2: If the value of the flag syntax element corresponding to the first QM is a first numeric value, determine that the first QM belongs to the valid QMs.

[0188] 3: If the value of the flag syntax element corresponding to the first QM is the second numeric value, determine that the first QM does not belong to the valid QMs.

[0189] In this embodiment, a flag syntax element is defined in the APS to indicate whether a QM belongs to a valid QM. The descriptor of the flag syntax element may be u(1), which represents a 1-bit unsigned integer. For example, a value of 1 for the flag syntax element indicates that the QM belongs to a valid QM and should be decoded. A value of 0 for the flag syntax element indicates that the QM does not belong to a valid QM and should not be decoded. For QMs that do not require decoding, all elements are predefined to have default values. Optionally, the default value is 16. See Equation 1. In this case, the scaling coefficients of all transform coefficients in the TB are all 1, which has the same effect as not using a QM.

[0190] Also, the first QM may be any one of the available QMs, ie, any one of the total 28 QMs mentioned above.

[0191] Optionally, the first parameter set is APS. Of course, in some other embodiments, the first parameter set may not be APS, and the embodiments of the present application are not limited thereto.

[0192] In an exemplary embodiment, taking the first parameter set as an APS as an example, the syntax elements and syntax structure table included in the APS are shown in Table 9 below.

[0193] [Table 9]

[0194] Optionally, the flag syntax element is scaling_matrix_present_flag. For scaling_matrix_present_flag[id], a value of 1 indicates that the current QM needs to be decoded, and a value of 0 indicates that the current QM does not need to be decoded, and the decoding device can infer that all elements of the QM are 16.

[0195] Optionally, a luma QM corresponds to one flag syntax element, indicating whether the luma QM needs to be decoded. A first chroma QM (i.e., a QM corresponding to Cb) and a second chroma QM (i.e., a QM corresponding to Cr) having the same prediction mode and the same size share the same flag syntax element to indicate whether the first chroma QM and the second chroma QM need to be decoded. That is, the first chroma QM and the second chroma QM do not need to use one flag syntax element independently, which further contributes to saving bit overhead of QM coding signaling.

[0196] In an exemplary embodiment, taking the first parameter set as an APS as an example, the syntax elements and syntax structure table included in the APS are shown in Table 10 below.

[0197] [Table 10]

[0198] When the value of scaling_matrix_present_flag[ predMode != MODE_INTRA ][ cIdx != 0 ][ sizeId ] is 1, it indicates that when decoding as a luma QM, the luma QM is coded in APS, and when decoding as a chroma QM, it indicates that the QMs corresponding to the chroma Cb and Cr whose prediction mode is predMode and whose size is the same are coded in APS. When the value of this syntax element is 0, it indicates that there is no need to decode the luma QM or the two chroma QMs, and the decoding device can infer that these elements are all 16.

[0199] It is important to note that when the encoding device sets the value of the flag syntax element corresponding to each QM, i.e., when determining which QMs need to be encoded and which QMs do not need to be encoded, it may do so based on the size of the QM, or based on the encoding prediction mode corresponding to the QM, or based on the YUV color components corresponding to the QM, or may take into consideration multiple elements of the QM size, encoding prediction mode, and YUV color components in an integrated manner, and the embodiments of the present application are not limited to this.

[0200] In this embodiment, by defining one flag syntax element in the first parameter set, the flag syntax element can indicate whether the QM belongs to a valid QM, thereby more flexibly indicating whether each QM needs to be decoded.

[0201] 11, which shows a flowchart of a video encoding method provided by one embodiment of the present application. In this embodiment, the method is mainly applied to the encoding device introduced above as an example. The method may include the following steps (1101-1102):

[0202] Step 1101: Determine an effective QM corresponding to the video frame to be coded, where the effective QM refers to the QM actually used when quantizing the transform coefficients in the coding process of the video frame to be coded.

[0203] The video frame to be coded may be any one video frame (also called an image frame) to be coded in the video to be coded.

[0204] Assuming that the number of QMs that may be used when quantizing transform coefficients is n, the number of effective QMs may be less than or equal to n, where n is a positive integer. For example, if all n QMs are actually used when quantizing transform coefficients, the number of effective QMs is n, and if some QMs (e.g., m QMs, where m is a positive integer less than n) of the total n QMs are actually used when quantizing transform coefficients, the number of effective QMs is m.

[0205] Optionally, for other QMs that do not belong to the valid QMs, all elements thereof are predefined to have a default value, which is optionally 16, as shown in Equation 1. In this case, the scaling coefficients of all transform coefficients in the TB are all 1, which has the same effect as not using a QM.

[0206] Step 1102: Perform encoding on the syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set, where the first parameter set includes a parameter set used to define the syntax elements associated with the QM.

[0207] After determining the valid QM, the number of valid QMs may be one or more, so the encoding device must perform encoding for each valid QM. Taking any valid QM as an example, when encoding the valid QM, an optimal mode corresponding to the valid QM can be determined, and then the valid QM is encoded based on the optimal mode. Here, the optimal mode may be the mode with the lowest bit cost selected from the three candidate modes introduced above: the copy model of the inter-frame prediction mode, the prediction mode of the inter-frame prediction mode, and the intra-frame prediction mode.

[0208] For example, referring to Table 1 above, if it is determined that the number of QMs that can be used when quantizing transform coefficients is 28, of which 12 are valid QMs, the encoding device only needs to encode the 12 valid QMs and does not need to encode the remaining 16 invalid QMs.

[0209] In addition, the encoding device needs to perform coding on a valid QM, and also needs to perform coding on syntax elements used to determine the valid QM, so that the decoding device can determine the valid QM based on the syntax elements. The encoding device performs coding on the syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set. The first parameter set may be an APS or another parameter set used to define syntax elements related to a QM, and the embodiments of the present application are not limited thereto.

[0210] As described above, the technical solutions provided by the embodiments of the present application determine an effective QM corresponding to a video frame to be coded, where the effective QM refers to the QM actually used when quantizing transform coefficients in the coding process of the video frame to be coded, and then perform coding on the syntax elements used to determine the effective QM and the effective QM to generate a codestream corresponding to a first parameter set. In this way, the encoder only encodes and transmits the effective QM, thereby saving the codewords required for QM signaling and reducing bit overhead, and the decoder only needs to decode using the effective QM, thereby reducing the computational complexity of the decoder.

[0211] In addition, the encoding process of the encoding side device corresponds to the decoding process of the decoding side device, and for details not described in detail in the encoding process, please refer to the introduction and description of the above embodiment of the decoding process, and will not be described in detail again here.

[0212] The following are examples of apparatuses of the present application, which can be used to carry out the method embodiments of the present application. For details not disclosed in the apparatus embodiments of the present application, reference can be made to the method embodiments of the present application.

[0213] Referring to Figure 12, it shows a block diagram of a video decoding device provided by one embodiment of the present application. The device has a function for implementing the above-mentioned example of the video decoding method, and the above function may be implemented by hardware, or by hardware executing corresponding software. The device may be the decoding side device introduced above, or may be installed on the decoding side device. The device 1200 may include a parameter acquisition module 1210, a QM determination module 1220, and a QM decoding module 1230.

[0214] The parameter acquisition module 1210 is used to acquire a first parameter set corresponding to a video frame to be decoded, where the first parameter set is a parameter set used to define a syntax element related to QM.

[0215] The QM determination module 1220 is used to determine an effective QM based on the syntax elements included in the first parameter set, where the effective QM refers to the QM actually used when performing inverse quantization on the quantized transform coefficients during the decoding process of the video frame to be decoded.

[0216] The QM decoding module 1230 is used to perform decoding on the valid QM.

[0217] In an exemplary embodiment, as shown in FIG. 13, the QM determination module 1220 includes a range determination unit 1221 and a QM determination unit 1222.

[0218] The range determining unit 1221 is used to determine the valid size range of the QM according to the syntax elements included in the first parameter set.

[0219] The QM determining unit 1222 is used for determining the QM that falls within the valid size range as the valid QM.

[0220] In an exemplary embodiment, the range determination unit 1221 is determining a minimum luma coding block size, a luma coding tree block size, and a maximum luma TB size based on syntax elements included in the first parameter set; determining a valid size range of a luma QM based on the minimum luma coding block size, the luma coding tree block size, and the maximum luma TB size, where the valid size range of the luma QM includes the minimum size and the maximum size of the luma QM; It is used for determining the valid size range of the chrominance QM based on the valid size range of the luminance QM and the sampling rate of the chrominance component relative to the luminance component, wherein the valid size range of the chrominance QM includes the minimum size and maximum size of the chrominance QM.

[0221] In an exemplary embodiment, the range determination unit 1221 is determining a minimum size of the luma QM based on the minimum luma coding block size; and determining a relatively larger value out of the block size of the luma coding tree and the maximum luma TB size as the maximum luma QM size.

[0222] In an exemplary embodiment, the range determination unit 1221 is determining a valid size range of the luminance QM based on syntax elements included in the first parameter set, where the valid size range of the luminance QM includes a minimum size and a maximum size of the luminance QM; It is used for determining the valid size range of the chrominance QM based on the valid size range of the luminance QM and the sampling rate of the chrominance component relative to the luminance component, wherein the valid size range of the chrominance QM includes the minimum size and maximum size of the chrominance QM.

[0223] In an exemplary embodiment, the range determination unit 1221 is calculating a minimum size of the chrominance QM based on a minimum size of the luma QM and a sampling rate of the chrominance components relative to the luma component; Calculating the maximum size of the chrominance QM based on the maximum size of the luma QM and the sampling rate of the chrominance component relative to the luma component.

[0224] In an exemplary embodiment, the QM determination unit 1222 is If the first QM satisfies one of the first and second conditions, the first QM is used to determine the valid QM; where the first condition is cIdx==0 && (matrixQMSize >= minQMSizeY && matrixQMSize <= maxQMSizeY), which indicates that the first QM belongs to the luma component used in the quantization process of luma TB, and the first QM is within the valid size range of the luma QM [MinQMSizeY, MaxQMSizeY], where MinQMSizeY represents the minimum size of the luma QM and MaxQMSizeY represents the maximum size of the luma QM; The second condition is cIdx!=0 && (matrixQMSize >= minQMSizeUV && matrixQMSize <= maxQMSizeUV), which indicates that the first QM belongs to the chrominance components used in the quantization process of the chrominance TB, and the first QM is within the valid size range of the chrominance QM [MinQMSizeUV, MaxQMSizeUV], where MinQMSizeUV represents the minimum size of the chrominance QM and MaxQMSizeUV represents the maximum size of the chrominance QM.

[0225] In an exemplary embodiment, as shown in FIG. 13, the QM determination module 1220 includes an element reading unit 1223 and a QM judging unit 1224.

[0226] The element reading unit 1223 is used to read the value of the flag syntax element corresponding to the first QM from the first parameter set.

[0227] The QM determination unit 1224 is used to determine that the first QM belongs to the valid QM if the value of the flag syntax element corresponding to the first QM is a first numerical value, and to determine that the first QM does not belong to the valid QM if the value of the flag syntax element corresponding to the first QM is a second numerical value.

[0228] In an example embodiment, the first and second chroma QMs with the same prediction mode and the same size share the same flag syntax element.

[0229] In an exemplary embodiment, the flag syntax element is scaling_matrix_present_flag.

[0230] In an exemplary embodiment, the first parameter set is APS.

[0231] In an exemplary embodiment, for other QMs that do not belong to the valid QMs, all elements thereof are predefined to be default values.

[0232] In the exemplary embodiment, the default value is 16.

[0233] As described above, the technical solutions provided by the embodiments of the present application obtain a first parameter set corresponding to a video frame to be decoded, and then determine an effective QM based on the syntax elements included in the first parameter set. The effective QM refers to the QM actually used when quantizing transform coefficients in the process of encoding and generating the video frame to be decoded, and then decoding is performed using the effective QM. In this way, the decoder only needs to perform decoding using the effective QM, thereby reducing the computational complexity on the decoder side.

[0234] Referring to Figure 14, it shows a block diagram of a video encoding device provided by one embodiment of the present application. The device has a function for implementing the above-mentioned example of the video encoding method, and the above function may be implemented by hardware, or by hardware executing corresponding software. The device may be the encoding side device introduced above, or may be installed on the encoding side device. The device 1400 may include a QM determination module 1410 and a QM encoding module 1420.

[0235] The QM determination module 1410 is used to determine an effective QM corresponding to the video frame to be coded, where the effective QM refers to the QM actually used when quantizing the transform coefficients in the coding process of the video frame to be coded.

[0236] The QM encoding module 1420 is configured to perform encoding on the syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set, where the first parameter set is a parameter set used to define the syntax elements related to the QM.

[0237] As described above, the technical solutions provided by the embodiments of the present application determine an effective QM corresponding to a video frame to be coded, where the effective QM refers to the QM actually used when quantizing transform coefficients in the coding process of the video frame to be coded, and then perform coding on the syntax elements used to determine the effective QM and the effective QM to generate a codestream corresponding to a first parameter set. In this way, the encoder only encodes and transmits the effective QM, thereby saving the codewords required for QM signaling and reducing bit overhead, and the decoder only needs to decode using the effective QM, thereby reducing the computational complexity of the decoder.

[0238] It should be noted that although the device provided in the above embodiments has been described using only the division of the above functional modules as an example to realize its functions, in actual applications, the functions can be completed by allocating them to different functional modules as needed. That is, the internal structure of the device can be divided into different functional modules to achieve all or part of the functions described above. Furthermore, the device provided in the above embodiments is based on the same concept as the method embodiments, and the specific implementation process thereof can be referred to in detail in the method embodiments, so it will not be described in detail again here.

[0239] 15, which shows a structural block diagram of a computer device provided by one embodiment of the present application. The computer device may be the encoding device introduced above, or the decoding device introduced above. The computer device 150 may include a processor 151, a memory 152, a communication interface 153, an encoder / decoder 154, and a bus 155.

[0240] The processor 151 includes one or more processing cores, and executes various functional applications and information processing by running software programs and modules.

[0241] The memory 152 can be used to store a computer program, and the processor 151 can be used to execute the computer program, thereby implementing the above-mentioned video encoding method or the above-mentioned video decoding method.

[0242] The communication interface 153 can be used to communicate with other devices, for example, to send and receive audio and video data.

[0243] The encoder / decoder 154 can be used to implement encoding and decoding functions, for example, to encode and decode audio and video data.

[0244] The memory 152 is coupled to the processor 151 by a bus 155 .

[0245] Additionally, memory 152 may be implemented with any type of volatile or non-volatile storage device or combination thereof, including, but not limited to, a magnetic or compact disk, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM (Erasable Programmable Read-Only Memory), an SRAM (Static Random-Access Memory), a ROM (Read-Only Memory), a magnetic memory, a flash memory, and a PROM (Programmable Read-Only Memory).

[0246] As will be appreciated by those skilled in the art, the structure shown in FIG. 15 does not constitute a limitation on computing device 150, which may include more or fewer components than shown, or may combine certain components, or may employ different components in different configurations.

[0247] In an exemplary embodiment, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, which, when executed by a processor, implements the video decoding method or the video encoding method.

[0248] In an exemplary embodiment, a computer program product is further provided, which, when executed by a processor, implements the video decoding method or is used to implement the video encoding method.

[0249] As will be understood, "plurality" as referred to herein refers to two or more than two. "And / or" describes a relationship between related objects and indicates that three types of relationships may exist, for example, A and / or B may represent three situations: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the related objects before and after it are in a type of "or" relationship.

[0250] The above are merely illustrative examples of the present application, and are not used to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should fall within the scope of protection of the present application. [Explanation of symbols]

[0251] 60 Video Encoder 62 Conversion Module 64 Quantization Module 66 Entropy Coding Module 70 Video Decoder 72 Entropy Decoding Module 74 Inverse quantization module 76 Inverse conversion module 150 Computer Equipment 151 processors 152 memory 153 Communication Interface 154 decoder 155 Bus 200 Communication Systems 210 1st device 220 2nd device 230 Third equipment 240 4th equipment 250 Network 301 Video Sources 302 Video Picture Stream 303 Video Encoder 304 Video Data 305 Streaming Server 310 Video Decoder 311 Output Video Picture Stream 312 Display 313 Collection Subsystem 1210 Parameter Acquisition Module 1220 QM Decision Module 1221 Range Determination Unit 1222 QM Decision Unit 1223 Element Reading Unit 1224 QM Judgment Unit 1230 QM Decoding Module 1410 QM Decision Module 1420 QM Encoding Module

Claims

1. 1. A video decoding method, the method comprising: obtaining a first parameter set corresponding to a video frame to be decoded, the first parameter set including syntax elements related to a quantization matrix QM, the QM having 28 identifier variables associated with different QM sizes and different QM types; determining a valid QM based on a syntax element included in the first parameter set, the syntax element being used to indicate whether a chroma QM needs to be decoded, the valid QM being a QM actually used when performing inverse quantization on quantized transform coefficients in a decoding process of the video frame to be decoded; determining a range of an identifier variable of the QM based on the syntax element included in the first parameter set, and determining a QM that falls within the range of the identifier variable of the QM as the valid QM; and performing decoding on the valid QM; The value of the syntax element is determined based on a sampling rate of the chrominance component relative to the luminance component; A video decoding method, wherein if the sampling rate of the chrominance component relative to the luma component is 4:0:0, the value of the syntax element is 0, indicating that the chrominance QM does not need to be decoded, and all elements of the chrominance QM are estimated to be 16.

2. determining a valid size range for a QM based on syntax elements included in the first parameter set; determining a minimum luma coding block size, a luma coding tree block size, and a maximum luma transform block size based on syntax elements included in the first parameter set; determining a valid size range of luma QM based on the minimum luma coding block size, the luma coding tree block size, and the maximum luma TB size, wherein the valid size range of luma QM includes the minimum size and the maximum size of the luma QM; 2. The method of claim 1, further comprising: determining a valid size range of a chrominance QM based on a valid size range of the luma QM and a sampling rate of the chrominance component relative to the luma component, wherein the valid size range of the chrominance QM includes a minimum size and a maximum size of the chrominance QM.

3. determining a valid size range for a luma QM based on the minimum luma coding block size, the luma coding tree block size, and the maximum luma TB size; determining a minimum size of the luma QM based on the minimum luma coding block size; and determining a larger value of the block size of the luma coding tree and the maximum luma TB size as the maximum size of the luma QM.

4. determining a valid size range for a QM based on syntax elements included in the first parameter set; determining a valid size range of a luminance QM based on syntax elements included in the first parameter set, the valid size range of the luminance QM including a minimum size and a maximum size of the luminance QM; 2. The method of claim 1, further comprising: determining a valid size range of a chrominance QM based on a valid size range of the luma QM and a sampling rate of the chrominance component relative to the luma component, wherein the valid size range of the chrominance QM includes a minimum size and a maximum size of the chrominance QM.

5. determining a valid size range of the chrominance QM based on a valid size range of the luma QM and a sampling rate of the chrominance component relative to the luma component, calculating a minimum size of the chrominance QM based on a minimum size of the luma QM and a sampling rate of the chrominance components relative to the luma component; 5. The method of claim 2, further comprising: calculating a maximum size of the chrominance QM based on the maximum size of the luma QM and a sampling rate of the chrominance components relative to the luma component.

6. The step of determining a QM that falls within the valid size range as the valid QM comprises: determining the first QM as the valid QM if the first QM satisfies one of a first condition and a second condition; The first condition is cIdx==0 && (matrixQMSize>=minQMSizeY && matrixQMSize<=maxQMSizeY), which indicates that the first QM belongs to a luma component used in the quantization process of a luma TB, and the first QM is within a valid size range of the luma QM [MinQMSizeY, MaxQMSizeY], where MinQMSizeY represents the minimum size of the luma QM and MaxQMSizeY represents the maximum size of the luma QM; 5. The method of claim 2 or 4, wherein the second condition is cIdx!=0 && (matrixQMSize>=minQMSizeUV && matrixQMSize<=maxQMSizeUV), wherein the second condition indicates that the first QM belongs to a chrominance component used in a quantization process of a chrominance TB, and the first QM is within a valid size range of the chrominance QM [MinQMSizeUV, MaxQMSizeUV], where MinQMSizeUV represents the minimum size of the chrominance QM and MaxQMSizeUV represents the maximum size of the chrominance QM.

7. The step of determining a valid QM based on syntax elements included in the first parameter set comprises: reading a value of a flags syntax element corresponding to a first QM from the first parameter set; determining that the first QM belongs to the valid QMs if the value of a flag syntax element corresponding to the first QM is a first numeric value; and determining that the first QM does not belong to the valid QMs if a value of a flag syntax element corresponding to the first QM is a second numeric value.

8. The method of claim 7 , wherein the first chroma QM and the second chroma QM having the same prediction mode and the same size share the same flag syntax element.

9. 8. The method of claim 7, wherein the flag syntax element is a scaling_matrix_present_flag.

10. The method of claim 1 , wherein the first parameter set is a self-adaptive parameter set APS.

11. The method of claim 1 , wherein all elements of other QMs that do not belong to the valid QMs are predefined to be default values.

12. The method of claim 11 , wherein the default value is 16.

13. 1. A video encoding method, the method comprising: determining an effective quantization matrix QM corresponding to a video frame to be coded, the effective QM being the QM actually used when quantizing transform coefficients; generating a codestream corresponding to a first parameter set by performing encoding on syntax elements used to determine the valid QMs and the valid QMs, wherein the first parameter set includes syntax elements related to QMs, the QMs having 28 identifier variables associated with different QM sizes and different QM types, the syntax elements being used to indicate whether a chroma QM needs to be decoded, and the valid QMs are determined by determining a range of the identifier variable of the QM based on the syntax elements included in the first parameter set, and determining a QM that falls within the range of the identifier variable of the QM as the valid QM; The value of the syntax element is determined based on a sampling rate of the chrominance component relative to the luminance component; A video encoding method, wherein if the sampling rate of the chrominance component relative to the luma component is 4:0:0, the value of the syntax element is 0, indicating that the chrominance QM does not need to be decoded, and all elements of the chrominance QM are estimated to be 16.

14. 1. A video decoding apparatus, comprising: a parameter obtaining module; a QM determining module; and a QM decoding module; the parameter acquisition module is used to acquire a first parameter set corresponding to a video frame to be decoded, the first parameter set including a syntax element related to a quantization matrix QM, the QM having 28 identifier variables respectively associated with different QM sizes and different QM types; the QM determination module is used to determine a valid QM based on a syntax element included in the first parameter set, the syntax element being used to indicate whether a chroma QM needs to be decoded, the valid QM being a QM actually used when performing inverse quantization on quantized transform coefficients in a decoding process of the video frame to be decoded; a range of an identifier variable of a QM is determined based on an element included in the first parameter set, and a QM falling within the range of the identifier variable of the QM is determined as the valid QM; the QM decoding module is used to perform decoding on the valid QM; The value of the syntax element is determined based on a sampling rate of the chrominance component relative to the luminance component; A video decoding device, wherein when the sampling rate of the chrominance component relative to the luma component is 4:0:0, the value of the syntax element is 0, indicating that the chrominance QM does not need to be decoded, and all elements of the chrominance QM are estimated to be 16.

15. A computing device comprising a processor and a memory, wherein at least one program, set of code or set of instructions is stored in the memory, and wherein the at least one program, set of code or set of instructions is loaded and executed by the processor to implement a method according to any one of claims 1 to 12.

16. 14. A computing device comprising a processor and a memory, wherein at least one program, set of code or set of instructions is stored in the memory, and wherein the at least one program, set of code or set of instructions is loaded and executed by the processor to implement the method of claim 13.

17. 13. A computer readable storage medium having stored thereon at least one program, set of code or set of instructions, said at least one program, set of code or set of instructions being loaded and executed by a processor to implement the method of any one of claims 1 to 12.

18. 14. A computer-readable storage medium having stored thereon at least one program, set of code or set of instructions, the at least one program, set of code or set of instructions being loaded and executed by a processor to implement the method of claim 13.

Images