Video decoding method, video encoding method, device, apparatus, and storage medium

Abstract

To provide a video decoding method, a video encoding method, a device, an apparatus, and a storage medium, and the application relates to the technical field of video encoding and decoding processing.SOLUTION: A decoding method comprises: acquiring a first parameter set corresponding to a video frame to be decoded; based on syntax elements included in the first parameter set, determining an effective QM, where the effective QM indicates a QM actually used when inverse quantization is carried out on a quantized transform coefficient in a decoding process of the video frame to be decoded; and decoding the effective QM. By adopting technical means of the present application, a decoder side only needs to decode the effective QM, thereby reducing computational complexity of the decoder side.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The embodiments of the present application relate to the technical field of video encoding and decoding, and in particular to a video decoding method, a video encoding method, an apparatus, a device, and a storage medium.

[0002] This application claims priority to a Chinese patent application filed on December 18, 2019, bearing application number 201911309768.6, and entitled "Video Decoding Method, Apparatus, Device, and Storage Medium," the entire contents of which are incorporated herein by reference. [Background technology]

[0003] H.266 is a next-generation video coding technology improved on H.265 / HEVC (High Efficient Video Coding), and has already been officially named VVC (Versatile Video Coding). It is continuously updated and developed under the guidance of the JVET (Joint Video Experts Team) organization.

[0004] At the 14th JVET meeting, it was decided that frequency-related scaling can be supported in VVC by using two types of quantization matrices: default quantization matrix and user-defined quantization matrix. When a quantization matrix is ​​enabled, independent quantization can be performed on the transform coefficients in a TB (Transform Block) based on the quantization coefficients (i.e., integer weighting values) contained in the quantization matrix.

[0005] Currently, the quantization matrix decoding method adopted in VVC has a relatively high degree of computational complexity on the decoder side. Summary of the Invention [Problem to be solved by the invention]

[0006] The embodiments of the present application provide a video decoding method, a video encoding method, an apparatus, a device and a storage medium, which can reduce the computational complexity of the decoder side. [Means for solving the problem]

[0007] In one aspect, embodiments of the present application provide a video decoding method, the method comprising: obtaining a first parameter set corresponding to a video frame to be decoded, the first parameter set including a parameter set used to define a syntax element related to a quantization matrix (QM); determining an effective QM based on syntax elements included in the first parameter set, the effective QM being a QM actually used when performing inverse quantization on quantized transform coefficients in a decoding process of the video frame to be decoded; and performing decoding on the valid QM.

[0008] In another aspect, embodiments of the present application provide a video encoding method, the method comprising: determining an effective QM corresponding to a video frame to be coded, the effective QM being a QM actually used when quantizing transform coefficients in a coding process of the video frame to be coded; and performing encoding on syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set, the first parameter set including a parameter set used to define syntax elements associated with a QM.

[0009] In yet another aspect, an embodiment of the present application provides a video decoding device, the device including: a parameter acquisition module; a QM determination module; and a QM decoding module; The parameter acquisition module is adapted to acquire a first parameter set corresponding to a video frame to be decoded, the first parameter set including a parameter set used to define a syntax element related to QM; The QM determination module is used to determine an effective QM based on a syntax element included in the first parameter set, and the effective QM refers to a QM actually used when performing inverse quantization on a quantized transform coefficient in a decoding process of the video frame to be decoded; The QM decoding module is used to perform decoding on the valid QM.

[0010] In yet another aspect, an embodiment of the present application provides a video encoding device, the device including: a QM determination module; and a QM encoding module; The QM determination module is used for determining an effective QM corresponding to a video frame to be encoded, and the effective QM refers to a QM actually used when quantizing a transform coefficient in an encoding process of the video frame to be encoded; The QM encoding module is used to perform encoding on syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set, the first parameter set including a parameter set used to define syntax elements associated with a QM.

[0011] In yet another aspect, an embodiment of the present application provides a computer device, the computer device including a processor and a memory, wherein at least one instruction, at least one program, code set or instruction set is stored in the memory, the at least one instruction, the at least one program, code set or instruction set being loaded and executed by the processor to implement the video decoding method or to implement the video encoding method.

[0012] In yet another aspect, an embodiment of the present application provides a computer-readable storage medium having stored thereon at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, code set or instruction set being loaded and executed by a processor to implement the video decoding method or to implement the video encoding method.

[0013] In yet another aspect, embodiments of the present application provide a computer program product, which, when executed by a processor, is used to implement the above video decoding method or to implement the above video encoding method. Effect of the Invention

[0014] The technical means provided by the embodiments of the present application may include the following beneficial effects.

[0015] By obtaining a first parameter set corresponding to the video frame to be decoded, a valid QM is determined according to the syntax elements included in the first parameter set, the valid QM refers to the QM actually used when quantizing the transform coefficients in the process of encoding and generating the video frame to be decoded, and then decoding is performed according to the valid QM. In this way, the decoder side only needs to decode according to the valid QM, thereby reducing the computational complexity of the decoder side. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a schematic diagram of video encoding as exemplarily shown in the present application; [Diagram 2] 1 is a simplified block diagram of a communication system provided by one embodiment of the present application. [Diagram 3] FIG. 2 is a schematic diagram of an arrangement of a video encoder and a video decoder in a streaming environment as exemplarily shown in the present application; [Figure 4]FIG. 2 is a schematic diagram of an encoding process under an inter-frame prediction mode provided by an embodiment of the present application. [Diagram 5] FIG. 2 is a schematic diagram of an encoding process under an intra-frame prediction mode provided by an embodiment of the present application. [Figure 6] FIG. 2 is a schematic diagram of functional modules of a video encoder provided by one embodiment of the present application. [Figure 7] FIG. 2 is a schematic diagram of functional modules of a video decoder provided by one embodiment of the present application. [Figure 8] FIG. 1 is a schematic diagram of generating a QM by downsampling copy provided by one embodiment of the present application. [Figure 9] FIG. 2 is a schematic diagram of a diagonal scan order provided by one embodiment of the present application. [Figure 10] 1 is a flowchart of a video decoding method provided by one embodiment of the present application. [Figure 11] 1 is a flowchart of a video encoding method provided by one embodiment of the present application. [Figure 12] 1 is a block diagram of a video decoding device provided by one embodiment of the present application. [Figure 13] 2 is a block diagram of a video decoding device provided by another embodiment of the present application; [Figure 14] 1 is a block diagram of a video encoding device provided by one embodiment of the present application; [Figure 15] FIG. 2 is a structural block diagram of a computer device provided by one embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] In order to make the objectives, technical means and advantages of the present application clearer, the embodiments of the present application will be described in more detail below with reference to the accompanying drawings.

[0018] As shown in Fig. 1, the current block 101 contains samples already found by the encoder during the motion estimation process, and the samples can be predicted based on previous blocks of the same size with spatial offsets. The MV (Motion Vector) can also be derived from metadata associated with one or more reference pictures, rather than directly encoding the MV. For example, the MV associated with any one of five surrounding samples A0, A1, B0, B1, and B2 (corresponding to 102 to 106, respectively) is used, and the MV is derived from metadata of the closest reference picture (according to the decoding order).

[0019] As shown in FIG. 2, it shows a simplified block diagram of a communication system provided by one embodiment of the present application. The communication system 200 includes a plurality of devices, which can communicate with each other, for example, through a network 250. For example, the communication system 200 includes a first device 210 and a second device 220 connected to each other through the network 250. In the embodiment of FIG. 2, the first device 210 and the second device 220 perform one-way data transmission. For example, the first device 210 can perform encoding on video data, for example, a video picture stream collected by the first device 210, and transmit the video data to the second device 220 through the network 250. The encoded video data is transmitted in the form of one or more encoded video code streams. The second device 220 can receive the encoded video data from the network 250, perform decoding on the encoded video data to recover the video data, and display the video pictures based on the recovered video data. One-way data transmission is common in applications such as media services.

[0020] In another embodiment, the communication system 200 includes a third device 230 and a fourth device 240 performing bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For the bidirectional data transmission, each of the third device 230 and the fourth device 240 may perform encoding on video data (for example, a video picture stream collected by the device) and transmit it to the other of the third device 230 and the fourth device 240 via the network 250. Each of the third device 230 and the fourth device 240 may further receive the encoded video data transmitted by the other of the third device 230 and the fourth device 240, perform decoding on the encoded video data to recover the video data, and display the video pictures on an accessible display device based on the recovered video data.

[0021] In the embodiment of FIG. 2, the first device 210, the second device 220, the third device 230, and the fourth device 240 may be computer devices such as servers, personal computers, and smartphones, but the principles disclosed herein are not limited thereto. The embodiment of the present application applies to a PC (Personal Computer), a mobile phone, a tablet computer, a media player, and / or a dedicated video conferencing device. The network 250 represents any number of networks that transmit encoded video data between the first device 210, the second device 220, the third device 230, and the fourth device 240, including, for example, wired and / or wireless communication networks. The communication network 250 can exchange data in circuit-switched and / or packet-switched channels. The network may include an electronic communication network, a local area network, a wide area network, and / or the Internet. For the purposes of the present application, unless otherwise interpreted below, the architecture and topology of the network 250 may be irrelevant to the operation disclosed herein.

[0022] As an example, Figure 3 illustrates a configuration of a video encoder and a video decoder in a streaming environment. The subject matter disclosed herein is equally applicable to other applications that support video, including video conferencing, digital TV, and storing compressed video on digital media, including Compact Discs (CDs), Digital Versatile Discs (DVDs), memory sticks, etc.

[0023] The streaming system may include a collection subsystem 313, which may include a video source 301, such as a digital camera, that creates an uncompressed video picture stream 302. In an embodiment, the video picture stream 302 includes samples taken by a digital camera. Compared to the encoded video data 304 (or encoded video code stream), the video picture stream 302 is drawn as a thick line to emphasize the high amount of data of the video picture stream, which may be processed by an electronic device 320. The electronic device 320 includes a video encoder 303 coupled to the video source 301. The video encoder 303 may include hardware, software, or a combination of software and hardware to realize or implement aspects of the disclosed subject matter, which are described in more detail below. Compared to the video picture stream 302, the encoded video data 304 (or encoded video code stream 304) is drawn as a thin line to emphasize the relatively low amount of data of the encoded video data 304 (or encoded video code stream 304), which may be stored in a streaming server 305 for future use. One or more streaming client terminal subsystems, such as client terminal subsystem 306 and client terminal subsystem 308 in FIG. 3, can access the streaming server 305 to retrieve duplicates 307 and 309 of the encoded video data 304. The client terminal subsystem 306 may include a video decoder 310, for example in an electronic device 330. The video decoder 310 performs decoding on the derived duplicates 307 of the encoded video data and produces an output video picture stream 311 that can be presented on a display 312 (e.g., a display screen) or another presentation device (not shown). In some streaming systems, the encoding of the encoded video data 304, video data 307, and video data 309 (e.g., a video code stream) can be performed according to some video encoding / compression standard.

[0024] It should be noted that electronic device 320 and electronic device 330 may include other components (not shown). For example, electronic device 320 may include a video decoder (not shown), and electronic device 330 may further include a video encoder (not shown). The video decoder is used to perform decoding on the received encoded video data, and the video encoder is used to perform encoding on the video data.

[0025] When coding an image block in a video frame, an inter-frame prediction mode or an intra-frame prediction mode can be used, and a prediction block is generated based on one or more coded reference blocks. The prediction block may be an estimated version of an original block. A residual block can be generated by subtracting the original block from the prediction block, or vice versa, and the residual block can be used to represent a prediction residual (also called a prediction error). A relatively high compression ratio can be achieved by coding the residual block, since the amount of data required to represent the prediction residual can usually be less than the amount of data required to represent the original block. For example, as shown in FIG. 4, for an inter-frame prediction mode, the coded reference block 41 and the block to be coded 42 are located in two different video frames. As shown in FIG. 5, for an intra-frame prediction mode, the coded reference block 51 and the block to be coded 52 are located in the same video frame.

[0026] Then, the residual values ​​of the residual block in the spatial domain can be transformed into transform coefficients in the frequency domain. The transform can be realized by a two-dimensional transform similar to, for example, Discrete Cosine Transform (DCT). In the transform matrix, low index transform coefficients (e.g., located in the upper left region) can correspond to large spatial features and have relatively large quantity values, and high index transform coefficients (e.g., located in the lower right region) can correspond to small spatial features and have relatively small quantity values. Furthermore, a quantization matrix including quantization coefficients can be applied to the transform matrix, thereby quantizing all transform coefficients into quantized transform coefficients. As a result of quantization, the scale or quantity value of the transform coefficients can be low. Some high index transform coefficients can be reduced to zero and then skipped in the subsequent scanning and encoding steps.

[0027] FIG. 6 shows a part of an exemplary video encoder 60 including a transform module 62, a quantization module 64, and an entropy coding module 66. Although not shown in FIG. 6, it should be understood that other modules may be included in the video encoder 60, such as a prediction module, a dequantization module, and a reconstruction module. In operation, the video encoder 60 may obtain a video frame, which may include a number of image blocks. For the sake of brevity, coding for one image block may be taken as an example here. To code for an image block, a prediction block may be generated first to be an estimate of the image block. In conjunction with the above, the prediction block may be generated by the prediction module through an inter-frame prediction or intra-frame prediction mode. Then, a difference value between the image block and the prediction block may be calculated to generate a residual block. The residual block may be transformed into a transform coefficient by the transform module 62. During the transformation, the residual value in the spatial domain includes large features and small features, and is transformed into a transform coefficient in the frequency domain, which includes a high-frequency band and a low-frequency band. A quantization module 64 may then quantize the transform coefficients using QM, thereby generating quantized transform coefficients, which may then be coded by an entropy coding module 66 and ultimately transmitted from the video encoder 60 as part of a bitstream.

[0028] FIG. 7 shows a part of an exemplary video decoder 70 including an entropy decoding module 72, an inverse quantization (dequantization) module 74, and an inverse transform module 76. Although not shown in FIG. 7, it should be understood that other modules may be included in the video decoder 70, such as a prediction module, a transform module, and a quantization module. In operation, the video decoder 70 can receive a bitstream output from the video encoder 60, perform decoding on the bitstream according to an inter-frame prediction or intra-frame prediction mode, and output a reconstructed video frame. Here, the entropy decoding module 72 can generate quantized transform coefficients by performing entropy decoding on the input bitstream. The inverse quantization module 74 can perform inverse quantization on the quantized transform coefficients based on the QM to obtain inverse quantized transform coefficients. The inverse transform module 76 performs inverse transform on the inverse quantized transform coefficients to generate a reconstructed residual block. Then, a reconstructed image block is generated based on the reconstructed residual block and the prediction block.

[0029] As can be seen from the above, QM is an essential part of the video encoding and decoding process. The setting of QM can determine how much information of the transform coefficients is to be withheld or filtered, and thus QM can affect the encoding performance and encoding quality. In fact, both the encoder and the decoder require QM. Specifically, in order to accurately decode an image, the encoder needs to encode information about the quantized coefficients in QM and transmit the information from the encoder to the decoder. In video encoding and decoding techniques and standards, QM may be referred to as a scaling matrix or weighting matrix. Thus, the term "QM" used in this specification may be a general term covering quantization matrix, scaling matrix, weighting matrix, and other equivalent terms.

[0030] In the following, some basic concepts related to the embodiments of the present application are introduced and explained.

[0031] 1. Quantization matrix

[0032] In the latest version of VTM (VVC Test Model, VTM7), it not only allows square TBs but also allows non-square TBs, so the number of QMs is relatively large. In order to reduce the number of QM signaling orders and memory needs, VVC adopts upsampling and copying design for non-square TBs and large square TBs.

[0033] Non-square QMs do not exist in the VVC bitstream, and they are obtained by copying the corresponding square QMs at the decoder side. More specifically, the 32×4 QM is obtained by copying rows 0, 8, 16, and 24 of the 32×32 QM. As shown in Figure 8, the 32×4 QM is obtained by downsampling the 32×32 QM. The shaded rows 0, 8, 16, and 24 are copied from the 32×32 QM to the 32×4 QM.

[0034] When the size of the square TB is larger than 8×8, the size of the corresponding QM in VTM7 is constrained to 8×8. For these 8×8 QMs, an upsampling method is adopted to generate 16×16, 32×32 and 64×64 QMs. More specifically, to generate a 16×16 size QM, each element in the corresponding 8×8 size QM is upsampled and copied to a 2×2 area, and to generate a 32×32 size QM, each element in the corresponding 8×8 size QM is upsampled and copied to a 4×4 area.

[0035] In VTM7, as many as 28 QMs need to be coded. Table 1 determines the identifier variable (id) of the QM based on the variables sizeId and matrixId specified in Tables 2 and 3, respectively, where sizeId represents the size of the quantization matrix, and matrixId is an identifier of the type of QM based on the prediction mode (predMode) and color components (cIdx).

[0036] [Table 1]

[0037] [Table 2]

[0038] [Table 3]

[0039] In Table 2, when sizeId is greater than 3, it has a DC (Direct Current) coefficient, and the DC coefficient is the element value at the (0,0) position in the QM. In VVC, when the DC value is 0, the QM may use the default QM, but the QM may still be transmitted. The main reason is that uncoded QM may need to refer to the QM. When the DC value is not 0, the QM uses a user-defined QM, and is transmitted after encoding using the encoding method introduced below.

[0040] In Table 3, MODE_INTRA represents an intra-frame prediction mode, MODE_INTER represents an inter-frame prediction mode, and MODE_IBC represents an IBC (Intra Block Copy) prediction mode. Y represents luminance, and Cb and Cr represent chrominance.

[0041] 2. Quantization matrix coding method

[0042] To reduce bit overhead, VTM7 employs intra-frame and inter-frame predictive coding to encode for 28 QMs.

[0043] In intra prediction mode, DPCM (Differential Pulse Code Modulation) coding is applied to the QM in a diagonal scan order. The DPCM intra residual also needs to be transmitted to the bitstream. For example, as shown in FIG. 9, taking a 4×4 size QM as an example, the diagonal scan order is (0,0), (1,0), (0,1), (2,0), (1,1), ..., (2,3), (3,3).

[0044] There are two kinds of interframe prediction modes: copy mode and predictive mode. In copy mode, the current QM to be encoded is exactly the same as one available for decoding, called reference QM. This also means that copy mode has zero interframe residual, and of course there is no need to signal the residual. The encoder needs to transmit an incremental ID between the current QM and its reference QM, so that the decoder reconstructs the current QM by directly copying the reference QM. Predictive mode is similar to copy mode, but with an additional interframe residual. DPCM coding is applied to the interframe residual in diagonal scan order, and the encoder needs to transmit the DPCM interframe residual in the bitstream.

[0045] As mentioned above, when the sizeId of a QM is larger than 3, an upsampling algorithm is applied to copy each element in the QM to a large square region. Because the DC coefficient at (0,0) position is the most important for video reconstruction, VTM7 directly encodes it instead of copying from the corresponding element of other QMs. For each QM, a mode decision is used to calculate the bit cost of three candidate modes of the QM (i.e., copy mode of inter prediction mode, prediction mode of inter prediction mode, and intra prediction mode), and the one with the smallest bit cost is selected as the final optimal mode. Then, the optimal mode is adopted to encode the QM.

[0046] 3. Quantization Matrix Signaling

[0047] By using QM, VVC supports frequency-related quantization of transform blocks. Suppose QM is W, and W[x][y] represents the QM weight of the transform coefficient at position (x,y) in TB. For the transform coefficient coeff[x][y], the following Equation 1 is adopted to calculate the quantized transform coefficient level[x][y].

[0048]

number

[0049] Here, QP is a quantization parameter (also called a quantization stride), and offset is an offset value. W[x][y]=16 indicates that no weighting is applied to the transform coefficient at position (x, y). Also, when all the element values ​​in QM are equal to 16, it is the same as the effect of not using QM.

[0050] The SPS (Sequence Parameter Set) syntax element sps_scaling_list_enable_flag is used to indicate whether to enable QM for those images whose Picture Header (PH) already references the SPS. When this flag is enabled, i.e., sps_scaling_list_enable_flag is enabled, an additional flag in the PH is used to control whether to use the default QM, which is equal to 16 for all elements, or to use a user-defined QM. In VTM7, the user-defined QM is signaled in the APS (Adaptive Parameter Set). If the user-defined QM is enabled in the SPS and PH, an APS index can be sent in the PH, which is used to specify the QM set for images that reference this PH.

[0051] In one APS, 28 groups of QM coding modes, Δid (increment id), AC, and DC coefficients should be notified. In each APS, the 28 groups of QMs are coded and decoded according to the increasing order of id.

[0052] In VVC Draft 7, the definitions of the QM coding mode, Δid (increment id), syntaxes and semantics of AC and DC coefficients are shown in Table 4 below.

[0053] [Table 4]

[0054] scaling_list_copy_mode_flag[id] equal to 1 indicates that the element values ​​of the current QM and its reference QM are the same. The reference QM is represented by scaling_list_pred_id_delta[id]. scaling_list_copy_mode_flag[id] equal to 0 indicates that scaling_list_pred_mode_flag is present.

[0055] scaling_list_pred_mode_flag[ id ] equal to 1 indicates that the current QM can be predicted from a reference QM. The reference QM is represented by scaling_list_pred_id_delta[ id ]. scaling_list_pred_mode_flag[ id ] equal to 0 indicates that an explicit signal is sent to inform the element value of the current QM. When not present, the value of scaling_list_pred_mode_flag[ id ] is inferred to be equal to 0.

[0056] scaling_list_pred_id_delta[id] represents the reference QM used to infer the predicted QM, i.e., ScalingMatrixPred[id]. When not present, the value of scaling_list_pred_id_delta[id] is inferred to be equal to 0. The value of scaling_list_pred_id_delta[id] should be in the range of 0 to maxIdDelta, where maxIdDelta is inferred based on id, as shown in Equation 2 below.

[0057]

number

[0058] That is, if id<2 then maxIdDelta=id, if id>=2 and <8 then maxIdDelta=id-2, if id>=8 then maxIdDelta=id-8.

[0059] The variables refId and matrixSize are calculated using the following formulas:

[0060] refId=id- scaling_list_pred_id_delta[ id ] Formula 3

number

[0061] That is, if id<2 then matrixSize=2, if id≧2 and <8 then matrixSize=4, if id≧8 then matrixSize=8.

[0062] The QM prediction matrix of matrixSize×matrixSize is denoted as ScalingMatrixPred[x][y], where x∈[0,matrixSize−1], y∈[0,matrixSize−1], and the variable ScalingMatrixDCPred is denoted as the predicted value of DC, which is specifically calculated as follows:

[0063] When scaling_list_copy_mode_flag[id] and scaling_list_pred_mode_flag[id] are both equal to 0, all elements of ScalingMatrixPred are set to 8, and the value of ScalingMatrixDCPred is set to 8.

[0064] Otherwise, when scaling_list_pred_id_delta[id] is equal to 0, set all elements of ScalingMatrixPred to be equal to 16, and set the value of ScalingMatrixDCPred to be equal to 16.

[0065] Otherwise, when scaling_list_copy_mode_flag[ id ] or scaling_list_pred_mode_flag[ id ] is equal to 1 and scaling_list_pred_id_delta[ id ] is greater than 0, then ScalingMatrixPred is set equal to ScalingMatrixPred[refId], and the calculation of the value of ScalingMatrixDCPred is as follows: if refId is greater than 13, then the value of ScalingMatrixDCPred is set equal to ScalingMatrixDCRec[ refId - 14 ], otherwise (i.e., refId is less than or equal to 13), the value of ScalingMatrixDCPred is set equal to ScalingMatrixPred[0][0].

[0066] scaling_list_dc_coef[id-14] is used to calculate the value of the variable ScalingMatrixDC[id-14] when id is greater than 13, and is shown in Equation 5 below.

[0067] ScalingMatrixDCRec[ id - 14 ] = ( ScalingMatrixDCPred + scaling_list_dc_coef[ id - 14 ] + 256 ) % 256 ) Formula 5

[0068] Here, % indicates that the remainder is to be found.

[0069] When not present, the value of scaling_list_dc_coef[ id - 14 ] is inferred to be equal to 0. The value of scaling_list_dc_coef[ id - 14 ] should be in the range of -128 to 127, inclusive. The value of ScalingMatrixDCRec[ id - 14 ] should be greater than 0.

[0070] scaling_list_delta_coef[id][i] represents the difference between the current matrix coefficient ScalingList[id][i] and the previous matrix coefficient ScalingList[id][i-1] when scaling_list_copy_mode_flag[id] is equal to 0. The value of scaling_list_delta_coef[id][i] should be within the range of -128 to 127 (including -128 and 127). When scaling_list_copy_mode_flag[id] is equal to 1, all elements of ScalingList[id] are set to 0.

[0071] The ScalingMatrixRec[id] of the QM of matrixSize×matrixSize can be calculated using the following formula 6.

[0072] ScalingMatrixRec[ id ][ x ][ y ] = ( ScalingMatrixPred[ x ][ y ] + ScalingList[ id ][ k ] + 256 ) % 256 ) Formula 6

[0073] Here, % represents the complement and k∈[0,(matrixSize×matrixSize-1)].

[0074] x= DiagScanOrder[ Log2( matrixSize ) ][ Log2( matrixSize ) ][ k ]

[0000] , and y = DiagScanOrder[ Log2( matrixSize ) ][ Log2( matrixSize ) ][ k ]

[0001] .

[0075] The value of ScalingMatrixRec[ id ][ x ][ y ] should be greater than 0.

[0076] Consider the decoding process of one QM, that is, the process of decoding based on the above syntax elements to obtain ScalingMatrixRec[id][x][y] and ScalingMatrixDCRec.

[0077] 4. Limit TB size with SPS

[0078] In VVC Draft 7, the definition of SPS syntax and semantics related to TB size constraints is shown in Table 5 below.

[0079] [Table 5]

[0080] sps_max_luma_transform_size_64_flag equal to 1 indicates that the maximum transform block size for luma sampling is equal to 64. sps_max_luma_transform_size_64_flag equal to 0 indicates that the maximum transform block size for luma sampling is equal to 32.

[0081] chroma_format_idc represents the chrominance sampling corresponding to the luma sampling and is shown in Table 6.

[0082] [Table 6]

[0083] In Table 6 above, SubWidthC and SubHeightC respectively represent the width and height of a coding tree unit (CTU) corresponding to the chrominance component, and Monochrome represents the absence of a chrominance component.

[0084] separate_colour_plane_flag equal to 1 indicates coding of each of the three color components of the 4:4:4 chroma format. separate_colour_plane_flag equal to 0 indicates not coding a color component separately. When separate_colour_plane_flag is not present, its value is inferred to be equal to 0.

[0085] When separate_colour_plane_flag is equal to 1, the coded image consists of three separate components, each consisting of coded samples of one colour plane (Y, Cb or Cr), and uses the monochrome coding syntax, where each colour plane is associated with a specific colour_plane_id value.

[0086] colour_plane_id specifies the colour plane associated with the slice associated with PH and when separate_colour_plane_flag is equal to 1, the value of colour_plane_id should be in the range 0 to 2 (inclusive). The colour_plane_id values ​​0, 1 and 2 correspond to the Y, Cb and Cr planes respectively. It should be noted that there is no dependency between the decoding process of images with different colour_plane_id values.

[0087] sps_log2_ctu_size_minus5+5 represents the size of the luma coding tree block of each CTU. The value of sps_log2_ctu_size_minus5 must be 2 or less to ensure bitstream consistency.

[0088] Based on sps_log2_ctu_size_minus5, the maximum luma coding block size can be calculated.

[0089] CtbLog2SizeY = sps_log2_ctu_size_minus5 + 5 CtbSizeY = 1 << CtbLog2SizeY

[0090] where CtbSizeY represents the maximum luma coding block size, CtbLog2SizeY represents the base 2 logarithm of CtbSizeY, and << is the left shift operator.

[0091] log2_min_luma_coding_block_size_minus2+2 represents the minimum luma coding block size. The value range of log2_min_luma_coding_block_size_minus2 should be within the range of 0 to sps_log2_ctu_size_minus5+3 (including 0 and sps_log2_ctu_size_minus5+3).

[0092] The calculation process of the variables MinCbLog2SizeY, MinCbSizeY and VSize is as follows:

[0093] MinCbLog2SizeY = log2_min_luma_coding_block_size_minus2 + 2 Formula 7 MinCbSizeY = 1 << MinCbLog2SizeY Formula 8 VSize = Min( 64, CtbSizeY ) Formula 9

[0094] where MinCbSizeY represents the minimum luma coding block size, MinCbLog2SizeY represents the logarithm of MinCbSizeY to the base 2, VSize represents the maximum luma coding block size, and << is the left shift operator. The value of MinCbSizeY should be less than or equal to VSize.

[0095] The width and height of each chrominance CTB (Coding Tree Block), that is, the variables CtbWidthC and CtbHeightC, are determined by employing the following method.

[0096] If chroma_format_idc is equal to 0 (monochrome) or Separate_color_Plane_flag is equal to 1, then CtbWidthC and CtbHeightC are both equal to 0.

[0097] Otherwise, CtbWidthC and CtbHeightC are calculated using the following formulas:

[0098] CtbWidthC = CtbSizeY / SubWidthC Formula 10 CtbHeightC = CtbSizeY / SubHeightC Formula 11

[0099] Here, CtbSizeY represents the size of the luminance CTB.

[0100] At present, the coding method for the quantization matrix adopted by VVC requires all 28 QMs to be coded and transmitted in APS, so that QM signaling needs to occupy a relatively large number of codewords, which causes large bit overhead and increases the computational complexity of the decoder side. In the technical means provided by the embodiment of the present application, a first parameter set corresponding to the video frame to be decoded is obtained, and an effective QM is determined according to the syntax elements contained in the first parameter set, and the effective QM refers to the QM actually used when quantizing the transform coefficients in the process of coding and generating the video frame to be decoded, and then decoding is performed for the effective QM. In this way, the encoder side only codes and transmits the effective QM, which contributes to saving the codewords that QM signaling needs to occupy, and reduces the bit overhead, and the decoder side only needs to decode for the effective QM, thereby reducing the computational complexity of the decoder side.

[0101] It is necessary to explain that the technical means provided in the embodiments of the present application can be applied to the H.266 / VCC standard or next-generation video encoding and decoding standards, but the embodiments of the present application are not limited thereto.

[0102] It is further to be noted that in the video decoding method provided by the embodiment of the present application, each step is executed by a decoding device, while in the video encoding method provided by the embodiment of the present application, each step is executed by an encoding device, and both the decoding device and the encoding device can be computer devices, which refer to electronic devices with data calculation, processing and storage capabilities, such as PCs, mobile phones, tablet computers, media players, dedicated video conferencing devices, or servers.

[0103] Moreover, the methods provided herein may be used alone or in combination with other methods in any order. The encoder and decoder based on the methods provided herein may be implemented by one or more processors or one or more integrated circuits. The technical solutions of the present application are introduced and described below through several embodiments.

[0104] Referring to Figure 10, it shows a flow chart of a video decoding method provided in one embodiment of the present application. In this embodiment, the method is mainly applied to the decoding side device introduced above as an example. The method may include the following steps (1001 to 1003).

[0105] Step 1001: Obtain a first parameter set corresponding to a video frame to be decoded.

[0106] The video frame to be decoded may be any one video frame (also called image frame) to be decoded in the video to be decoded. The first parameter set includes a parameter set used to define a syntax element related to QM, for example, the decoding device can obtain QM by decoding based on the syntax element in the first parameter set.

[0107] Optionally, the first parameter set is APS. Of course, in some other embodiments, the first parameter set may not be APS, and may even be SPS, etc., and the embodiments of the present application are not limited thereto.

[0108] Step 1002: According to the syntax elements included in the first parameter set, determine an effective QM, which refers to a QM actually used when performing inverse quantization on the quantized transform coefficients in the decoding process of the video frame to be decoded.

[0109] Assuming that the number of QMs that may be used when performing inverse quantization on the quantized transform coefficients is n, the number of effective QMs may be less than or equal to n, where n is a positive integer. For example, if all n QMs are actually used when performing inverse quantization on the quantized transform coefficients, the number of effective QMs is n, and if some QMs (e.g., m QMs, where m is a positive integer less than n) out of the total n QMs are actually used when performing inverse quantization on the quantized transform coefficients, the number of effective QMs is m.

[0110] By defining syntax elements used to determine a valid QM in the first parameter set, the decoding device can determine which QM is valid and which is not valid by reading the syntax elements. For a QM that is not a valid QM (which may also be referred to as an invalid QM), i.e., a QM that is not actually used when quantizing a transform coefficient in the process of encoding and generating a video frame to be decoded, the decoding device may not need to decode it.

[0111] Optionally, for other QMs that do not belong to the valid QM, predefine all its elements as a default value, optionally the default value is 16, referring to Equation 1, at this time, the stretch quantization coefficients of all transform coefficients in TB are all 1, so the effect is the same as not using QM.

[0112] Step 1003: Decoding is performed on the valid QM.

[0113] After determining the valid QM, the number of valid QMs may be one or more, so the decoding device needs to perform decoding for each valid QM. Take any valid QM as an example, when decoding the valid QM, the coding mode corresponding to the valid QM can be determined, and then the valid QM is decoded according to the coding mode.

[0114] For example, referring to Table 1 above, assuming that it is determined that the number of QMs that can be used when quantizing transform coefficients is 28, of which 12 are valid QMs, the decoding device only needs to decode the 12 valid QMs and does not need to decode the remaining 16 invalid QMs.

[0115] As described above, in the technical means provided by the embodiments of the present application, a first parameter set corresponding to a video frame to be decoded is obtained, and an effective QM is determined according to the syntax elements included in the first parameter set. The effective QM refers to the QM actually used when quantizing transform coefficients in the process of encoding and generating the video frame to be decoded, and then decoding is performed according to the effective QM. In this way, the decoder side only needs to perform decoding according to the effective QM, thereby reducing the computational complexity of the decoder side.

[0116] In an exemplary embodiment, the step of determining a valid QM based on syntax elements included in the first parameter set includes the following sub-steps.

[0117] 1: Determine the valid size range of the QM based on the syntax elements included in the first parameter set.

[0118] The valid size range of the QM defines the minimum and maximum sizes of the QM that are actually used when performing inverse quantization on the quantized transform coefficients during the decoding process, and the numerical values ​​of the QM sizes are powers of 2, such as 2, 4, 8, 16, 32, and 64.

[0119] 2: QMs that fall within the valid size range are determined as valid QMs.

[0120] For example, when the valid size range of a QM is [4,32], the valid QMs include a 4x4 QM, an 8x8 QM, a 16x16 QM, and a 32x32 QM. Also, when the valid size range of a QM is [8,16], the valid QMs include an 8x8 QM and a 16x16 QM.

[0121] For example, assuming that the valid size range of QMs is determined to be [8,16], and referring to Tables 1 and 2 above, the sizeId corresponding to a QM of 8x8 size is 3, and the sizeId corresponding to a QM of 16x16 size is 4. The decoding side device determines that a total of 12 QMs with ids from 8 to 19 are valid QMs, and the remaining 16 QMs with ids from 0 to 7 and 20 to 27 are invalid QMs.

[0122] In one example, the following approach is adopted to determine the valid size range of the QM based on the syntax elements included in the first parameter set.

[0123] 1.1: Determine the minimum luma coding block size, the luma coding tree block size and the maximum luma TB size based on the syntax elements included in the first parameter set.

[0124] Optionally, a first syntax element is defined in the first parameter set, and the first syntax element is used to indicate a minimum luma coding block size. A second syntax element is defined in the first parameter set, and the second syntax element is used to indicate a block size of a luma coding tree. A third syntax element is defined in the first parameter set, and the third syntax element is used to indicate a maximum luma TB size. A decoding device reads the first syntax element, the second syntax element, and the third syntax element from the first parameter set to determine the minimum luma coding block size, the block size of a luma coding tree, and the maximum luma TB size.

[0125] 1.2: Determine a valid size range of the luma QM based on the minimum luma coding block size, the block size of the luma coding tree, and the maximum luma TB size, where the valid size range of the luma QM includes the minimum size and the maximum size of the luma QM.

[0126] Optionally, the decoding device determines the minimum size of the luma QM based on the smallest luma coding block size. For example, the smallest luma coding block size is determined as the minimum size of the luma QM. The decoding device determines a relatively larger value of the block size of the luma coding tree and the maximum luma TB size as the maximum size of the luma QM. For example, when the block size of the luma coding tree is larger than the maximum luma TB size, the block size of the luma coding tree is determined as the maximum size of the luma QM, and when the block size of the luma coding tree is less than the maximum luma TB size, the maximum luma TB size is determined as the maximum size of the luma QM. When the block size of the luma coding tree is equal to the maximum luma TB size, the two are equal, so the block size of the luma coding tree is determined as the maximum size of the luma QM or the maximum luma TB size is determined as the maximum size of the luma QM, and the result is the same.

[0127] 1.3: Determine the effective size range of the chrominance QM based on the effective size range of the luma QM and the sampling rate of the chrominance component relative to the luma component, where the effective size range of the chrominance QM includes the minimum size and maximum size of the chrominance QM.

[0128] Optionally, a fourth syntax element is defined in the first parameter set, the fourth syntax element being used to indicate a sampling rate of the chrominance component relative to the luma component.

[0129] Optionally, the decoding side device calculates the minimum size of the chrominance QM based on the minimum size of the luma QM and the sampling rate of the chrominance component relative to the luma component, and calculates the maximum size of the chrominance QM based on the maximum size of the luma QM and the sampling rate of the chrominance component relative to the luma component.

[0130] In an exemplary embodiment, taking the first parameter set as an example, the syntax elements and syntax structure table included in the APS are shown in Table 7 below.

[0131] [Table 7]

[0132] aps_qm_size_info_present_flag indicates whether the syntax element related to QM size is present in the bitstream. Its value is 1, it indicates that the syntax element related to QM size appears in the bitstream, and based on this, it is possible to determine the valid size range of QM, and thus which size of QM needs to be decoded. Its value is 0, it indicates that the syntax element related to QM size does not exist in the bitstream, and all sizes of QM need to be decoded.

[0133] For aps_log2_ctu_size_minus5, its value +5 indicates the block size of the luma coding tree. Its value is specified to be the same as the numerical value of the syntax element sps_log2_ctu_size_minus5.

[0134] For aps_log2_min_luma_coding_block_size_minus2, its value +2 indicates the minimum luma coding block size. Its value is specified to be the same as the numerical value of the syntax element sps_log2_min_luma_coding_block_size_minus2.

[0135] Regarding aps_max_luma_transform_size_64_flag, a value of 1 indicates that the maximum luma TB size is 64, and a value of 0 indicates that the maximum luma TB size is 32. The value is specified to be the same as the numerical value of the syntax element sps_max_luma_transform_size_64_flag.

[0136] aps_chroma_format_idc indicates the sampling rate of the chrominance component relative to the luminance component, and is specifically shown in Table 6. The value is specified to be the same as the numerical value of the syntax element chroma_format_idc.

[0137] Based on the above syntax elements, the derivation process of the variables minQMSizeY (representing the minimum size of the luminance QM) and maxQMSizeY (representing the maximum size of the luminance QM) is as follows:

[0138] When the value of the syntax element aps_qm_size_info_present_flag is 1, the following applies:

[0139] minQMSizeY = 1 << (aps_log2_min_luma_coding_block_size_minus2+2) Equation 12

number

[0140] where << is the left shift operator,

number

[0141] When the value of the syntax element aps_qm_size_info_present_flag is 0, the following applies:

[0142] minQMSizeY = 4, maxQMSizeY = 64.

[0143] The derivation process of the variables minQMSizeUV (representing the minimum size of the chrominance QM) and maxQMSizeUV (representing the maximum size of the chrominance QM) is as follows.

[0144] When the value of the syntax element aps_qm_size_info_present_flag is 1, the following applies:

[0145]

number

[0146] where ! represents the logical negation operation,

number

[0147] The above formulas 14 and 15 can be interpreted as follows.

[0148] If aps_chroma_format_idc does not exist, then minQMSizeUV=0, and conversely, minQMSizeUV = minQMSizeY / SubWidthC. If aps_chroma_format_idc is not present, maxQMSizeUV = 0 and conversely, maxQMSizeUV = maxQMSizeY / SubHeightC.

[0149] When the value of the syntax element aps_qm_size_info_present_flag is 0, the following applies:

[0150] minQMSizeUV = 2, maxQMSizeUV = 32.

[0151] In the syntax structure table shown in Table 7, the variable cIdx represents the chrominance component corresponding to the current QM. For the luma component Y, its value is 0, for the chroma component Cb, its value is 1, and for the chroma component Cr, its value is 2. The variable matrixSize represents the actual encoding size of the current QM and is pointed to by the third column of Table 2. The variable matrixQMSize represents the TB size corresponding to the current QM and is pointed to by Tables 1 and 2.

[0152] In the syntax structure table shown in Table 7, the decoding device first judges the two conditions proposed by the present application, and then decides whether to decode the current QM. Take the example of determining whether the first QM is a valid QM (the first QM can be any one of the available QMs, i.e., any one of the total 28 QMs mentioned above), if the first QM satisfies one of the first and second conditions, the first QM is determined to be a valid QM.

[0153] Here, the first condition is cIdx==0 && (matrixQMSize >= minQMSizeY && matrixQMSize <= maxQMSizeY), which indicates that the first QM belongs to the luma component used in the quantization process of luma TB, and the first QM is within the valid size range of luma QM [MinQMSizeY, MaxQMSizeY], where MinQMSizeY represents the minimum size of luma QM and MaxQMSizeY represents the maximum size of luma QM. The second condition is cIdx != 0 && (matrixQMSize >= minQMSizeUV && matrixQMSize <= maxQMSizeUV), which indicates that the first QM belongs to the chrominance components used in the quantization process of the chrominance TB, and the first QM is within the valid size range of the chrominance QM [MinQMSizeUV, MaxQMSizeUV], where MinQMSizeUV represents the minimum size of the chrominance QM and MaxQMSizeUV represents the maximum size of the chrominance QM.

[0154] In the above example, the decoding device needs to calculate the valid size range of the QM according to the syntax element included in the first parameter set, and then determine the valid QM according to the valid size range. In the example introduced below, the syntax element of the valid size range of the luminance QM may be directly defined in the first parameter set, and the decoding device can directly obtain the valid size range of the luminance QM after reading the syntax element, and then determine the valid size range of the chrominance QM according to the valid size range of the luminance QM together with the chrominance format. Specifically, as follows:

[0155] In another example, the following approach is adopted to determine the valid size range of the QM based on the syntax elements included in the first parameter set.

[0156] 1.1: Determine a valid size range of the brightness QM based on syntax elements included in the first parameter set, where the valid size range of the brightness QM includes a minimum size and a maximum size of the brightness QM.

[0157] Optionally, a fifth syntax element is defined in the first parameter set, and the fifth syntax element is used to indicate a minimum size of the luma QM. A sixth syntax element is defined in the APS, and the fifth syntax element is used to indicate a maximum size of the luma QM. The decoding device reads the fifth and sixth syntax elements from the first parameter set to determine the minimum and maximum sizes of the luma QM.

[0158] 1.2: Determine the effective size range of the chroma QM based on the effective size range of the luma QM and the sampling rate of the chroma component relative to the luma component, where the effective size range of the chroma QM includes the minimum size and the maximum size of the chroma QM.

[0159] Optionally, a fourth syntax element is defined in the first parameter set, the fourth syntax element being used to indicate a sampling rate of the chrominance component relative to the luma component.

[0160] Optionally, the decoding side device calculates the minimum size of the chrominance QM based on the minimum size of the luma QM and the sampling rate of the chrominance component relative to the luma component, and calculates the maximum size of the chrominance QM based on the maximum size of the luma QM and the sampling rate of the chrominance component relative to the luma component.

[0161] In an exemplary embodiment, taking the first parameter set as an example, the syntax elements and syntax structure table included in the APS are shown in Table 8 below.

[0162] [Table 8]

[0163] aps_qm_size_info_present_flag indicates whether the syntax element related to QM size is present in the bitstream. Its value is 1, it indicates that the syntax element related to QM size appears in the bitstream, and based on this, it is possible to determine the valid size range of QM, and thus which size of QM needs to be decoded. Its value is 0, it indicates that the syntax element related to QM size does not exist in the bitstream, and all sizes of QM need to be decoded.

[0164] For aps_log2_min_luma_qm_size_minus2, the value +2 indicates the minimum size of the luma QM.

[0165] For aps_log2_max_luma_qm_size_minus5, the value +5 indicates the maximum size of the luma QM.

[0166] Based on the above syntax elements, the derivation process of the variables minQMSizeY (representing the minimum size of the luminance QM) and maxQMSizeY (representing the maximum size of the luminance QM) is as follows:

[0167] When the value of the syntax element aps_qm_size_info_present_flag is 1, the following applies:

[0168] minQMSizeY = 1 << (aps_log2_min_luma_qm_size_minus2 + 2) Equation 16 maxQMSizeY = 1 << (aps_log2_max_luma_qm_size_minus5 + 5) Formula 17

[0169] where << is the left shift operator.

[0170] When the value of aps_qm_size_info_present_flag is 1, minQMSizeY and maxQMSizeY are defined to be equal to the numerical values ​​of the TB size variables MinCbSizeY and VSize, respectively, calculated and obtained by the SPS syntax element.

[0171] When the value of the syntax element aps_qm_size_info_present_flag is 0, the following applies:

[0172] minQMSizeY = 4, maxQMSizeY = 64.

[0173] aps_chroma_format_idc indicates the sampling rate of the chrominance component relative to the luminance component, and is specifically shown in Table 6. The value is specified to be the same as the numerical value of the syntax element chroma_format_idc.

[0174] The derivation process of the variables minQMSizeUV (representing the minimum size of the chrominance QM) and maxQMSizeUV (representing the maximum size of the chrominance QM) is as follows.

[0175] When the value of the syntax element aps_qm_size_info_present_flag is 1, the following applies:

[0176]

number

[0177] where ! represents the logical negation operation,

number

[0178] When the value of the syntax element aps_qm_size_info_present_flag is 0, the following applies:

[0179] minQMSizeUV = 2, maxQMSizeUV = 32.

[0180] In some other examples, the decoding device may determine a valid QM based on syntax elements included in the SPS. Specifically, the decoding device may calculate a valid size range [MinQMSizeY, MaxQMSizeY] of the luma QM and a valid size range [MinQMSizeUV, MaxQMSizeUV] of the chroma QM based on syntax elements included in the SPS. Here, the variable MinQMSizeY represents the minimum size of the luma QM, the variable MaxQMSizeY represents the maximum size of the luma QM, the variable MinQMSizeUV represents the minimum size of the chroma QM, and the variable MaxQMSizeUV represents the maximum size of the chroma QM.

[0181] As referred to in conjunction with the SPS syntax structure table shown in Table 5 above, the above variables can be calculated and obtained by the following formulas:

[0182]

number

[0183] where << is the left shift operator and ! represents the logical negation operation.

number

[0184] Compared to determining a valid QM based on syntax elements contained in an SPS, by defining relevant syntax elements in an APS and determining a valid QM based on the relevant syntax elements defined in the APS, the parsing dependency between the APS and SPS codestreams can be eliminated, such that decoding of the APS does not need to depend on the syntax elements of the SPS.

[0185] In an exemplary embodiment, the step of determining a valid QM based on syntax elements included in the first parameter set includes the following sub-steps.

[0186] 1: Read the value of the flags syntax element corresponding to the first QM from the first parameter set.

[0187] 2: If the value of the flags syntax element corresponding to the first QM is a first numeric value, determine that the first QM belongs to the valid QMs.

[0188] 3: If the value of the flags syntax element corresponding to the first QM is the second numeric value, determine that the first QM does not belong to the valid QMs.

[0189] In this embodiment, a flag syntax element is defined in the APS, and the flag syntax element indicates whether the QM belongs to a valid QM. The descriptor of the flag syntax element may be u(1), which represents a 1-bit unsigned integer. For example, the value of the flag syntax element is 1, which indicates that the QM belongs to a valid QM, and decoding is required for it, and the value of the flag syntax element is 0, which indicates that the QM does not belong to a valid QM, and decoding is not required for it. For QMs that are not to be decoded, all elements are predefined to be default values. Optionally, the default value is 16, and also refer to Equation 1. At this time, the stretch quantization coefficients of all transform coefficients in the TB are all 1, so the effect is the same as not using QM.

[0190] Also, the first QM may be any one of the available QMs, ie, any one of the total 28 QMs mentioned above.

[0191] Optionally, the first parameter set is APS. Of course, in some other embodiments, the first parameter set may not be APS, and the embodiments of the present application are not limited thereto.

[0192] In an exemplary embodiment, taking the first parameter set as an example, the syntax elements and syntax structure table included in the APS are shown in Table 9 below.

[0193] [Table 9]

[0194] Optionally, the flag syntax element is scaling_matrix_present_flag. For scaling_matrix_present_flag[id], a value of 1 indicates that the current QM needs to be decoded, and a value of 0 indicates that the current QM does not need to be decoded, and the decoding device can infer that all elements of the QM are 16.

[0195] Optionally, the luma QM corresponds to one flag syntax element, which indicates whether the luma QM needs to be decoded. For the first chroma QM (i.e., the QM corresponding to Cb) and the second chroma QM (i.e., the QM corresponding to Cr) having the same prediction mode and the same size, the same flag syntax element is shared to indicate whether the first chroma QM and the second chroma QM need to be decoded. That is, the first chroma QM and the second chroma QM do not need to use one flag syntax element by themselves, which contributes to further saving the bit overhead of QM coding signaling.

[0196] In an exemplary embodiment, taking the first parameter set as an example, the syntax elements and syntax structure table included in the APS are shown in Table 10 below.

[0197] [Table 10]

[0198] When the value of scaling_matrix_present_flag[predMode != MODE_INTRA][cIdx != 0][sizeId] is 1, it indicates that when decoding as a luma QM, the luma QM is coded in the APS, and when decoding as a chroma QM, it indicates that the prediction mode is predMode in the APS and QMs corresponding to chroma Cb and Cr, which have the same size, are coded. When the value of this syntax element is 0, it indicates that there is no need to decode the luma QM or the two chroma QMs, and the decoding device can infer that these elements are all 16.

[0199] It is necessary to explain that, for the encoding side device, when setting the value of the flag syntax element corresponding to each QM, i.e., when determining which QMs need to be encoded and which QMs do not need to be encoded, it may do so based on the size of the QM, or based on the encoding prediction mode corresponding to the QM, or further based on the YUV color components corresponding to the QM, or it may take into consideration multiple elements of the QM size, the encoding prediction mode, and the YUV color components in an integrated manner, and the embodiments of the present application are not limited to this.

[0200] In this embodiment, by defining one flag syntax element in the first parameter set, the flag syntax element can indicate whether a QM belongs to a valid QM, thereby more flexibly indicating whether each QM needs to be decoded.

[0201] Referring to Figure 11, it shows a flow chart of a video encoding method provided in one embodiment of the present application. In this embodiment, the method is mainly applied to the encoding side device introduced above as an example. The method may include the following steps (1101-1102).

[0202] Step 1101: determining an effective QM corresponding to a video frame to be coded, where the effective QM refers to a QM actually used when quantizing transform coefficients in the coding process of the video frame to be coded.

[0203] The video frame to be coded may be any one of the video frames (also called image frames) to be coded in the video to be coded.

[0204] Assuming that the number of QMs that may be used when quantizing the transform coefficients is n, the number of effective QMs may be less than or equal to n, where n is a positive integer. For example, if all n QMs are actually used when quantizing the transform coefficients, the number of effective QMs is n, and if a portion of the total n QMs (e.g., m QMs, where m is a positive integer less than n) are actually used when quantizing the transform coefficients, the number of effective QMs is m.

[0205] Optionally, for other QMs that do not belong to the valid QM, predefine all its elements as a default value, optionally the default value is 16, referring to Equation 1, at this time, the stretch quantization coefficients of all transform coefficients in TB are all 1, so the effect is the same as not using QM.

[0206] Step 1102: Perform encoding on the syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set, where the first parameter set includes a parameter set used to define the syntax elements associated with the QM.

[0207] After determining the valid QM, the number of valid QMs may be one or more, so the encoding side device needs to perform encoding for each valid QM. Take any one valid QM as an example, when encoding the valid QM, the optimal mode corresponding to the valid QM can be determined, and then the valid QM is encoded according to the optimal mode. Here, the optimal mode may be the mode with the smallest bit cost selected from the three candidate modes, the copy model of the inter-frame prediction mode, the prediction mode of the inter-frame prediction mode, and the intra-frame prediction mode, introduced above.

[0208] For example, referring to Table 1 above, assuming that it is determined that the number of QMs that can be used when quantizing the transform coefficients is 28, of which 12 are valid QMs, the encoding device only needs to encode the 12 valid QMs and does not need to encode the remaining 16 invalid QMs.

[0209] In addition, the encoding device needs to perform coding on the valid QM, and also needs to perform coding on the syntax elements used to determine the valid QM, so that the decoding device determines the valid QM based on the syntax elements. The encoding device performs coding on the syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set. The first parameter set may be an APS, or may be another parameter set used to define the syntax elements related to the QM, and the embodiment of the present application is not limited thereto.

[0210] As described above, in the technical means provided by the embodiments of the present application, by determining an effective QM corresponding to a video frame to be coded, the effective QM refers to the QM actually used when quantizing transform coefficients in the coding process of the video frame to be coded, and then coding is performed on the syntax elements used for determining the effective QM and the effective QM to generate a codestream corresponding to a first parameter set. In this way, the encoder side only codes and transmits the effective QM, thereby saving the codewords that need to be occupied by QM signaling and reducing bit overhead, and the decoder side only needs to decode on the effective QM, thereby reducing the computational complexity of the decoder side.

[0211] In addition, the encoding process of the encoding side device corresponds to the decoding process of the decoding side device, and for details not described in detail in the encoding process, please refer to the introduction and description of the embodiment of the decoding process above, and they will not be described in detail again here.

[0212] The following are examples of the apparatus of the present application, which can be used to carry out the method of the present application. For details not disclosed in the apparatus of the present application, reference can be made to the method of the present application.

[0213] Referring to FIG. 12, it shows a block diagram of a video decoding device provided by one embodiment of the present application. The device has a function of implementing the above example of the video decoding method, and the above function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The device may be the above-introduced decoding side device, or may be installed on the decoding side device. The device 1200 may include a parameter acquisition module 1210, a QM determination module 1220, and a QM decoding module 1230.

[0214] The parameter acquisition module 1210 is used for acquiring a first parameter set corresponding to a video frame to be decoded, where the first parameter set is a parameter set used for defining a syntax element related to QM.

[0215] The QM determination module 1220 is used to determine an effective QM based on the syntax elements included in the first parameter set, and the effective QM refers to the QM actually used when performing inverse quantization on the quantized transform coefficients in the decoding process of the video frame to be decoded.

[0216] The QM decoding module 1230 is used to perform decoding on the valid QM.

[0217] In an exemplary embodiment, as shown in FIG. 13, the QM determining module 1220 includes a range determining unit 1221 and a QM determining unit 1222.

[0218] The range determining unit 1221 is used for determining the valid size range of the QM according to the syntax elements included in the first parameter set.

[0219] The QM determining unit 1222 is used for determining a QM that falls within the valid size range as the valid QM.

[0220] In an exemplary embodiment, the range determining unit 1221 is determining a minimum luma coding block size, a luma coding tree block size, and a maximum luma TB size based on syntax elements included in the first parameter set; determining a valid size range of a luma QM based on the minimum luma coding block size, the luma coding tree block size, and the maximum luma TB size, where the valid size range of the luma QM includes a minimum size and a maximum size of the luma QM; Determine a valid size range of a chrominance QM based on the valid size range of the luminance QM and the sampling rate of the chrominance component relative to the luminance component, where the valid size range of the chrominance QM includes the minimum size and maximum size of the chrominance QM.

[0221] In an exemplary embodiment, the range determining unit 1221 is determining a minimum size of the luma QM based on the minimum luma coding block size; determining a relatively larger value among the block size of the luma coding tree and the maximum luma TB size as the maximum luma QM size.

[0222] In an exemplary embodiment, the range determining unit 1221 is determining a valid size range of the luminance QM based on syntax elements included in the first parameter set, where the valid size range of the luminance QM includes a minimum size and a maximum size of the luminance QM; Determine a valid size range of a chrominance QM based on the valid size range of the luminance QM and the sampling rate of the chrominance component relative to the luminance component, where the valid size range of the chrominance QM includes the minimum size and maximum size of the chrominance QM.

[0223] In an exemplary embodiment, the range determining unit 1221 is calculating a minimum size of the chrominance QM based on a minimum size of the luma QM and a sampling rate of the chrominance components relative to the luma component; Calculating the maximum size of the chrominance QM based on the maximum size of the luma QM and a sampling rate of the chrominance component relative to the luma component.

[0224] In an exemplary embodiment, the QM determination unit 1222 is if the first QM satisfies one of the first and second conditions, the first QM is used to determine the valid QM; Wherein, the first condition is cIdx==0 && (matrixQMSize>=minQMSizeY && matrixQMSize<=maxQMSizeY), the first condition indicates that the first QM belongs to a luma component used in the quantization process of luma TB, and the first QM is within the valid size range [MinQMSizeY,MaxQMSizeY] of the luma QM, where MinQMSizeY indicates the minimum size of the luma QM and MaxQMSizeY indicates the maximum size of the luma QM; The second condition is cIdx!=0 && (matrixQMSize >= minQMSizeUV && matrixQMSize <= maxQMSizeUV), which indicates that the first QM belongs to the chrominance components used in the quantization process of the chrominance TB, and the first QM is within the valid size range [MinQMSizeUV, MaxQMSizeUV] of the chrominance QM, where MinQMSizeUV represents the minimum size of the chrominance QM and MaxQMSizeUV represents the maximum size of the chrominance QM.

[0225] In an exemplary embodiment, as shown in FIG. 13, the QM determination module 1220 includes an element reading unit 1223 and a QM judging unit 1224 .

[0226] The element reading unit 1223 is used to read the value of the flag syntax element corresponding to the first QM from the first parameter set.

[0227] The QM determination unit 1224 is used to determine that the first QM belongs to the valid QM if the value of the flag syntax element corresponding to the first QM is a first numerical value, and to determine that the first QM does not belong to the valid QM if the value of the flag syntax element corresponding to the first QM is a second numerical value.

[0228] In an example embodiment, a first chroma QM and a second chroma QM having the same prediction mode and the same size share the same flags syntax element.

[0229] In an exemplary embodiment, the flag syntax element is scaling_matrix_present_flag.

[0230] In an exemplary embodiment, the first parameter set is APS.

[0231] In an exemplary embodiment, for other QMs that do not belong to the above valid QMs, all elements thereof are predefined to be default values.

[0232] In the exemplary embodiment, the default value is 16.

[0233] As described above, in the technical means provided by the embodiments of the present application, a first parameter set corresponding to a video frame to be decoded is obtained, and an effective QM is determined according to the syntax elements included in the first parameter set. The effective QM refers to the QM actually used when quantizing transform coefficients in the process of encoding and generating the video frame to be decoded, and then decoding is performed according to the effective QM. In this way, the decoder side only needs to perform decoding according to the effective QM, thereby reducing the computational complexity of the decoder side.

[0234] Referring to FIG. 14, it shows a block diagram of a video encoding device provided by one embodiment of the present application. The device has a function of implementing the above-mentioned example of the video encoding method, and the above-mentioned function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The device may be the encoding side device introduced above, or may be installed on the encoding side device. The device 1400 may include a QM determination module 1410 and a QM encoding module 1420.

[0235] The QM determination module 1410 is used to determine an effective QM corresponding to the video frame to be encoded, where the effective QM refers to the QM actually used when quantizing the transform coefficients in the encoding process of the video frame to be encoded.

[0236] The QM encoding module 1420 is configured to perform encoding on the syntax elements used to determine the valid QM and the valid QM to generate a codestream corresponding to a first parameter set, where the first parameter set is a parameter set used to define a syntax element related to a QM.

[0237] As described above, in the technical means provided by the embodiments of the present application, by determining an effective QM corresponding to a video frame to be coded, the effective QM refers to the QM actually used when quantizing transform coefficients in the coding process of the video frame to be coded, and then coding is performed on the syntax elements used for determining the effective QM and the effective QM to generate a codestream corresponding to a first parameter set. In this way, the encoder side only codes and transmits the effective QM, thereby saving the codewords that need to be occupied by QM signaling and reducing bit overhead, and the decoder side only needs to decode on the effective QM, thereby reducing the computational complexity of the decoder side.

[0238] It is necessary to explain that, although the above embodiment provides the device by only dividing each of the above-mentioned functional modules, the above-mentioned functions can be assigned to different functional modules as required in the actual application. That is, the internal structure of the device is divided into different functional modules to complete all or part of the above-mentioned functions. In addition, the device provided by the above embodiment belongs to the same idea as the method embodiment, and the specific implementation process thereof can be referred to the method embodiment, so it will not be described in detail again here.

[0239] 15, which shows a structural block diagram of a computer device provided by one embodiment of the present application. The computer device may be the encoding device introduced above, or may be the decoding device introduced above. The computer device 150 may include a processor 151, a memory 152, a communication interface 153, an encoder / decoder 154, and a bus 155.

[0240] The processor 151 includes one or more processing cores, and executes various functional applications and information processing by running software programs and modules.

[0241] The memory 152 can be used to store a computer program, and the processor 151 can be used to execute the computer program, thereby implementing the above-mentioned video encoding method or implementing the above-mentioned video decoding method.

[0242] The communication interface 153 can be used to communicate with other devices, for example to send and receive audio and video data.

[0243] The encoder / decoder 154 can be used to implement encoding and decoding functions, for example, to encode and decode audio and video data.

[0244] The memory 152 is coupled to the processor 151 by a bus 155 .

[0245] Additionally, memory 152 may be implemented with any type of volatile or non-volatile storage device or combination thereof, including, but not limited to, a magnetic or compact disk, an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Static Random-Access Memory (SRAM), a Read-Only Memory (ROM), a magnetic memory, a flash memory, and a Programmable read-only memory (PROM).

[0246] As will be appreciated by those skilled in the art, the structure shown in FIG. 15 does not constitute a limitation on computing device 150, which may include more or fewer components than shown, or may combine certain components, or may employ different components in an arrangement.

[0247] In an exemplary embodiment, a computer-readable storage medium is further provided, wherein at least one instruction, at least one program, code set or instruction set is stored on the computer-readable storage medium, and the at least one instruction, at least one program, code set or instruction set, when executed by a processor, realizes the video decoding method or realizes the video encoding method.

[0248] In an exemplary embodiment, there is further provided a computer program product, which, when executed by a processor, implements the above video decoding method or is used to implement the above video encoding method.

[0249] As will be understood, "plurality" as referred to herein refers to two or more than two. "And / or" describes a relationship between related objects and indicates that three types of relationships may exist, for example, A and / or B may represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the related objects before and after are in a type of "or" relationship.

[0250] The above are merely illustrative embodiments of the present application, and are not used to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included within the scope of protection of the present application. [Explanation of symbols]

[0251] 60 Video Encoder 62 Conversion Module 64 Quantization Module 66 Entropy Coding Module 70 Video Decoder 72 Entropy Decoding Module 74 Inverse quantization module 76 Reverse conversion module 150 Computer Equipment 151 processor 152 Memory 153 Communication Interface 154 Decoder 155 Bus 200 Communication Systems 210 1st device 220 2nd device 230 Third equipment 240 4th equipment 250 Network 301 Video Source 302 Video Picture Stream 303 Video Encoder 304 Video Data 305 Streaming Server 310 Video Decoder 311 Output Video Picture Stream 312 Display 313 Collection Subsystem 1210 Parameter Acquisition Module 1220 QM Decision Module 1221 Range Determination Unit 1222 QM Decision Unit 1223 Element Reading Unit 1224 QM Judgment Unit 1230 QM Decoding Module 1410 QM Decision Module 1420 QM encoding module

Claims

1. 1. A video decoding method, the method comprising: obtaining a first parameter set corresponding to a video frame to be decoded, the first parameter set including a syntax element related to a quantization matrix QM, the QM having 28 identifier variables, the 28 identifier variables being associated with different QM sizes and different QM types; determining a valid QM based on a syntax element included in the first parameter set, the syntax element being used to indicate whether a chroma QM needs to be decoded, the valid QM being a QM actually used when performing inverse quantization on quantized transform coefficients in a decoding process of the video frame to be decoded; determining a range of an identifier variable of the QM based on the syntax element included in the first parameter set, and determining a QM that falls within the range of the identifier variable of the QM as the valid QM; and performing decoding on the valid QM; The value of the syntax element is determined based on a sampling rate of the chrominance component relative to the luminance component; A video decoding method, wherein if the sampling rate of the chrominance component relative to the luma component is 4:0:0, the value of the syntax element is 0, indicating that the chrominance QM does not need to be decoded, and all elements of the chrominance QM are set to 16.

2. The step of determining a valid QM based on syntax elements included in the first parameter set comprises: reading a value of a flags syntax element corresponding to a first QM from the first parameter set; determining that the first QM belongs to the valid QMs if the value of a flag syntax element corresponding to the first QM is a first numeric value; and determining that the first QM does not belong to the valid QMs if a value of a flag syntax element corresponding to the first QM is a second numeric value.

3. The method described in claim 2, wherein the first color difference QM and the second color difference QM having the same prediction mode and the same size share the same flag syntax element.

4. The method described in claim 1, wherein the first parameter set is a self-adaptive parameter set APS.

5. The method described in claim 1, wherein for other QMs that do not belong to the valid QM, all elements thereof are predefined to be default values.

6. The method described in claim 5, wherein the default value is 16.

7. A video encoding method, the method comprising: generating an encoded video bitstream corresponding to a first parameter set corresponding to a video frame to be encoded, the first parameter set including a syntax element associated with a quantization matrix QM, the QM having 28 identifier variables, the 28 identifier variables being associated with different QM sizes and different QM types; determining a valid quantization matrix QM based on the syntax element included in the first parameter set, the syntax element being used to indicate whether a chroma QM needs to be coded, where the valid QM refers to a QM actually used when performing inverse quantization on quantized transform coefficients in coding a video frame to be coded; determining a range of an identifier variable of the QM based on the syntax element included in the first parameter set; and determining a QM within the range of the identifier variable of the QM as the valid QM; and encoding the valid QM; The value of the syntax element is determined based on a sampling rate of the chrominance component relative to the luminance component; A video encoding method, wherein if the sampling rate of the chrominance component relative to the luma component is 4:0:0, the value of the syntax element is 0, indicating that the chrominance QM does not need to be decoded, and all elements of the chrominance QM are set to 16.

8. A method for generating and storing an encoded video bitstream, said method comprising: generating an encoded video bitstream corresponding to a first parameter set corresponding to a video frame to be encoded, the first parameter set including a syntax element associated with a quantization matrix QM, the QM having 28 identifier variables, the 28 identifier variables being associated with different QM sizes and different QM types; determining a valid quantization matrix QM based on the syntax element included in the first parameter set, the syntax element being used to indicate whether a chroma QM needs to be coded, where the valid QM refers to a QM actually used when performing inverse quantization on quantized transform coefficients in coding a video frame to be coded; determining a range of an identifier variable of the QM based on the syntax element included in the first parameter set; and determining a QM within the range of the identifier variable of the QM as the valid QM; encoding the valid QM; and storing the generated encoded video bitstream; The value of the syntax element is determined based on a sampling rate of the chrominance component relative to the luminance component; A method in which, if the sampling rate of the chrominance component relative to the luma component is 4:0:0, the value of the syntax element is 0, indicating that the chrominance QM does not need to be decoded, and all elements of the chrominance QM are set to 16.

9. A video decoding device, comprising: a parameter acquisition module; a QM determination module; and a QM decoding module; The parameter acquisition module is used to acquire a first parameter set corresponding to a video frame to be decoded, the first parameter set including a syntax element related to a quantization matrix QM, the QM having 28 identifier variables, the 28 identifier variables being associated with different QM sizes and different QM types; the QM determination module is used to determine a valid QM based on a syntax element included in the first parameter set, the syntax element being used to indicate whether a chroma QM needs to be decoded, the valid QM being a QM actually used when performing inverse quantization on quantized transform coefficients in a decoding process of the video frame to be decoded; a range of an identifier variable of a QM is determined based on an element included in the first parameter set, and a QM falling within the range of the identifier variable of the QM is determined as the valid QM; the QM decoding module is used to perform decoding on the valid QM; The value of the syntax element is determined based on a sampling rate of the chrominance component relative to the luminance component; A video decoding device, wherein when the sampling rate of the chrominance component relative to the luma component is 4:0:0, the value of the syntax element is 0, indicating that the chrominance QM does not need to be decoded, and all elements of the chrominance QM are set to 16.

10. A computer device comprising a processor and a memory, wherein at least one program, code set or instruction set is stored in the memory, and wherein the at least one program, code set or instruction set is loaded and executed by the processor to realize a method according to any one of claims 1 to 6.

11. A computer device comprising a processor and a memory, wherein at least one program, code set or instruction set is stored in the memory, and wherein the at least one program, code set or instruction set is loaded and executed by the processor to realize the method of claim 7.

12. A computer device comprising a processor and a memory, wherein at least one program, code set or instruction set is stored in the memory, and wherein the at least one program, code set or instruction set is loaded and executed by the processor to realize the method of claim 8.

13. A computer program which, when executed by a processor, causes the processor to perform a method according to any one of claims 1 to 6.

14. A computer program which, when executed by a processor, causes the processor to perform the method of claim 7.

15. A computer program which, when executed by a processor, causes the processor to perform the method of claim 8.

Images