Video decoder and decoding method

Abstract

A method of decoding an image frame from a stream of bits is disclosed. The method comprises the steps that N image blocks of an image frame are grouped into P sub-images, the P sub-images are not overlapped with one another, the sub-images of the P sub-images comprise Q columns of image blocks, N is an integer exceeding 1, P and Q are positive integers, the first column of image blocks in the Q columns of image blocks are divided into M sub-slices, the M sub-slices are not overlapped with one another, M is an integer exceeding 1, and the M is an integer exceeding 1. The processor is used for acquiring the first image block information from the memory before decoding the current sub-slice in the M sub-slices, decoding the current sub-slice in the M sub-slices according to the first image block information by the processor, and storing the second image block information in the memory after the current sub-slice in the M sub-slices is decoded.

Description

Technical Field

[0001] The present invention relates to video processing, and more particularly to a video decoder and a low-delay decoding method. Background Art

[0002] A video decoder is a device that converts the encoded (compressed) video stream from a video encoder into a raw (uncompressed) video stream for display on a monitor. Video decoders are widely used in display devices such as smartphones, laptops, desktop computers, and game consoles.

[0003] The video encoder / decoder (codec) may adopt a tile-based video coding standard such as the high efficiency video coding (HEVC) standard. In a tile-based video coding standard, an image is divided into one or more tiles, and each tile includes a set of largest coding units (LCUs), which can be encoded or decoded independently of another tile. Typically, the tiles of an image are decoded in a left-to-right, top-to-bottom order, i.e., a tile-based raster scan order. The pixels of an image are displayed in sequence from left to right and top to bottom, i.e., a pixel-based raster scan order. Since the shape of a tile is different from a whole row of pixels, the decoding order may be different from the display order, and the display device cannot display the pixels of the image before decoding one or more rows of pixels, thus causing display delay. Summary of the invention

[0004] An embodiment of the present invention provides a method for decoding an image frame from a bit stream. The method includes grouping N tiles of the image frame into P sub-images, the P sub-images do not overlap with each other, the sub-images of the P sub-images include Q columns of tiles, N is an integer greater than 1, P and Q are positive integers, the first column of tiles in the Q columns of tiles is divided into M sub-slices, the M sub-slices do not overlap with each other, M is an integer greater than 1, before decoding a current sub-slice in the M sub-slices, obtaining first tile information from a memory, a processor decoding the current sub-slice in the M sub-slices according to the first tile information, and after the decoding of the current sub-slice in the M sub-slices is completed, storing second tile information in the memory.

[0005] An embodiment of the present invention provides a video decoder for decoding an image frame from a bit stream. The video decoder includes a sub-image circuit, a sub-slice circuit, a memory controller, and a processor. The sub-image circuit is used to group N blocks of the image frame into P sub-images, the P sub-images do not overlap with each other, and the sub-images of the P sub-images include Q columns of blocks, N is an integer greater than 1, and P and Q are positive integers. The sub-slice circuit is coupled to the sub-image circuit, and is used to divide the first column of blocks in the Q columns of blocks into M sub-slices, the M sub-slices do not overlap with each other, and M is an integer greater than 1. The memory controller is coupled to the memory, and is used to obtain first block information from the memory before decoding a current sub-slice in the M sub-slices. The processor is coupled to the sub-slice circuit and the memory controller, and is used to decode the current sub-slice in the M sub-slices according to the first block information. After the current sub-slice in the M sub-slices is decoded, the memory controller stores the second block information in the memory.

[0006] An embodiment of the present invention provides a method for decoding an image frame from a bit stream. The method includes dividing a Q-column tile into S sub-slices, the S sub-slices include k groups of Q sub-slices, each group of Q sub-slices is divided from the first column to the Qth column, the S sub-slices do not overlap with each other, Q and k are positive integers, S is an integer greater than 1, before decoding a current sub-slice among the S sub-slices, obtaining first tile information from a memory, a processor decoding the current sub-slice among the S sub-slices according to the first tile information, and after completing the decoding of the current sub-slice among the S sub-slices, storing second tile information in the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 It is a schematic diagram of the decoding order of images in an embodiment of the present invention.

[0008] Figure 2A Displays the video format of AV1.

[0009] Figure 2B Displays the HEVC video format.

[0010] Figure 3 is a schematic diagram of sub-images and sub-slices.

[0011] Figure 4 is a flowchart of the low-latency video decoder program.

[0012] Figure 5 and Figure 6 is a block diagram of a low-latency video decoder system.

[0013] Figure 7 is a schematic diagram of a low-latency decoding order for a single processor.

[0014] Figure 8 is a diagram of another low-latency decoding order for a single processor.

[0015] Fig. 9 Yes Operation Fig.16 or Fig.17 Flow chart of a method for operating an image frame in FIG.

[0016] Fig.10 is a diagram of another low-latency decoding order for a single processor.

[0017] Fig.11 is a diagram of another low-latency decoding order for dual processors.

[0018] Fig.12 is a diagram of another low-latency decoding order for dual processors.

[0019] Fig.13 is a flow chart of a method for decoding an image frame from a bit stream.

[0020] Fig.14 is a diagram of another low-latency decoding order for multiple processors.

[0021] Fig.15 is a diagram of another low-latency decoding order for multiple processors.

[0022] Fig.16 is a diagram of another low-latency decoding order for multiple processors.

[0023] Fig.17 is a diagram of another low-latency decoding order for multiple processors.

[0024] Fig.18 is a diagram of another low-latency decoding order for multiple processors.

[0025] Fig.19 is a diagram of another low-latency decoding order for multiple processors.

[0026] Fig. 20 is a flow chart of a method for decoding an image frame from a bit stream.

[0027] Fig.21 2 is a schematic diagram of a reconstruction buffer and a display driver using a normal decoding order when t=t0+46*tctu.

[0028] Fig. 22 1 is a schematic diagram of a reconstruction buffer and a display driver using a normal decoding order when t=t0+59*tctu.

[0029] Fig.23Schematic diagram of a reconstruction buffer and display driver using a low-delay decoding order at t=t0+13*tctu.

[0030] Fig.24 2 is a schematic diagram of a reconstruction buffer and a display driver using a normal decoding order when t=t0+26*tctu.

[0031] Fig.25 1 is a schematic diagram of a reconstruction buffer and a display driver using a normal decoding order when t=t0+90*tctu.

[0032] Fig.26 1 is a schematic diagram of a reconstruction buffer and a display driver using a normal decoding order when t=t0+103*tctu.

[0033] Fig. 27 Schematic diagram of a reconstruction buffer and display driver using a low-delay decoding order at t=t0+13*tctu.

[0034] Fig.28 2 is a schematic diagram of a reconstruction buffer and a display driver using a normal decoding order when t=t0+26*tctu.

[0035] The reference numerals are described as follows:

[0036] 4,9,13,20: Methods

[0037] 5: Video decoder

[0038] 501:Search Circuit

[0039] 502: Sub-image circuit

[0040] 503: Sub-slice circuit

[0041] 504: memory controller

[0042] 505,611: Processor

[0043] 506: Memory

[0044] 507: Random Access Memory

[0045] 508: Rebuild buffer

[0046] 509: Display system

[0047] 510: Display panel

[0048] S401 to S419, S901 to S907, S131 to S137, S201 to S205: Steps

[0049] C0 to C3: Tile columns

[0050] R0 to R4: Tile rows

[0051] S0: Sequence header

[0052] E0 to EM, E(M+1), EN, EK, E(K+1), EH: starting point mark

[0053] SPS0: Sequence Parameter Set

[0054] P0 to PN: Image

[0055] H0: frame header

[0056] TH1,TH2: Tile group header

[0057] PPS0: Picture Parameter Set

[0058] SH1, SH2, SH(M+1), SHK: Slice header

[0059] SD0,SD1,SD(M+1),SDK: Slice data DETAILED DESCRIPTION

[0060] Embodiments of the present invention are applicable to video decoders or video encoders / decoders (video codecs) that support tile-based video coding standards, such as high efficiency video coding (HEVC), versatile video coding (VVC), video processor 9 (VP9), alliance for open media video 1 (AV1), and audio video coding standard 3 (AVS3). Generally speaking, an image can be divided into one or more tiles, each of which includes one or more largest coding units (LCUs). For H.264, HEVC, VVC, and AVS3, an LCU can be called a coding tree unit (CTU). For VP9 and AV1, an LCU can be called a super block (SB). The size of each LCU (CTU / SB) is at least 16x16 pixels. In various embodiments of the present invention, tiles are defined by thick lines and LCUs (CTU / SB) are defined by thin lines. Tiles have no dependencies and can be used as the smallest unit for parallel decoding or multi-processor decoding. The decoding order of video decoders / video codecs is based on both tiles and LCUs, from left to right and from top to bottom, also known as tile-based raster order decoding. In an image, tiles are decoded according to tile-based raster order. The shapes of tiles can be the same or different, and in each tile, the LCUs are decoded according to the LCU-based raster order.

[0061] exist Figure 1 In an embodiment of the present invention, an image is divided into tiles 0 to 7, arranged as 2 columns of tiles (referred to as tile columns) and 4 rows of tiles (referred to as tile rows), wherein tile 0 includes 4x4 LCUs, tile 1 includes 4x4 LCUs, tile 2 includes 3x4 LCUs, tile 3 includes 2x4 LCUs, tile 4 includes 4x4 LCUs, tile 5 includes 4x4 LCUs, tile 6 includes 3x4 LCUs, and tile 7 includes 2x4 LCUs. Figure 1, the image is decoded from tile 0 to 7 in the raster scan order of the tiles, and each LCU in tiles 0 to 7 is decoded from left to right and from top to bottom in the raster scan order of the LCUs. For example, tiles 0 to 7 are decoded sequentially from the top tile row to the bottom tile row in the direction of the arrows, and the 4x4 LCUs of tile 0 are decoded sequentially from the top LCU row to the bottom LCU row in the direction of the arrows. When decoding, the final output decoded bitstream is based on the raster scan order of the pixels regardless of their decoding order. Delays or failures in the decoded bitstream may cause image defects. Therefore, display will not begin until the reconstruction buffer holds more than one row of decoded pixels.

[0062] In an embodiment of the present invention, an image frame decoded from a bitstream includes N tiles, where N is an integer greater than 1. For example, if N=8, the image frame includes 8 tiles. The image can be divided by vertical tile boundaries (e.g., 3 vertical tile boundaries) to obtain one or more tile columns (e.g., 4 tile columns), and can be divided by horizontal tile boundaries (e.g., 1 horizontal tile boundary) to obtain one or more tile rows (e.g., 2 tile rows). When the number of tile columns in a video picture exceeds the number of available cores (or processors) in a video decoder and is less than the number of tile columns, the decoding order can be adjusted to ensure efficient use of hardware resources and minimize display delay. Display delay refers to the time required from receiving the bitstream from the video decoder to displaying the decoded bitstream on the display panel. To solve the above problems, an embodiment of the present invention uses sub-pictures and / or sub-slices. Generally, sub-images can be decoded in parallel by multiple processors of a video decoder to speed up the decoding process, and the decoding order of sub-slices in each sub-image can be aligned with the display order to ensure that the decoded data is displayed as quickly as possible, thereby reducing display delay.

[0063] The embodiment of the present invention adopts the concepts of sub-image and sub-slice, wherein the definitions of sub-image and sub-slice are as follows:

[0064] 1. A sub-image is an image partition for multi-processor decoding. An image may be divided into at least one sub-image, and a sub-image may include at least one tile, and LCUs or tiles of any two sub-images do not overlap. In addition, tiles in a sub-image may be arranged in a rectangular or non-rectangular shape. The tiles in a sub-image are decoded according to the raster scan order of the tiles, and the LCUs in the tiles are internally decoded according to the raster scan order of the LCUs. In a multi-processor device, each sub-image may be assigned to one processor, and each processor may decode one or more assigned sub-images. For example, if an image includes two sub-images and a video decoder includes two processors, one of the two sub-images may be assigned to a first processor for decoding, and the other of the two sub-images may be assigned to a second processor for decoding. In another example, if an image includes three sub-images and a video decoder includes two processors, two of the three sub-images may be assigned to a first processor for decoding, and the remaining sub-images of the three sub-images may be assigned to a second processor for decoding. The first processor and the second processor can work simultaneously to speed up the decoding speed, which is beneficial to reduce the display delay.

[0065] 2. A subslice is a partition of a sub-image, which is used to provide breakpoints for decoding and thus adjust the decoding order. Each tile column of a sub-image can be divided into subslices, and the subslices do not overlap with each other. A subslice may include one or more consecutive LCUs, and multiple LCUs in a subslice may be arranged in a rectangular or non-rectangular shape. Each subslice may be marked with a sequence number for decoding. For example, a subslice with a sequence number of 0 may be decoded first, followed by another subslice with a sequence number of 1, followed by another subslice with a sequence number of 2, and so on. In some embodiments, 2 subslices in 2 adjacent tile columns may be labeled with consecutive sequence numbers. In other embodiments, 2 subslices in 1 tile column may be labeled with non-consecutive sequence numbers. In other embodiments, 2 subslices in 1 tile column may be labeled with consecutive sequence numbers. A subslice may occupy one or more tiles in the same tile column, but may not occupy 2 or more tiles in the same or different tile rows of a sub-image. In some embodiments, 1

[0066] A sub-slice can occupy parts of 2 tiles in the same tile column. For example, Fig.10 In the single processor decoding shown, sub-slice 3 occupies the entire tile 3 in tile column C3, and sub-slice 5

[0067] Occupies part of tile 1 and part of tile 5 in tile column C1, and sub-slice 9 occupies tile column C1

[0068] , the partial tile 5, the entire tile 9, the entire tile 13, and the entire tile 17. Similarly, Figure 3 In the dual processor decoding shown, subslices 0-3 occupy part of tile 1 in tile column C1.

[0069] and part of block 5. For example, in Fig.10 In the single processor decoding shown, tile 3 includes the entire sub-slice 3, and tile 2 includes the entire sub-slice 2, the entire sub-slice 6, and a portion of the sub-slice 10. Similarly, in Figure 3 In the dual processor decoding shown, tile 0 includes the entire subslices 0-0 and 0-2.

[0071] Figure 2A and Figure 2B Displays AV1 video format and HEVC video format respectively. Figure 2A and Figure 2B Shows how to use the independent encoding / decoding properties of each tile to adjust the encoding / decoding sequence, as explained below: 1. The video decoder can decode each tile independently and can identify the start of each tile in the bitstream

[0072] point (or starting point). For example, Figure 2A In the , these starting points are marked as E0 to EN,

[0073] And in Figure 2B In the example, the starting points are marked as E0 to EH. Figure 2A and Figure 2B As shown by the arrow in Figure 2A , the bit stream includes a sequence header S0 and pictures P0 to PN. Picture P0

[0074] The frame header H0 includes a first tile group header TH1, tiles 0 to M corresponding to the first tile group header TH1, a second tile group header TH2, and tiles (M+1) to (M+2) corresponding to the second tile group header TH2.

[0075] N, and other groups of tiles. The frame header H0 may include the number of columns of tiles in the picture P0 (e.g., num_tile_columns_minus1) and the number of rows of tiles (e.g., num_tile_rows_minus1).

[0076] Related information. Figure 2B The bitstream includes a sequence parameter set SPS0 and pictures P0 to PN. Picture P0 includes a picture parameter set PPS0, a slice header SH1, and a tile 0 of slice data SD0.

[0077] The slice data may occupy multiple tiles. For example, the slice data SD0 occupies tiles 0 to M, and the slice data SD1 occupies tiles (K+1) to N. For example, tile (M+1) may include multiple sets of slice headers SH(M+1) and slice data SD(M+1), and tile K may include multiple sets of slice headers SHK and slice data SDK, each set of slice headers and slice data corresponding to one slice segment. The video decoder may identify the number of tiles in each image according to the information related to the number of tile rows and the number of tile columns, and determine the starting points (E0 to EN) of multiple tiles in each image accordingly.

[0078] In some embodiments, the video encoder / decoder may

[0079] (num_tile_columns_minus1+1)*(num_tile_rows_minus1+1) is determined as the number of tiles in the image, and the starting point of the tile in the image is determined based on the number of tiles in the image. For example, if num_tile_columns_minus1=3 and num_tile_rows_minus1=1, then the tile

[0080] = 8 (= (3 + 1) * (1 + 1)). The video encoder / decoder can determine the starting points of 8 tiles accordingly. 2. The video decoder can include multiple processors and can divide the image into equal parts as the number of processors.

[0081] Sub-images, such as Figure 3 For example, if the video decoder includes two processors, the video decoder may divide the image into sub-image 0 and sub-image 1.

[0082] 3. Divide a tile column into at least one sub-slice according to the latency requirement. Figure 3 As shown, block 0

[0083] To 7, it is divided into 2 tile rows (R0 and R1) and 4 tile columns (C0 to C3). The height of the sub-slice can be set to meet the low latency requirement. For example, the height of the sub-slice can be set to 1 CTU / SB to achieve the lowest latency requirement. To simplify the control setting, the sub-slice can be set to a rectangle and the width of the sub-slice can be set to the width of the tile. The decoding order can be determined according to actual needs. After decoding the sub-slices in the same tile column, the video decoder can switch to another tile column to decode the sub-slice until all sub-slices in the sub-image are decoded. The internal decoding of the sub-slice can be performed in the CTU / SB raster scan order. Figure 3As shown, for the sub-image 0 on the left, the decoding order may be sub-slice 0-0, sub-slice 0-1, sub-slice 0-2, sub-slice 0-3,

[0084] Sub-slices 0-4, sub-slices 0-5, sub-slices 0-6. For sub-image 1 on the right, the decoding order may be sub-slices 1-0, sub-slices 1-1, sub-slices 1-2, sub-slices 1-3, sub-slices 1-4, sub-slices 1-5, sub-slices 1-6, sub-slices 1-7, and sub-slices 1-8.

[0085] 4. To ensure decoding accuracy, after the last CTU / SB in a sub-slice is decoded, the currently decoded tile information needs to be stored. Tile information includes arithmetic decoding information, context required for arithmetic decoding, bitstream consumption, and other information. In some embodiments, other information may include the offset and range of arithmetic decoding. Figure 3 As shown, if the current CTU / SB is the last CTU / SB of the tile (e.g., the 16th CTU of tile 0), the current

[0086] This is because tiles are decoded independently of each other, and the tile information of the current CTU / SB will not be used for decoding the next tile (for example, the tile information of tile 0 will not be used for decoding tile 1).

[0087] 5. Before decoding the first CTU / SB of a sub-slice, read the previously reserved slice information to achieve continuous decoding, such as Figure 3 If the current CTU / SB is the first CTU / SB of the block (e.g. Figure 3 For example, if the first CTU of tile 4 in the decoded image is selected, the tile information can be selectively read (for example, the tile information of tile 0 can be selectively read before decoding tile 4). This is because tiles are decoded independently of each other, so the tile information of other tiles will not be used.

[0088] 6. The memory space required by the video decoder to store tile information is (number of tile rows)*(space for tile information). For example, if the number of tile rows is 4 and the space for tile information is 1 byte, the required memory space is 4 bytes.

[0089] Figure 4 is a flowchart of a low-delay video decoder process 4 according to an embodiment of the present invention. Figure 5 is a block diagram of a low-delay video decoder system 5. Figure 6 is a block diagram of another low-delay video decoder system 6. Figure 5 Display single processor video decoder and Figure 6 Displays multiprocessor video decoders. Figure 5 and Figure 6The video decoder in Figure 4 The low-latency video decoding program achieves low-latency decoding.

[0090] Method 4 includes steps S401 to S419. Any reasonable technical changes or step adjustments fall within the scope of the present invention. Steps S401 to S419 are explained as follows:

[0091] Step S401: Image starts;

[0092] Step S403: Divide the image into sub-images;

[0093] Step S405: Divide the sub-image into sub-slices;

[0094] Step S407: Searching for a block start point in the bit stream;

[0095] Step S409: Is the current CTU / SB the first CTU / SB in the sub-slice? If yes, proceed to step S411; If no, proceed to step S413;

[0096] Step S411: Restore the block information;

[0097] Step S413: CTU / SB decoding;

[0098] Step S415: Is the current CTU / SB the last CTU / SB in the sub-slice? If yes, proceed to step S417; if no, proceed to step S419;

[0099] Step S417: storing the image block information;

[0100] Step S419: Is the current CTU / SB the last CTU / SB in the image? If yes, the image is complete; if no, proceed to step S409.

[0101] Reference now Figure 5 Video Decoder 5 in the Figure 4 The low-latency video decoder process in 4. Figure 5As shown, the video decoder is used to decode image frames from a bit stream, and may include a search circuit 501, a sub-image circuit 502, a sub-slice circuit 503, a memory controller 504 and a processor 505. In step S403, the sub-image circuit 502 groups N tiles into P sub-images, which do not overlap with each other. The first sub-image may include Q columns of tiles, and the second sub-image may include R columns of tiles. N is an integer greater than 1, and P, Q, and R are positive integers. For example, if N=8, P=2, and Q=2, the sub-image circuit groups 8 tiles into 2 non-overlapping sub-images, and each sub-image is decoded by a single processor. The first sub-image includes 2 columns and 2 rows of tiles, and the second sub-image also includes 2 columns and 2 rows of tiles. Depending on the shape of the P sub-images, the number of columns and rows of tiles of these sub-images may be the same or different. In some embodiments, each of the P sub-images includes a plurality of tiles arranged in a rectangle, such as Figure 3 In other embodiments, the sub-images in the P sub-images include a plurality of tiles arranged in a non-rectangular shape, such as Fig.19 As shown, the following paragraphs will discuss in detail. In some embodiments, the sizes of 2 sub-images in the P sub-images may be the same (for example, sub-image 0 includes 8×8 CTUs arranged in a rectangular shape, and sub-image 1 includes 8×8 CTUs arranged in a rectangular shape). In other embodiments, the sizes of 2 sub-images in the P sub-images may be different (for example, sub-image 0 includes 8×8 CTUs arranged in a rectangular shape, and sub-image 1 includes 5×8 CTUs arranged in a rectangular shape).

[0102] The sub-slice circuit 503 is coupled to the sub-image circuit 502 so as to divide the first column of tiles in the Q columns of tiles into M sub-slices in step S405, where the M sub-slices do not overlap with each other, and M is an integer greater than 1. For example, if M=8, the sub-slice circuit 503 divides the first column of tiles into 8 non-overlapping sub-slices. Similarly, in step S405, the sub-slice circuit 503 may further divide the second column of tiles in the first sub-image into 8 or other numbers of non-overlapping sub-slices. In some embodiments, each of the M sub-slices includes a plurality of CTUs arranged in a rectangular shape, such as Figure 7 and Figure 8 As shown in the following paragraphs, the height of the rectangle may be equal to the height of the LCU, and the width of the rectangle may be equal to the width of the tile column, so that the decoding order and the display order match each other, thereby reducing the display latency. In other embodiments, the sub-slice of the M sub-slices includes multiple CTUs arranged in a non-rectangular shape, such as Fig.10 In some embodiments, the M sub-slices may include 2 sub-slices of the same size (for example, each of the M sub-slices includes multiple 4x1 CTUs arranged as Figure 8In other embodiments, the M sub-slices may include two sub-slices of different sizes (eg, Fig.10 The sizes of sub-slice 0 and sub-slice 4 in the example are different).

[0103] In step S405, the sub-slice circuit 503 may further divide the first column of tiles in the R column of tiles into O sub-slices, where the O sub-slices do not overlap with each other, and O is an integer greater than 1. For example, if O=8, the sub-slice circuit 503 divides the first column of tiles into 8 non-overlapping sub-slices. Similarly, in step S405, the sub-slice circuit 503 may further divide the second column of tiles in the second sub-image into 8 or other numbers of non-overlapping sub-slices. In some embodiments, each of the O sub-slices includes a plurality of CTUs arranged in a rectangular shape, such as Fig.11 and Fig.12 In other embodiments, a sub-slice in the O sub-slices includes a plurality of CTUs arranged in a non-rectangular arrangement, such as Figure 3 In some embodiments, the size of two of the O sub-slices may be the same (e.g., each of the O sub-slices includes a plurality of 3×1 CTUs arranged as Fig.11 In other embodiments, the sizes of two of the O sub-slices may be different (e.g., Fig.11 , sub-slice 1-0 includes 3x1 CTUs arranged in a rectangle, while sub-slice 1-1 includes 2x1 CTUs arranged in a rectangle).

[0104] In step S407, the search circuit 501 searches for N block start points of N blocks from the bitstream, and outputs the N block start points to the processor 505. Each of the N blocks includes a plurality of LCUs. For example, if N=8, the search circuit 501 searches for 8 block start points of 8 blocks from the bitstream.

[0105] In step S411, the memory controller 504 is coupled to the memory 506 to obtain first tile information from the memory before decoding a current sub-slice of the M sub-slices in the first sub-image. The first tile information may include a first context, a first offset, a first arithmetic decoding range, and a first number of decoded bits after the previous sub-slice of the M sub-slices is decoded. For example, in the HEVC standard, the first context may come from a ctxTable, the first range may be an ivlRange parameter, and the first offset may be an ivlOffset parameter. During decoding, the bit stream may be decoded according to the first context to generate a bit value, and the bit value falls within a first range. The bit value and the first offset are compared to generate a decoded bit. For example, if the bit value is greater than the first offset, the decoded bit may be 1, and if the bit value is less than the first offset, the decoded bit may be 0. The memory 506 may be located inside or outside the video decoder 5. The processor 505 (first processor) can receive a bit stream and can be coupled to the sub-slice circuit 503 and the search circuit 501 to receive M sub-slices and N tile start points. The processor 505 can be coupled to a random access memory (RAM) 507 for temporarily storing data. In addition, in step S413, the processor 505 is coupled to the memory controller 504 to decode the current sub-slice of the M sub-slices according to the first tile information to generate a decoded bit stream. In step S417, after completing the decoding of the current sub-slice of the M sub-slices, the memory controller 504 stores the second tile information in the memory. The second tile information may include a second context, a second offset, a second arithmetic decoding range, and a second decoded bit number after the current sub-slice of the M sub-slices is decoded. The interpretation of the second context, the second offset, and the second range can be similar to the first context, the first offset, and the first range, respectively, and will not be repeated here. In step S419, if the last CTU / SB of the sub-image has not been reached, the processor 505 of the video decoder will continue to loop (S409 to S419) until the decoding of the last CTU / SB of the sub-image is completed. The reconstruction buffer 508 is coupled to the processor 505 for buffering the decoded bit stream and transmitting the decoded bit stream to the display system 509 for display on the display panel 510.

[0106] Similarly, the memory controller 504 may obtain the third tile information from the memory before decoding the current sub-slice of the O sub-slices in the second sub-image. The third tile information may include a third context, a third offset, a third arithmetic decoding range, and a third number of decoded bits after the previous sub-slice of the O sub-slices is decoded. The interpretation of the third context, the third offset, and the third range may be similar to the first context, the first offset, and the first range, respectively, and will not be repeated here. Figure 6 is a block diagram of another low-latency video decoder system 6. The difference between the low-latency video decoder system 6 and the low-latency video decoder system 5 is that the low-latency video decoder system 6 further includes a processor 611 (also referred to as a second processor). The rest of the low-latency video decoder system 5 and the low-latency video decoder system 6 are similar, and their descriptions are not repeated here. Figure 6 As shown, the processor 611 can receive a bit stream and can be coupled to the sub-slice circuit 503 and the search circuit 501 to receive O sub-slices and N tile start points. In addition, the processor 611 can be coupled to the memory controller 504 to decode the current sub-slice in the O sub-slices according to the third tile information to generate a decoded bit stream. When the decoding of the current sub-slice in the O sub-slices is completed, the memory controller 504 further stores the fourth tile information in the memory 506. The fourth tile information may include a fourth context, a fourth offset, a fourth arithmetic decoding range, and a fourth number of decoded bits after the current sub-slice in the O sub-slices is decoded. The interpretation of the fourth context, the fourth offset, and the fourth range can be similar to the first context, the first offset, and the first range, respectively, and will not be repeated here. In step S419, if the last CTU / SB of the sub-image has not been reached, the processor 611 of the video decoder will continue to loop (S409 to S419) until the decoding of the last CTU / SB of the sub-image is completed. The reconstruction buffer 508 is coupled to the processor 505 and is used for buffering the decoded bit stream and transmitting the decoded bit stream to the display system 509 for display on the display panel 510 .

[0107] Please refer to Figure 1 , Figure 1 This is an embodiment of video decoding with tile columns=4 and tile rows=2. Figure 1 The arrows indicate the decoding order of the original image. The thick lines indicate the boundaries of tiles, and the thin lines indicate the boundaries of CTU / SB. The decoding order follows the direction of the arrows. The image is decoded in the tile raster scan order, and each tile is decoded in the CTU / SB raster scan order.

[0108] Figure 7Shows the low-latency decoding order (CTU / SB raster scan order) of a video decoder when a picture has tile columns = 4, tile rows = 2, subslice width = tile width, and subslice height = 1, using a single processor. The dashed lines represent the boundaries of subslices (32 subslices total). The widths of the subslices from left to right (e.g., subslices 0 to 3) are 4 CTU / SB, 4 CTU / SB, 4 CTU / SB, and 2 CTU / SB, respectively, and the heights are 4 CTU / SB, 4 CTU / SB, 4 CTU / SB, and 2 CTU / SB, respectively. The subslices can be decoded in the CTU / SB raster scan order indicated by the direction of the arrows. Therefore, the decoding order matches the display order, thereby reducing display latency and the size of the reconstruction buffer. Figure 7 The configuration of tile columns, tile rows, and sub-slices is only an example and is not intended to be limiting.

[0109] Figure 8 Shows an alternative low latency decoding order for a video decoder when a picture has tile columns = 4, tile rows = 2, and subslices are rectangular in various sizes, using a single processor. Subslices 0 to 15 include 4×2 CTU / SB, 4×2 CTU / SB, 3×2 CTU / SB, 2×2 CTU / SB, 4×2 CTU / SB, 4×2 CTU / SB, 3×2 CTU / SB, 2×2 CTU / SB, 4×1 CTU / SB, 4×1 CTU / SB, 3×1 CTU / SB, 2×1 CTU / SB, 4×3 CTU / SB, 4×3 CTU / SB, 3×3 CTU / SB, and 3×2 CTU / SB, respectively. Each subslice is rectangular. The decoding order follows the direction of the arrows, which is roughly consistent with the display order, thereby reducing display latency and reconstruction buffer size. Figure 8 The configuration of tile columns, tile rows, and sub-slices is only an example and is not intended to be limiting.

[0110] FIG. 9 is a flowchart of a method 9 for decoding an image frame. A video decoder can use the method 9 to decode Figure 7 and Figure 8 The method 9 includes steps S901 to S907. Any reasonable technical changes or step adjustments fall within the scope of the present invention. Steps S901 to S907 are explained as follows: Step S901: Divide the Q-row image blocks into S sub-slices, the S sub-slices include k groups of Q sub-slices, each group of Q sub-slices is divided from the first row to the Qth row, the S sub-slices do not overlap with each other, Q and k are positive integers, and S is an integer greater than 1;

[0111] Step S903: before decoding a current sub-slice among the S sub-slices, obtaining first picture block information from a memory;

[0112] Step S905: The processor decodes a current sub-slice among the S sub-slices according to the first tile information;

[0113] Step S907: When the current sub-slice among the S sub-slices is decoded, the second tile information is stored in a memory.

[0114] The following paragraphs refer to Figure 7 The image frames in the method are explained in 9. Figure 7 , Q=4, S=32, k=8. A single-processor video decoder may use method 9 to define subslices in an image without defining a sub-image. The video decoder divides 4 tile columns into 32 subslices (step S901). The 32 subslices do not overlap with each other and may be divided into 8 groups of subslices, each group of subslices including 4 subslices. The video decoder may decode each group of 4 subslices sequentially from the subslice in the first tile column to the subslice in the fourth tile column. The first group of 4 subslices sequentially includes subslices 0 to 3. The second group of 4 subslices sequentially includes subslices 4 to 7, the third group of 4 subslices sequentially includes subslices 8 to 11, the fourth group of 4 subslices sequentially includes subslices 12 to 15, the fifth group of 4 subslices sequentially includes subslices 16 to 19, the sixth group of 4 subslices sequentially includes subslices 20 to 23, the seventh group of 4 subslices sequentially includes subslices 24 to 23, and the eighth group of 4 subslices sequentially includes subslices 28 to 31. The video decoder may decode each group of 4 subslices sequentially from subslice 0 in the first tile row to subslice 3 in the fourth tile row. The decoding order of the second to eighth groups of 4 subslices may be similar to the first group of 4 subslices.

[0115] The following paragraphs refer to Figure 8 The image frames in the method are explained in 9. Figure 8 , Q=4, S=16, k=4. A single-processor video decoder may use method 9 to define subslices in an image without defining a sub-image. The video decoder divides 4 tile columns into 16 subslices (step S901). The 16 subslices do not overlap with each other and may be divided into 4 groups of subslices, each group of subslices including 4 subslices. The video decoder may decode each group of 4 subslices sequentially from a subslice in the first tile column to a subslice in the fourth tile column. The first group of 4 subslices sequentially includes subslices 0 to 3. The second group of 4 subslices sequentially includes subslices 4 to 7, the third group of 4 subslices sequentially include subslices 8 to 11, and the fourth group of 4 subslices sequentially include subslices 12 to 11. The video decoder may decode each group of 4 subslices sequentially from subslice 0 in the first tile column to subslice 3 in the fourth tile column. The decoding order of the second to fourth groups of 4 subslices may be similar to the first group of 4 subslices. Steps S903 to S907 are used to decode each sub-slice. The processor decodes k groups of Q sub-slices in sequence from the first column of tiles to the last column of tiles. The explanation is similar to Figure 4 For the sake of brevity, I will not go into details here.

[0116] Fig.10 Another low latency decoding order for a video decoder is shown, using a single processor, when a picture has tile columns C0 to C3, tile rows R0 to R4, and sub-slices 0 to 14 of various sizes. Sub-slice 0 includes 3 CTU / SBs, sub-slice 1 includes 6 CTU / SBs, sub-slice 2 includes 4 CTU / SBs, sub-slice 3 includes 6 CTU / SBs, sub-slice 4 includes 11 CTU / SBs, sub-slice 5 includes 13 CTU / SBs, sub-slice 6 includes 4 CTU / SBs, sub-slice 7 includes 2 CTU / SBs, sub-slice 8 includes 2 CTU / SBs, sub-slice 9 includes 13 CTU / SBs, sub-slice 10 includes 16 CTU / SBs, sub-slice 11 includes 3 CTU / SBs, sub-slice 12 includes 8 CTU / SBs, sub-slice 13 includes 5 CTU / SBs, and sub-slice 14 includes 8 CTU / SBs.

[0117] The following paragraphs refer to Fig.10 The image frames in the method are explained in 9. Fig.10 , Q=4, S=15, k=3. A single-processor video decoder may use method 9 to define subslices in an image without defining a sub-image. The video decoder divides 4 tile columns into 15 subslices (step S901). The 15 subslices do not overlap with each other and may be divided into 3 groups of subslices, each group of subslices including 4 subslices. The video decoder may decode each group of 4 subslices sequentially from a subslice in tile column C0 to a subslice in tile column C3. The first group of 4 subslices sequentially includes subslices 0 to 3. The second group of 4 subslices sequentially includes subslices 4 to 7, and the third group of 4 subslices sequentially includes subslices 8 to 11. The video decoder may decode each group of 4 subslices sequentially, from subslice 0 in tile column C0 to subslice 3 in tile column C3. The decoding order of the second to third groups of 4 subslices may be similar to that of the first group of 4 subslices. The S sub-slices may further include r sub-slices in w tile columns in the Q tile columns, where r≥w, r, w are positive integers. Fig.10 , Q=4, S=15, r=3, w=2. In this embodiment, the 15 subslices further include subslices 12 and 14 in tile column C0, and subslice 13 in tile column C3. After decoding 3 groups of 4 subslices (subslice 0 to subslice 11), the processor further decodes subslices 12, 13, and 14. That is, after decoding 3 groups of Q subslices, the processor further decodes 3 subslices. The decoding order is subslice 0, subslice 1, subslice 2... until the decoding of subslice 14 is completed. Fig.10 The configuration of tile columns, tile rows, and sub-slices is only an example and is not intended to be limiting.

[0118] Fig.11 Shows an alternative low latency decoding order for a video decoder using dual processors when the image has tile columns = 4, tile rows = 2, subslice width = tile width, and subslice height = 1 CTU / SB. The video decoder has parallel processing capabilities. In this example, the video decoder has 2 processors and can decode 2 tiles simultaneously. Different processors decode the CTU / SB of the blank background and the CTU / SB of the dotted background. The decoding order can follow the direction of the arrows. Each subslice is rectangular, the width of the subslice is the width of the tile, and the height of the subslice is 1 CTU / SB high. Each processor decodes in the raster scan order of the CTU / SB.

[0119] Fig.12 Shows an alternative low latency decoding order for a video decoder when an image has tile columns = 4, tile rows = 2, sub-slice width = tile width, and sub-slices of different heights, using dual processors. Different processors decode the CTU / SB of the blank background and the CTU / SB of the dotted grid. The decoding order can follow the direction of the arrows. The sub-slice is rectangular and the width of the sub-slice is the tile width. The height of the sub-slice is not limited to 1 CTU / SB. The first processor decodes the sub-image on the blank background in the raster scan order of the tile, and the decoding order corresponds to the sub-slice heights of 1 CTU / SB, 4 CTU / SB, 2 CTU / SB, and 2 CTU / SB. The second processor decodes the sub-image on the dotted grid in the raster scan order of the tile, and the decoding order corresponds to the sub-slice heights of 3 CTU / SB, 1 CTU / SB, 1 CTU / SB, and 3 CTU / SB.

[0120] Figure 3Another embodiment of the present invention is shown, in which the video decoder has two processors. Different processors decode the CTU / SB of the blank background and the CTU / SB of the dotted grid bottom respectively. The sub-slice sizes decoded by the left processor are shown below. Sub-slice 0-0 includes 3 CTU / SBs, sub-slice 0-1 includes 6 CTU / SBs, sub-slice 0-2 includes 13 CTU / SBs, sub-slice 0-3 includes 13 CTUs or SBs, slice 0-4 includes 8 CTU / SBs, sub-slice 0-5 includes 13 CTU / SBs, and sub-slice 0-6 includes 8 CTU / SBs. The sub-slice sizes decoded by the right processor are shown below. Subslice 1-0 includes 4 CTU / SBs, subslice 1-1 includes 6 CTU / SBs, subslice 1-2 includes 4 CTU / SBs, subslice 1-3 includes 2 CTU / SBs, subslice 1-4 includes 13 CTUs or SBs, subslice 1-5 includes 3 CTU / SBs, subslice 1-6 includes 3 CTU / SBs, subslice 1-7 includes 3 CTU / SBs, and subslice 1-8 includes 2 CTU / SBs.

[0121] Fig.13 is a flow chart of the method 13 for decoding an image frame. The video decoder can use the method 13 to decode Figure 3 The method 13 includes steps S131 to S137. Any reasonable technical changes or step adjustments fall within the scope of the present invention. Steps S131 to S137 are explained as follows:

[0122] Step S131: Divide the N image blocks into P sub-images, wherein the P sub-images do not overlap each other, and the first sub-image of the P sub-images includes Q columns of image blocks, where N is an integer greater than 1.

[0123] N is an integer greater than 1, and P and Q are positive integers;

[0124] Step S133: Divide the Q-column image block into M sub-slices, where the M sub-slices include k groups of Q sub-slices, each group of Q sub-slices is divided into the Q-column image block, and the M sub-slices do not overlap with each other, where M is an integer greater than 1, and k is a positive integer;

[0125] Step S135 : The processor decodes each group of Q sub-slices sequentially from the sub-slices of the first column of tiles to the sub-slices of the Qth column of tiles.

[0126] Step S137: After decoding k groups of Q sub-slices, the processor decodes r sub-slices.

[0127] The following paragraphs refer to Figure 3 Method 13 is used to interpret the image frame in the video decoder. A dual-processor video decoder can adopt method 13. Figure 3, N=8, P=2, the video decoder groups 8 tiles into 2 sub-images (sub-image 0 and sub-image 1), the first processor decodes sub-image 0, and the second processor decodes sub-image 1 (step S131). For simplicity, the decoding of sub-image 0 (blank background) will be discussed below, and the decoding of sub-image 1 (dotted background) can be performed according to similar principles.

[0128] In sub-image 0, Q=2, M=7, sub-image 0 includes 2 tile columns, the 2 tile columns are divided into 7 sub-slices (sub-slices 0-0 to 0-6), and the 7 sub-slices of each sub-slice do not overlap with each other (step S133). In addition, k=3, the 7 sub-slices include 3 groups of 2 sub-slices, and each group of 2 sub-slices is divided into 2 tile columns. The first group of 2 sub-slices includes sub-slice 0-0 and sub-slice 0-1, the second group of 2 sub-slices includes sub-slice 0-2 and sub-slice 0-3, and the third group of 2 sub-slices includes sub-slice 0-0 and sub-slice 0-1. The first processor sequentially decodes each group of 2 sub-slices from the sub-slice in the first tile column to the sub-slice in the Qth tile column (step S135). In this embodiment, the first processor sequentially decodes the first group of 2 sub-slices from sub-slice 0-0 to sub-slice 0-1, and then sequentially decodes the second group of 2 sub-slices from sub-slice 0-2 to sub-slice 0-1.

[0129] The M sub-slices may further include r sub-slices in w tile columns in the Q tile columns, where r≥w, r, w are positive integers. Figure 3 , the 7 subslices further include subslices 0-6 in the first tile row, and the first processor continues to decode subslices 0-6 after decoding 3 groups of 2 subslices (subslices 0-0 to subslices 0-6). That is, the first processor decodes r subslices after k groups of Q subslices.

[0130] Fig.14 Another low-latency decoding order for a video decoder is shown when a picture has tile columns = 5, tile rows = 1, sub-slice width = tile width, and sub-slice height = 1 CTU / SB, using 3 processors. The picture frame is divided into 3 sub-pictures, and each sub-picture is decoded by a corresponding processor. The decoding order can follow the direction of the arrows. The first sub-picture is represented by a blank background and is divided into sub-slices 0-0 to 0-15. The second sub-picture is represented by a dotted background and is divided into sub-slices 1-0 to 1-7. The third sub-picture is represented by a diagonal background and is divided into sub-slices 2-0 to 2-15. Each sub-slice has a width of the tile and a height of 1 CTU / SB. The first processor can decode the first sub-picture in the order of sub-slices 0-0 to 0-15, the second processor can decode the second sub-picture in the order of sub-slices 1-0 to 1-7, and the third processor can decode the third sub-picture in the order of sub-slices 2-0 to 2-15.

[0131] Fig.15 Another low-latency decoding order for a video decoder is shown when a picture has tile columns = 5, tile rows = 1, sub-slice width = tile width, and sub-slices with different heights, using 3 processors. The picture frame is divided into 3 sub-pictures, and each sub-picture is decoded by a corresponding processor. The decoding order can follow the direction of the arrows. The first sub-picture is represented by a blank background and is divided into sub-slices 0-0 to 0-5. The second sub-picture is represented by a dotted background and is divided into sub-slices 1-0 to 1-3. The third sub-picture is represented by a diagonal background and is divided into sub-slices 2-0 to 2-17. The decoding order can follow the direction of the arrows. The width of the sub-slice can be the width of the tile, but the height of the sub-slice is not the height of the CTU / SB. The sub-slices are rectangular. The first processor can decode the first sub-picture in the order of sub-slices 0-0 to 0-5, the second processor can decode the second sub-picture in the order of sub-slices 1-0 to 1-3, and the third processor can decode the third sub-picture in the order of sub-slices 2-0 to 2-7. Each processor can decode each individual sub-segment in CTU / SB raster order. Sub-slices 0-0 and 0-1 are 2 CTU / SBs high, sub-slices 0-2, 0-3, 0-4, 0-5, 1-1, and 1-3 are 3 CTU / SBs high, sub-slices 1-0, 1-2, 2-2, 2-3, 2-4, 2-5 are 1 CTU / SB high, and sub-slices 2-0 and 2-1 are 4 CTU / SBs.

[0132] Fig.16 Shows an alternative low-latency decoding order for a video decoder when the image has tile columns = 5, tile rows = 1, and has sub-slices of different sizes, using 3 processors. The image frame is divided into 3 sub-images, each of which is decoded by a corresponding processor. The first sub-image is represented by a blank background and is divided into sub-slices 0-0 to 0-6. The second sub-image is represented by a dotted background and is divided into sub-slices 1-0 to 1-3. The third sub-image is represented by a diagonal background and is divided into sub-slices 2-0 to 2-5. Fig.16 The sub-slice of the image includes a plurality of CTUs arranged in a non-rectangular shape. The first processor may decode the first sub-image in the order of sub-slices 0-0 to 0-6, the second processor may decode the second sub-image in the order of sub-slices 1-0 to 1-3, and the third processor may decode the third sub-image in the order of sub-slices 2-0 to 2-5. Each processor may decode each individual sub-segment in CTU / SB raster order.

[0133] Fig.17An alternative low-latency decoding order for a video decoder is shown when a picture has tile columns = 4, tile rows = 3, sub-slice width = tile width, and sub-slice height = 1 CTU / SB, using 3 processors. The picture frame is divided into 3 sub-pictures, and each sub-picture is decoded by a corresponding processor. The first sub-picture is represented by a blank background and is divided into sub-slices 0-0 to 0-9. The second sub-picture is represented by a dotted background and is divided into sub-slices 1-0 to 1-9. The third sub-picture is represented by a slashed background and is divided into sub-slices 2-0 to 2-11. The first processor may decode the first sub-picture in the order of sub-slices 0-0 to 0-9, the second processor may decode the second sub-picture in the order of sub-slices 1-0 to 1-9, and the third processor may decode the third sub-picture in the order of sub-slices 2-0 to 2-11. The decoding order may follow the direction of the arrows. The width of each sub-slice is the width of the tile and the height is 1 CTU / SB. Each processor may decode each individual sub-slice in CTU / SB raster scan order.

[0134] Fig.18 Shows an alternative low-latency decoding order for a video decoder when the image has tile columns = 4, tile rows = 3, and has sub-slices of different sizes, using 3 processors. The image frame is divided into 3 sub-images, each of which is decoded by a corresponding processor. The first sub-image is represented by a blank background and is divided into sub-slices 0-0 to 0-4. The second sub-image is represented by a dotted background and is divided into sub-slices 1-0 to 1-5. The third sub-image is represented by a diagonal background and is divided into sub-slices 2-0 to 2-7. Fig.18 The sub-slice of includes a plurality of CTUs arranged in a non-rectangular shape. The first processor may decode the first sub-image in the order of sub-slices 0-0 to 0-4, the second processor may decode the second sub-image in the order of sub-slices 1-0 to 1-5, and the third processor may decode the third sub-image in the order of sub-slices 2-0 to 2-7. Each processor may decode each individual sub-segment in CTU / SB raster order.

[0135] Method 13 can be applied to Figure 11-12 and embodiments of 14-18.

[0136] Fig.19 Shows an alternative low-latency decoding order for a video decoder when the image has tile columns = 4, tile rows = 3, and sub-slice sizes are different, using 3 processors. The image frame is divided into 3 sub-images, each of which is decoded by a corresponding processor. The first sub-image is represented by a blank background and is divided into sub-slices 0-0 to 0-3. The second sub-image is represented by a dotted background and is divided into sub-slices 1-0 to 1-10. The third sub-image is represented by a diagonal background and is divided into sub-slices 2-0 to 2-5. Fig.18The sub-slice of the image includes a plurality of CTUs arranged in a non-rectangular shape. The first processor may decode the first sub-image in the order of sub-slices 0-0 to 0-3, the second processor may decode the second sub-image in the order of sub-slices 1-0 to 1-10, and the third processor may decode the third sub-image in the order of sub-slices 2-0 to 2-5. Each processor may decode each individual sub-segment in CTU / SB raster order.

[0137] Fig. 20 is a flow chart of a method 20 for decoding an image frame. A video decoder may use the method 20 to decode Fig.19 The method 20 comprises steps S201 to S205, which are explained as follows:

[0138] Step S201: divide N blocks into P sub-images, the P sub-images do not overlap each other, and I sub-images in the P sub-images include I blocks distributed in Q columns of blocks, N

[0139] is an integer greater than 1, and I, P, and Q are positive integers;

[0140] Step S203: Divide I block into M sub-slices, where I block includes J1 sub-slice on the ath row and J2 sub-slice on the (a+hth)th row, and the M sub-slices do not overlap with each other.

[0141] M, a, J1, J2 are positive integers;

[0142] Step S205: The processor decodes J1 sub-slices before decoding J2 sub-slices.

[0143] The following paragraphs refer to Fig.19 The method 20 is explained by the image frame in the three-processor video decoder. Fig.19 , N=12, P=3, the video decoder groups 12 tiles into 3 sub-images (sub-image 0, sub-image 1 and sub-image 2), the first processor decodes sub-image 0, the second processor decodes sub-image 1, and the third processor decodes sub-image 2 (step S201). For sub-image 0, Q=2, I=3, sub-image 0 includes 3 tiles distributed in 2 tile columns. For sub-image 1, Q=3, I=5, sub-image 1 includes 5 tiles distributed in 3 tile columns. For sub-image 2, Q=3, I=3, sub-image 2 includes 3 tiles distributed in 3 tile columns. For simplicity, the following discussion will be about the decoding of sub-image 3 (diagonal line background), and the decoding of sub-image 0 (blank background) and sub-image 1 (dotted background) can be performed according to similar principles.

[0144] In addition, in step S203, M=6, the three tiles are divided into six sub-slices (sub-slices 2-0 to 2-5), and each of the six sub-slices does not overlap with each other. In addition, a=1, J1=4, h=1, J2=2, the two tiles on the first tile row of sub-image 2 include four sub-slices (sub-slice 2-0 to sub-slice 2-3), and the one tile on the second tile row of sub-image 2 includes two sub-slices (sub-slice 2-4 and sub-slice 2-5). J1 sub-slices include k1 groups of Q1 sub-slices, and J2 sub-slices include k2 groups of Q2 sub-slices, and k1, k2, Q1, and Q2 are positive integers less than or equal to Q. As Fig.19 As shown, k1=2, Q1=2, and sub-slices 2-0 to 2-3 include the first group of 2 sub-slices (sub-slice 2-0 and sub-slice 2-1). In addition, k2=0, Q2=0. In step S205, the processor first decodes J1 sub-slices and then decodes J2 sub-slices. Fig.19 The third processor decodes sub-slices 2-0 to 2-3 and then sub-slices 2-4 and 2-5 in a top-to-bottom tile order. The processor decodes each group of 2 sub-slices in sequence from the sub-slices in the first tile row to the sub-slices in the last tile row. Fig.19 The decoding order may be the first group of 2 subslices (subslices 2-0 and 2-1), and the second group of 2 subslices (subslices 2-2 and 2-3). The J1 subslices may further include the r1 subslices in the w1 column of the Q1 column, r1 ≥ w1, r1 and w1 are integers; the J2 subslices may further include the r2 subslices in the w2 column of the Q2 column, r2 ≥ w2, r2 and w2 are integers. Fig.19 , r1=0, w1=0, r2=2, w2=1, the second tile row of sub-image 2 includes sub-slices 2-4 and sub-slices 2-5, and the third processor can decode sub-slices 2-4 and sub-slices 2-5. The processor can decode r1 sub-slices after k1 groups of Q1 sub-slices, and decode r2 sub-slices after k2 groups of Q2 sub-slices.

[0145] The following paragraphs explain the display waiting time in each embodiment.

[0146] exist Figure 21 to Figure 24 , the picture is divided into 4 tile columns and 2 tile rows and decoded in normal tile order, where the thick lines mark tiles and the thin lines mark CTU / SB. Fig.21 and 22 The image on the left shows the status of the reconstruction buffer, where the squares with a slashed bottom represent CTUs / SBs that have been decoded and stored, and the blank squares represent CTUs / SBs that have not yet been decoded. Fig.21 and 22The image on the right shows the status of the display driver, where the squares with a slashed bottom represent the CTU / SB that have been displayed, and the blank squares represent the CTU / SB that have not yet been displayed. To ensure that the image is displayed correctly, the display driver must display the decoded CTU / SB only after the rightmost CTU / SB is decoded. Assuming that the decoding speed of each CTU / SB is fixed at t=tctu, the display speed of displaying the decoded CTU / SB is also fixed, and the time required to display the equivalent number of pixels of 1 decoded CTU / SB is equal to the time required to decode 1 CTU / SB. At t=t0, the decoding time starts. Fig.21 , t = t0 + 46 * tctu, 46 CTU / SB have been decoded and stored in the reconstruction buffer, and the display driver can start driving the pixels of the first row of decoded CTU / SB. Fig. 22 , t=t0+59*tctu, 59 CTU / SBs have been decoded and stored in the reconstruction buffer, and the display driver has completed driving the pixels of the first row of decoded CTU / SB.

[0147] According to the above example, the image is divided into tile columns = 4, tile rows = 2, and decoded using a single processor in CTU / SB raster scan order, with each sub-slice width being the tile width and height being 1 CTU / SB. At t = t0, decoding begins. Fig.23 , t=t0+13*tctu, the display driver starts to drive the first row of pixels. Fig.24 , t=t0+26*tctu, the display driver completes driving the first row of pixels.

[0148] from Fig. 22 and Fig.24 It can be seen that decoding the image in CTU / SB raster scan order through a single processor can reduce the decoding time by 33*tctu when displaying the same number of pixels compared to normal tile order.

[0149] exist Figure 25 to Figure 28 In another example, the image is divided into 4 tile columns and 1 tile row, and is decoded in normal tile order. Assume that the decoding speed of each CTU / SB is fixed at t=tctu, the display speed of the decoded CTU / SB is also fixed, and the time required to display the equivalent number of pixels of 1 decoded CTU / SB is equal to the time required to decode 1 CTU / SB. At t=t0, the decoding time starts and the video decoder starts decoding. Fig.25 , t=t0+90*tctu, the video decoder has completed decoding the rightmost CTU / SB, and the display driver can start outputting the pixels of the first row of decoded CTU / SB. Fig.26, t=t0+103*tctu, the square at the bottom of the right diagonal grid represents the displayed CTU / SB.

[0150] According to the above example, the image is divided into 4 tile columns and 1 tile row, and decoded using a single processor in CTU / SB raster scan order, with each sub-slice width being the tile width and each sub-slice height being the CTU / SB height. At t=t0, decoding starts. Fig. 27 , t=t0+13*tctu, the display driver starts to drive the first row of pixels. Fig.28 , t=t0+26*tctu, the display driver completes driving the first row of pixels.

[0151] from Fig.26 and Fig.28 It can be seen that decoding the image in CTU / SB raster scan order through a single processor can reduce the decoding time by 77*tctu when displaying the same number of pixels compared to normal tile order.

[0152] The embodiment of the present invention discloses a video decoder and a decoding method, which use sub-slices within a frame to match a decoding order and a display order, thereby reducing display delay.

[0153] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for decoding an image frame from a bit stream, characterized in that The method includes: Grouping N tiles of the image frame into P sub-images, wherein the P sub-images do not overlap each other, a first sub-image of the P sub-images includes Q rows of tiles, N is an integer greater than 1, and P and Q are positive integers; Divide a first row of image blocks in the Q rows of image blocks into M sub-slices, wherein the M sub-slices do not overlap with each other, and M is an integer greater than 1; Before decoding a current sub-slice among the M sub-slices, obtaining first tile information from a memory; A first processor decodes the current sub-slice among the M sub-slices according to the first tile information; and When the current sub-slice among the M sub-slices is decoded, the second picture block information is stored in the memory.

2. The method according to claim 1, characterized in that The method further includes searching the bitstream for N entry points of the N tiles, each of the N tiles including a plurality of largest coding units (LCUs).

3. The method according to claim 1, characterized in that Each of the P sub-images includes a plurality of tiles arranged in a rectangular shape.

4. The method according to claim 1, characterized in that A sub-image of the P sub-images includes a plurality of tiles arranged in a non-rectangular shape.

5. The method according to claim 1, characterized in that Each of the M sub-slices includes a plurality of LCUs arranged in a rectangular shape.

6. The method according to claim 5, characterized in that A height of the rectangle is equal to a height of an LCU, and a width of the rectangle is equal to a width of the first row of blocks in the Q rows of blocks.

7. The method according to claim 1, characterized in that A sub-slice among the M sub-slices includes a plurality of LCUs in a non-rectangular arrangement.

8. The method according to claim 1, characterized in that: The first tile information includes a first context, a first offset, a first arithmetic decoding range, and a first decoding bit number after decoding of a previous sub-slice among the M sub-slices is completed; and The second tile information includes a second context, a second offset, a second arithmetic decoding range, and a second decoding bit number after the current sub-slice in the M sub-slices is decoded.

9. The method according to claim 1, characterized in that: A second sub-image of the P sub-images includes R columns of blocks, where R is a positive integer; and The method further comprises: Divide a first row of image blocks in the R row of image blocks into O sub-slices, wherein the O sub-slices do not overlap with each other, and O is an integer greater than 1; Obtaining third tile information from the memory before decoding a current sub-slice of the O sub-slices; A second processor decodes the current sub-slice among the O sub-slices according to the third tile information; and After decoding of the current sub-slice among the O sub-slices is completed, the fourth block information is stored in the memory.

10. The method according to claim 1, characterized in that: The M sub-slices include k groups of Q sub-slices, each group of Q sub-slices is divided into Q columns, and k is a positive integer; and The method further includes: the processor sequentially decoding each group of Q sub-slices from a first column of tiles to a Qth column of tiles.

11. The method according to claim 10, characterized in that The processor decodes k groups of Q subslices sequentially from each group of Q subslices in a first row of tiles to a last row of tiles.

12. The method according to claim 10, characterized in that The M sub-slices further include r sub-slices in w columns of the Q columns, where r>=w, and r and w are positive integers.

13. The method according to claim 12, characterized in that The method further includes decoding, by the processor, the r sub-slices following the k groups of Q sub-slices.

14. The method according to claim 10, characterized in that Some of the M sub-slices occupy parts of 2 tiles in the same column of tiles.

15. The method of claim 1, wherein: The first sub-image includes I blocks distributed in the Q columns of blocks, where I is a positive integer; The I picture block includes J1 sub-slices on an a-th row of the I picture block and J2 sub-slices on an (a+h)-th row of the I picture block, where h, a, J1, and J2 are positive integers; and The method further includes: the processor decoding the J1 sub-slices before decoding the J2 sub-slices.

16. The method of claim 15, wherein: The J1 sub-slices include k1 groups of Q1 sub-slices, the J2 sub-slices include k2 groups of Q2 sub-slices, and k1, k2, Q1, and Q2 are positive integers less than or equal to Q; and The processor decoding the J1 sub-slice before decoding the J2 sub-slices includes: The processor decodes each group of Q1 subslices sequentially from a subslice in a first column of tiles in the a-th row to a subslice in a Q-th column of tiles in the a-th row; and The processor decodes each group of Q2 subslices sequentially from a subslice in a first column of tiles in the (a+h)th row to a subslice in a Q2th column of tiles in the (a+h)th row.

17. The method of claim 15, wherein: The J1 sub-slices further include r1 sub-slices in the w1 column of the Q1 column, where r1>=w1, r1 and w1 are integers; and The J2 sub-slices further include r2 sub-slices in the w2 column of the Q2 column, where r2>=w2, and r2 and w2 are integers.

18. The method according to claim 17, characterized in that The method further comprises: The processor first decodes the k1 group of Q1 sub-slices and then decodes the r1 sub-slices; and The processor first decodes the k2 groups of Q2 sub-slices and then decodes the r2 sub-slices.

19. A video decoder for decoding an image frame from a bit stream, characterized in that The video decoder includes: a sub-image circuit for grouping the N image blocks of the image frame into P sub-images, wherein the P sub-images do not overlap each other, a first sub-image of the P sub-images includes Q columns of image blocks, N is an integer greater than 1, and P and Q are positive integers; a sub-slice circuit coupled to the sub-image circuit, for dividing a first row of image blocks in the Q rows of image blocks into M sub-slices, wherein the M sub-slices do not overlap with each other, and M is an integer greater than 1; a memory controller coupled to a memory, configured to obtain first tile information from the memory before decoding a current sub-slice among the M sub-slices; and a first processor, coupled to the sub-slice circuit and the memory controller, for decoding the current sub-slice among the M sub-slices according to the first tile information; After the memory controller completes decoding of the current sub-slice among the M sub-slices, The second block information is stored in the memory.

20. The video decoder of claim 19, wherein The video decoder further comprises: A search circuit is coupled to the first processor to search N block start points of the N blocks from the bit stream and output the N block start points to the processor, wherein each of the N blocks includes a plurality of largest coding units (LCUs).

21. The video decoder of claim 19, wherein: A sub-image of the P sub-images includes a plurality of tiles arranged in a non-rectangular shape.

22. The video decoder of claim 19, wherein: Each of the M sub-slices includes a plurality of LCUs arranged in a rectangular shape.

23. The video decoder of claim 22, wherein A height of the rectangle is equal to a height of an LCU, and a width of the rectangle is equal to a width of the first row of tiles.

24. The video decoder of claim 19, wherein: A sub-slice among the M sub-slices includes a plurality of LCUs in a non-rectangular arrangement.

25. The video decoder of claim 19, wherein: The first tile information includes a first context, a first offset, a first arithmetic decoding range, and a first decoding bit number after decoding of an end LCU of a previous sub-slice among the M sub-slices is completed; and The second tile information includes a second context, a second offset, a second arithmetic decoding range, and a second decoding bit number after decoding of an end LCU of the current sub-slice among the M sub-slices is completed.

26. The video decoder of claim 19, wherein: A second sub-image of the P sub-images includes R columns of blocks, where R is a positive integer; and The sub-slice circuit further divides a first column of the R columns of blocks into O sub-slices, the O sub-slices do not overlap each other, and O is an integer greater than 1; The memory controller further obtains third block information from the memory before decoding a current sub-slice among the O sub-slices; The video decoder further includes a second processor coupled to the sub-slice circuit and the memory controller to decode the current sub-slice of the O sub-slices according to the third tile information; and When decoding of the current sub-slice among the O sub-slices is completed, the memory controller further stores fourth tile information in the memory.

27. A method for decoding an image frame from a bit stream, characterized in that The method includes: Divide the Q columns of tiles into S sub-slices, the S sub-slices include k groups of Q sub-slices, Each group of Q sub-slices is divided from a first column to a Qth column, the S sub-slices do not overlap each other, Q and k are positive integers, and S is an integer greater than 1; Before decoding a current sub-slice among the S sub-slices, obtaining first tile information from a memory; A processor decodes the current sub-slice among the S sub-slices according to the first tile information; and After decoding of the current sub-slice among the S sub-slices is completed, the second picture block information is stored in the memory.

28. The method of claim 27, wherein The method further includes the processor sequentially decoding each group of Q subslices from a subslice in a first column of tiles to a subslice in a Qth column of tiles.

29. The method of claim 27, wherein: The processor decodes k groups of Q subslices sequentially from a group of Q subslices in a first row of tiles to a group of Q subslices in a last row of tiles.

30. The method of claim 27, wherein: The S sub-slices further include r sub-slices in the w columns in the Q columns, where r≥w, r, w are positive integers.

31. The method of claim 30, wherein The processor first decodes the k groups of Q sub-slices and then decodes the r sub-slices.

32. The method of claim 27, wherein: Some of the S sub-slices occupy parts of 2 tiles in the same column of tiles.

33. The method of claim 27, wherein: Each of the S sub-slices includes a plurality of LCUs arranged in a rectangular shape.

34. The method of claim 33, wherein A height of the rectangle is equal to a height of an LCU, and a width of the rectangle is equal to a width of a row of tiles.

35. The method of claim 27, wherein: A sub-slice among the S sub-slices includes a plurality of LCUs in a non-rectangular arrangement.

36. The method of claim 27, wherein: The first tile information includes a first context, a first offset, a first arithmetic decoding range, and a first decoding bit number after decoding of a previous sub-slice among the S sub-slices is completed; and The second tile information includes a second context, a second offset, a second arithmetic decoding range, and a second decoding bit number after the current sub-slice among the S sub-slices is decoded.

Images