Image encoding device, image encoding method and program, image decoding device, image decoding method and program

Abstract

To reduce an encoding amount of a bit stream generated by eliminating redundancy of a syntax in regard to a sub-picture.SOLUTION: An image encoding device having encoding means of dividing an image into a plurality of sub-pictures and encoding so as to independently decode each sub-picture, is constructed by: first determination means of determining whether or not each sub-picture constructing the image is defined as only one rectangle; second determination means of determining a number of basic pixels as vertical and horizontal sizes of a grid constructing each sub-picture; applying means of applying the number to each grid divided with the number of basic pixels; and third determination means of determining the maximum number of the sub-pictures which are the number applied in the applying means. Each sub-picture is a collection of the grids applied the same number, and encodes a value obtained by subtracting 2 from the maximum number determined by the third determination means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image encoding device, an image encoding method and program, an image decoding device, an image decoding method and program, and image coded data, and more particularly to an image encoding and decoding method in which a picture is divided into rectangles and can be extracted as independent coded data. [Background technology]

[0002] The High Efficiency Video Coding (HEVC) coding method (hereafter referred to as HEVC) is known as a coding method for compressing and recording moving images. To improve coding efficiency, HEVC uses basic blocks larger than conventional macroblocks (16x16 pixels). These large basic blocks are called Coding Tree Units (CTUs) and can be up to 64x64 pixels in size. CTUs are further divided into sub-blocks, which serve as units for prediction and transformation.

[0003] Furthermore, HEVC allows a picture to be divided into multiple tiles or slices for encoding. There is little data dependency between tiles or slices, making it possible to perform encoding and decoding processes in parallel. One of the major advantages of dividing a picture into tiles and slices is that it allows parallel processing to be performed on a multi-core CPU or the like, thereby reducing processing time. Patent Document 1 discloses technology related to tiles and slices.

[0004] In recent years, efforts have begun to internationally standardize a more efficient coding method as a successor to HEVC. The Joint Video Experts Team (JVET) was established between ISO / IEC and ITU-T, and standardization is underway as the Versatile Video Coding (VVC) coding method (hereinafter referred to as VVC). VVC has sub-pictures, which are configured to contain one or more slices and form a rectangle. These sub-pictures can divide a picture into one or more parts, and each sub-picture can be processed as independent coded data. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-11638 Summary of the Invention [Problem to be solved by the invention]

[0006] For sub-pictures in VVC, the syntax defines the value representing the maximum number of sub-pictures, the basic pixel count such as the vertical and horizontal sizes of the grid that defines the boundary positions of the sub-pictures, the ID of each sub-picture, and the control flags corresponding to each sub-picture.

[0007] The syntax relating to subpictures still contains many redundant parts, which increases the amount of code.

[0008] Therefore, the present invention has been made to solve the above-mentioned problems, and aims to reduce the amount of code in the generated bitstream by eliminating the redundancy in the syntax related to subpictures. [Means for solving the problem]

[0009] In order to solve the above problems, an image coding apparatus according to the present invention has the following configuration. Each consists of one or more slicesAn image coding device that codes an image including one or more sub-pictures into a bitstream, comprising: a first coding means that codes a flag related to the presence of sub-picture information into a sequence parameter set of the bitstream; a second coding means that, when the flag has a value indicating the presence of sub-picture information, codes a syntax element related to the number of sub-pictures included in the image and first information related to IDs of the sub-pictures included in the image into the sequence parameter set of the bitstream; and a syntax element used to specify an arrangement of the sub-pictures included in the image into the sequence parameter set of the bitstream. Information of a given size, which is equal to the result of dividing the given size by 64 and subtracting 1 and a fourth encoding means for encoding a loop_filter_across_subpic_enabled_flag related to a filtering process into the sequence parameter set of the bitstream, wherein the second information and the loop_filter_across_subpic_enabled_flag are encoded into the sequence parameter set of the bitstream only when the value of the syntax element to be encoded into the sequence parameter set of the bitstream is 1 or greater, a state in which the value of the syntax element is 1 or greater corresponds to the number of subpictures included in the image being 2 or greater, and when the flag does not have a value indicating the presence of subpicture information, the syntax element and the first information are not encoded into the sequence parameter set, and when the second information and the loop_filter_across_subpic_enabled_flag are encoded, the second information is encoded before the loop_filter_across_subpic_enabled_flag in the sequence parameter set.

[0010] The image decoding device of the present invention has the following configuration: Each consists of one or more slicesAn image decoding device that decodes a bitstream obtained by encoding an image including one or more subpictures, comprising: first decoding means that decodes a flag related to the presence of subpicture information from a sequence parameter set of the bitstream; second decoding means that, when the flag has a value indicating the presence of subpicture information, decodes from the sequence parameter set of the bitstream a syntax element related to the number of subpictures included in the image and first information related to IDs of the subpictures included in the image; and second decoding means that is used to identify the arrangement of the subpictures included in the image from the sequence parameter set of the bitstream. Information of a given size, which is equal to the result of dividing the given size by 64 and subtracting 1 the second information and the loop_filter_across_subpic_enabled_flag are decoded from the sequence parameter set of the bitstream only when the value of the syntax element decoded from the sequence parameter set of the bitstream is 1 or greater, the state in which the value of the syntax element is 1 or greater corresponds to the number of subpictures included in the image being 2 or greater, the syntax element and the first information are not decoded from the sequence parameter set when the flag does not have a value indicating the presence of subpicture information, and when the second information and the loop_filter_across_subpic_enabled_flag are decoded, the second information is decoded from the sequence parameter set before the loop_filter_across_subpic_enabled_flag.

[0011] The image coding method of the present invention has the following configuration: Each consists of one or more slicesAn image coding method for coding an image including one or more sub-pictures into a bitstream, comprising: a first coding step of coding a flag related to the presence of sub-picture information into a sequence parameter set of the bitstream; a second coding step of coding, if the flag has a value indicating the presence of sub-picture information, a syntax element related to the number of sub-pictures included in the image and first information related to IDs of the sub-pictures included in the image into the sequence parameter set of the bitstream; and a second coding step of coding, if the flag has a value indicating the presence of sub-picture information, a syntax element related to the number of sub-pictures included in the image and first information related to IDs of the sub-pictures included in the image into the sequence parameter set of the bitstream, Information of a given size, which is equal to the result of dividing the given size by 64 and subtracting 1 and a fourth encoding step of encoding a loop_filter_across_subpic_enabled_flag related to a filtering process into the sequence parameter set of the bitstream, wherein the second information and the loop_filter_across_subpic_enabled_flag are encoded into the sequence parameter set of the bitstream only when the value of the syntax element to be encoded into the sequence parameter set of the bitstream is 1 or greater, a state in which the value of the syntax element is 1 or greater corresponds to the number of subpictures included in the image being two or more, and when the flag does not have a value indicating the presence of subpicture information, the syntax element and the first information are not encoded into the sequence parameter set, and when the second information and the loop_filter_across_subpic_enabled_flag are encoded, the second information is encoded before the loop_filter_across_subpic_enabled_flag in the sequence parameter set.

[0012] The image decoding method of the present invention has the following configuration: Each consists of one or more slicesAn image decoding method for decoding a bitstream obtained by encoding an image including one or more subpictures, the method comprising: a first decoding step of decoding a flag relating to the presence of subpicture information from a sequence parameter set of the bitstream; a second decoding step of decoding, if the flag has a value indicating the presence of subpicture information, from the sequence parameter set of the bitstream, a syntax element relating to the number of subpictures included in the image and first information relating to IDs of the subpictures included in the image; and a second decoding step of decoding, from the sequence parameter set of the bitstream, a syntax element used to identify the arrangement of the subpictures included in the image. Information of a given size, which is equal to the result of dividing the given size by 64 and subtracting 1 and a fourth decoding means for decoding loop_filter_across_subpic_enabled_flag related to a filtering process from the sequence parameter set of the bitstream, wherein the second information and the loop_filter_across_subpic_enabled_flag are decoded from the sequence parameter set of the bitstream only when the value of the syntax element decoded from the sequence parameter set of the bitstream is 1 or greater, a state in which the value of the syntax element is 1 or greater corresponds to the number of subpictures included in the image being two or more, and when the flag does not have a value indicating the presence of subpicture information, the syntax element and the first information are not decoded from the sequence parameter set, and when the second information and the loop_filter_across_subpic_enabled_flag are decoded, the second information is decoded from the sequence parameter set before the loop_filter_across_subpic_enabled_flag. [Effects of the Invention]

[0013] According to the present invention, it is possible to efficiently code syntax relating to sub-pictures that make up an image, thereby improving coding efficiency. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 is a block diagram showing a configuration of an image encoding device. [Figure 2] FIG. 1 is a block diagram showing a configuration of an image decoding device. [Figure 3] 1 is a flowchart showing an image encoding process in an image encoding device. [Figure 4] 10 is a flowchart showing an image decoding process in the image decoding device. [Figure 5] FIG. 2 is a block diagram illustrating an example of a hardware configuration. [Figure 6] FIG. 1 is a diagram illustrating a bitstream structure. [Figure 7] FIG. 10 is a diagram illustrating an example of dividing an image. [Figure 8] FIG. 10 is a diagram illustrating an example of dividing an image. [Figure 9] FIG. 1 is a diagram illustrating a bitstream structure. [Figure 10] FIG. 10 is a diagram illustrating an example of dividing an image. [Figure 11] FIG. 1 is a diagram illustrating a bitstream structure. DETAILED DESCRIPTION OF THE INVENTION

[0015] The present invention will be described in detail below based on preferred embodiments thereof with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[0016] (Embodiment 1) An embodiment of the present invention will now be described with reference to the drawings. Fig. 1 is a block diagram showing an image coding apparatus according to this embodiment. In Fig. 1, reference numeral 101 denotes a terminal for inputting image data.

[0017] An image division unit 102 divides an input image into one or more tile rows and one or more tile columns. A tile is a set of consecutive basic blocks that covers a rectangular area in the image. The image division unit 102 further divides the tiles into one or more bricks. Each brick is a rectangle made up of one or more basic block rows, which are rows of basic blocks within the tile. The image division unit 102 divides the image into slices, each made up of one or more tiles or one or more bricks within a tile. A slice is the basic unit of encoding, and header information, such as information indicating the type of slice, is added to each slice.

[0018] In this embodiment, a picture is divided into two sub-pictures as shown in Fig. 8. For ease of explanation, the sub-pictures, slices, tiles, and bricks are assumed to have the same vertical and horizontal sizes and positions, but this is not limiting.

[0019] A block dividing unit 103 divides the basic block row image output from the image dividing unit 102 into a plurality of basic blocks, and outputs the image in units of basic blocks to the subsequent stage.

[0020] Reference numeral 104 denotes a prediction unit, which determines sub-block division of image data in units of basic blocks, and performs intra-frame prediction (intra-prediction) or inter-frame prediction (inter-prediction) on a sub-block basis to generate predicted image data. Intra-prediction across bricks and motion vector prediction are not performed. Furthermore, a prediction error is calculated from the input image data and the predicted image data, and output. Information necessary for prediction, such as sub-block division, prediction mode, motion vector, etc., is also output along with the prediction error. Hereinafter, this information necessary for prediction will be referred to as prediction information.

[0021] A transform / quantization unit 105 performs orthogonal transform on the prediction errors in subblock units to obtain transform coefficients, and then quantizes the transform coefficients to obtain quantized coefficients. An inverse quantization / inverse transform unit 106 inverse quantizes the quantized coefficients output from the transform / quantization unit 105 to reproduce transform coefficients, and then performs inverse orthogonal transform on the quantized coefficients to reproduce the prediction errors. A frame memory 108 stores the reproduced image data.

[0022] Reference numeral 107 denotes an image reproduction unit. Based on the prediction information output from the prediction unit 104, the image reproduction unit 107 generates predicted image data by appropriately referencing a frame memory 108, and generates and outputs reconstructed image data from this and the input prediction error. Reference numeral 109 denotes an in-loop filter unit. The in-loop filter unit 109 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the reconstructed image, and outputs the filtered image.

[0023] Reference numeral 110 denotes a coding unit. The coding unit 110 codes the quantization coefficients output from the transform / quantization unit 105 and the prediction information output from the prediction unit 104 to generate and output coded data. Reference numeral 111 denotes an integrated coding unit. The integrated coding unit 111 receives segmentation information from the image segmentation unit 102 and generates header coded data. This is further combined with the coded data output from the coding unit 110 to form and output a bitstream. Reference numeral 112 denotes a terminal which outputs the bitstream generated by the integrated coding unit 111 to the outside.

[0024] The image encoding operation in the image encoding device of embodiment 1 will be described below. In this embodiment, moving image data is input in frame units, but still image data for one frame may also be input. In addition, for ease of explanation, this embodiment will only describe intra-prediction encoding processing, but this is not limited to this and the invention can also be applied to inter-prediction encoding processing. Furthermore, in this embodiment, for the sake of explanation, the block division unit 103 will divide the image into basic blocks of 64 x 64 pixels, but this is not limited to this.

[0025] In this embodiment, one frame of image data input from terminal 101 is first divided into sub-pictures by image division unit 102, as shown in FIG. 8. In this embodiment, a case will be described in which one image data is divided into two sub-pictures, each consisting of 18 grids. The vertical and horizontal dimensions of one grid are 256 x 256, and the vertical and horizontal dimensions of the sub-picture are 768 x 768. An INDEX value of 0 is set for the grid that constitutes the left sub-picture, and an INDEX value of 1 is set for the grid that constitutes the right sub-picture. In this way, in this embodiment, one image data is divided into multiple grids and divided into multiple sub-pictures based on these multiple grids. Furthermore, an INDEX value is assigned to each grid as a number, and the same INDEX value is assigned to the grids that constitute one sub-picture.

[0026] In this embodiment, each subpicture is composed of one slice and one tile of one brick. Information on the sizes of these subpictures and slices is sent as division information to the integrated encoding unit 111. Furthermore, each brick is divided into basic block row images, which are image data in units of basic block rows, and sent to the block division unit 103.

[0027] The block division unit 103 divides the input basic block row image into a plurality of basic blocks, and outputs an image in units of basic blocks to the prediction unit 104. In this embodiment, it is assumed that an image in units of basic blocks of 64×64 pixels is output.

[0028] The prediction unit 104 performs prediction processing on the image data in units of basic blocks input from the block division unit 103. Specifically, the prediction unit 104 determines sub-block division for dividing the basic blocks into smaller sub-blocks, and further determines an intra-prediction mode such as horizontal prediction or vertical prediction for each sub-block.

[0029] Prediction image data is generated from the determined intra prediction mode and coded pixels, and a prediction error is generated from the input image data and the predicted image data, and output to the transform / quantization unit 105. Information such as sub-block division and intra prediction mode is output to the coding unit 110 and image reproduction unit 107 as prediction information.

[0030] The transform / quantization unit 105 performs orthogonal transform / quantization on the input prediction errors to generate quantized coefficients. First, an orthogonal transform process corresponding to the size of the sub-block is performed to generate orthogonal transform coefficients, and then the orthogonal transform coefficients are quantized to generate quantized coefficients. The generated quantized coefficients are output to the encoding unit 110 and the inverse quantization / inverse transform unit 106.

[0031] The inverse quantization and inverse transform unit 106 inverse quantizes the input quantized coefficients to reproduce transform coefficients, and then performs inverse orthogonal transform on the reproduced transform coefficients to reproduce prediction errors. The reproduced prediction errors are output to the image reproduction unit 107.

[0032] The image reproduction unit 107 reproduces a predicted image by appropriately referring to the frame memory 108 based on the prediction information input from the prediction unit 104. Then, image data is reproduced from the reproduced predicted image and the reproduced prediction error input from the inverse quantization and inverse transform unit 106, and is input to and stored in the frame memory 108.

[0033] The in-loop filter unit 109 reads out the reconstructed image from the frame memory 108 and performs in-loop filtering such as deblocking filtering on the image, and then inputs the filtered image back into the frame memory 108 and stores it again.

[0034] The encoding unit 110 entropy-encodes, on a block-by-block basis, the quantization coefficients generated by the transform / quantization unit 105 and the prediction information input from the prediction unit 104, to generate coded data. There is no particular specification as to the entropy coding method, but Golomb coding, arithmetic coding, Huffman coding, etc. can be used. The generated coded data is output to the integrated coding unit 111.

[0035] The integrated encoding unit 111 receives segmentation information from the image segmentation unit 102, generates encoded header data, and multiplexes the encoded data input from the encoding unit 110 to form a bit stream. Finally, the bit stream is output to the outside from a terminal 112.

[0036] The format of VVC-based coded data coded by the image coding device of this embodiment is shown in Fig. 6. The coded data in Fig. 6 first contains a sequence parameter set, which is header information containing information related to the coding of a sequence. This is followed by a picture parameter set, which is header information containing information related to the coding of a picture, a slice header, which is header information containing information related to the coding of a slice, and coded data for each brick.

[0037] The sequence parameter set contains image size information such as pic_width_in_luma_samples and pic_height_in_luma_samples. These represent the number of horizontal and vertical luminance pixels of the image, respectively. In this embodiment, since the image shown in Figure 8 is encoded, pic_width_in_luma_samples is 1536 and pic_height_in_luma_samples is 768. In addition, basic block data division information contains log2_ctu_size_minus2, which indicates the size of the basic block. The number of vertical and horizontal pixels of a basic block is expressed as 1<<(log2_ctu_size_minus2+2). In this embodiment, the basic block is 64 x 64 pixels, so the value of log2_ctu_size_minus2 is 4.

[0038] Further subpicture information includes subpics_present_flag, which indicates whether or not the subpicture is divided. When subpics_present_flag is 1, it indicates that the subpicture is divided into one or more subpictures, and when it is 0, it indicates that the subpicture is not divided.

[0039] When subpics_present_flag is 1, the following information is also coded: For example, information coded at this time includes max_subpics_minus2, which indicates the maximum number of subpictures. Furthermore, information coded at this time includes subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1, which indicate the basic number of pixels in a grid. Furthermore, information coded at this time includes subpic_grid_idx[i][j] as an INDEX value. Furthermore, information coded at this time includes subpic_treated_as_pic_flag[i] and loop_filter_across_subpic_enabled_flag[i], which are related to filtering.

[0040] Conventionally, the maximum number of subpictures has been expressed as "maximum number - 1." This was designed so that the minimum amount of code would be used when the maximum number of subpictures was 1, i.e., when there was one subpicture, and the subpicture was not actually divided. In contrast, this embodiment expresses the maximum number of subpictures as "maximum number - 2," so that when divided into two subpictures, the value of max_subpics_minus2 becomes 0, allowing the minimum amount of code to be used.

[0041] The picture parameter set contains tile data information, brick data information, slice data information, and basic block data information. The slice header contains slice data information, followed by the coded data for each brick.

[0042] 3 is a flowchart showing the encoding process in the image encoding device according to embodiment 1. First, in step S301, the image dividing unit 102 divides the image into tiles, bricks, and slices as described above, and sends the division information to the integrated encoding unit 111. In step S309, the division information is converted into header information by the integrated encoding unit 111 and encoded into a bitstream. The image dividing unit 102 also divides the image into sub-pictures and sends them to the block dividing unit 103.

[0043] In step S302, the block division unit 103 divides the basic block row image into basic block units. In step S303, the prediction unit 104 performs prediction processing on the basic block unit image data generated in step S302, and generates prediction information such as sub-block division information and intra prediction mode, as well as predicted image data. Furthermore, a prediction error is calculated from the input image data and the predicted image data.

[0044] In step S304, the transform / quantization unit 105 performs orthogonal transform on the prediction error calculated in step S303 to generate transform coefficients, and performs quantization to generate quantized coefficients.

[0045] In step S305, the inverse quantization and inverse transform unit 106 inverse quantizes the quantized coefficients generated in step S304 to regenerate transform coefficients, and then performs inverse orthogonal transform on the transform coefficients to regenerate prediction errors.

[0046] In step S306, the image reproduction unit 107 reproduces a predicted image based on the prediction information generated in step S303, and further reproduces image data from the reproduced predicted image and the prediction error generated in step S305.

[0047] In step S307, the encoding unit 110 encodes the prediction information generated in step S303 and the quantized coefficients generated in step S304 to generate coded data.

[0048] In step S308, the image encoding device determines whether encoding of all basic blocks in the slice has been completed. If encoding has been completed, the image encoding device proceeds to step S309; ​​if not, the image encoding device returns to step S302 for the next basic block.

[0049] In step S309, the integration encoding unit 111 generates header information based on the division information sent from the image division unit 102 and encodes it.

[0050] A specific example of this embodiment is shown based on the image division in Figure 8. In this embodiment, the image is divided into two subpictures, so subpics_present_flag is set to 1. Next, max_subpics_minus2, which indicates the maximum number of subpictures, is set to the maximum number minus 2, which is 0 in this embodiment. When this information is Golomb-coded, one bit of data, "0" indicating "0", is coded. This can improve coding efficiency compared to coding three bits of data, "010" indicating the value "1".

[0051] subpic_grid_col_width_minus1 is 63, and subpic_grid_row_height_minus1 is also 63. These two values ​​are defined in units of 4 pixels, and specifically, the value is the number of basic pixels in the grid divided by 4 minus 1.

[0052] The subpicture has an INDEX value of subpic_grid_idx[i][j]. Here, [i][j] indicates the position of the grid, with i indicating the Row direction and j indicating the Col direction. In this embodiment, the Row direction ranges from 0 to 2 and the Col direction ranges from 0 to 5, with [0][0], [0][1], [0][2], [1][0], [1][1], [1][2], [2][0], [2][1], and [2][2] being 0. Also, [0][3], [0][4], [0][5], [1][3], [1][4], [1][5], [2][3], [2][4], and [2][5] being 1. Based on this subpicture information, coded data of the slices that make up the subpicture is spliced ​​together to create coded data of the subpicture.

[0053] In step S310, the image encoding device determines whether encoding of all basic blocks in the frame has been completed. If so, the process proceeds to step S311; if not, the process returns to step S302 for the next basic block.

[0054] In step S311, the in-loop filter unit 109 performs in-loop filtering on the image data reproduced in step S306 to generate a filtered image, and then the process ends.

[0055] With the above configuration and operation, especially in step S309, when the image is divided into sub-pictures, the information indicating the maximum number of sub-pictures is coded as the maximum number minus 2, thereby enabling efficient coding of the syntax related to the sub-pictures.

[0056] In this embodiment, a subpicture is composed of one slice, one tile, and one brick, but this is not limiting. Fig. 7 shows an example in which one slice is divided into multiple tiles. Specifically, the slices of INDEX0 and INDEX3 are composed of four tiles, the slices of INDEX1 and INDEX2 are composed of two tiles, and the slice of INDEX4 is composed of six tiles. This type of subpicture configuration is also applicable. In this case, the tiles and bricks have the same basic pixel size.

[0057] Note that in this embodiment, the value obtained by subtracting 2 from the maximum number of subpictures is coded, but this is not limiting. Alternatively, max_subpics_minus1, which is the maximum number of subpictures minus 1, may be coded instead, and the next parameter may be coded only when this value is 1 or greater, i.e., when the maximum number of subpictures is 2 or greater. That is, parameters such as subpic_grid_col_width_minus1, subpic_grid_row_height_minus1, and subpic_grid_idx[i][j] may be coded. In other words, when the maximum number is 1, this parameter may be omitted and not coded.

[0058] The format of the coded data in this case is shown in Figure 9. This makes it possible to reduce the amount of information in subpic_grid_col_width_minus1, subpic_grid_row_height_minus1, and subpic_grid_idx[0][0] when the maximum number of subpictures is 1. This allows for efficient coding.

[0059] In the above explanation, the following parameters are coded only when the value of max_subpics_minus1 is 1 or greater, i.e., when the maximum number of subpictures is 2 or greater. That is, the parameters subpic_grid_col_width_minus1, subpic_grid_row_height_minus1, and subpic_grid_idx[i][j] are coded when the maximum number of subpictures is 2 or greater. Furthermore, the following two values ​​may also be coded only when the maximum number of subpictures is 2 or greater. That is, subpic_treated_as_flag and loop_filter_across_subpic_enabled_flag may also be coded only when the value of max_subpics_minus1 is 1 or greater. This further reduces the amount of information and enables more efficient coding.

[0060] Furthermore, while this embodiment has described an example in which a picture is divided into two subpictures, FIG. 10 shows a case in which a picture is divided into three subpictures. subpic_grid_idx can be set to a value that can be expressed using up to Ceil(Log2(max_subpics_minus1+1)) bits, but encoding can be performed more efficiently by limiting the value to no more than max_subpics_minus1. Specifically, the maximum number of subpictures is three (three divisions), so max_subpics_minus1 is 2, obtained by subtracting 1 from 3. In this case, the number of bits is 2, calculated as Ceil(Log2(2+1)), and a value that can be expressed using two bits can be set. Specifically, four values ​​(0, 1, 2, and 3) can be set, but by limiting the maximum value that can be set to max_subpics_minus1, the values ​​are limited to 0, 1, and 2, as shown in FIG. 10, and syntax related to subpictures can be encoded accurately.

[0061] In this embodiment, grids are defined in units of four pixels, but they may also be defined in units of CTU rows and columns. When the basic pixel count of a CTU is 64 x 64, the value of subpic_grid_col_width_minus1 is 3 and the value of subpic_grid_row_height_minus1 is also 3, which reduces the amount of information to be coded and enables the amount of coding to be reduced.

[0062] In the present embodiment, subpic_grid_idx, an ID indicating the position of a rectangular area of ​​a subpicture, is coded into the sequence parameter set, but this is not limiting. FIG. 11 shows a bitstream structure in which subpic_idx, the ID of the corresponding subpicture, is coded in the slice header, which is the header of each slice. This structure allows the amount of information indicating the position of a rectangular area of ​​a subpicture to be reduced from the number of grids to the number of slices. In the example shown in FIG. 8, the number of grids is 18, and since the number of subpictures is equal to the number of slices, there are two slices, so the amount of information to be coded can be reduced from 18 to 2.

[0063] Although the subpic_idx is placed in the slice header, it is not limited to this and may be placed in the brick coded data. This allows the amount of coding to be reduced in the same way as when it is stored in the slice header.

[0064] (Embodiment 2) 2 is a block diagram showing the configuration of an image decoding device according to a second embodiment of the present invention. The image decoding device according to this embodiment is capable of decoding a bitstream including coded data coded for each of a plurality of sub-pictures. In the following, this embodiment will be described taking as an example the decoding of coded data generated in the first embodiment.

[0065] Reference numeral 201 denotes a terminal for inputting an encoded bitstream. Reference numeral 202 denotes a separate decoding unit, which separates the bitstream into information related to the decoding process and coded data related to coefficients, and sends the separated data to a decoding unit 203. It also decodes coded data present in the header of the bitstream. In this embodiment, the separate decoding unit 202 decodes header information related to image division, such as the size of tiles, bricks, slices, and basic blocks, to generate division information, which is output to an image reproduction unit 205. The separate decoding unit 202 performs the reverse operation of the integrated coding unit 111 in FIG. 1.

[0066] Reference numeral 203 denotes a decoding unit, which decodes the coded data output from the separate decoding unit 202 and reproduces the quantized coefficients and prediction information. Reference numeral 204 denotes an inverse quantization and inverse transform unit, which performs inverse quantization on the quantized coefficients to obtain transform coefficients and then performs inverse orthogonal transform to reproduce the prediction error. Reference numeral 206 denotes a frame memory, which stores the image data of the reproduced picture. Reference numeral 205 denotes an image reproduction unit, which generates predicted image data by appropriately referencing the frame memory 206 based on the input prediction information. Then, reproduced image data is generated from this predicted image data and the prediction error reproduced by the inverse quantization and inverse transform unit 204. The generated reproduced image data is output after identifying the positions of tiles, bricks, and slices within the frame based on the partition information input from the separate decoding unit 202.

[0067] An in-loop filter unit 207 performs in-loop filtering such as deblocking filtering on the reconstructed image, similar to the in-loop filter unit 109 in Fig. 1, and outputs the filtered image. A terminal 208 outputs the reconstructed image data to the outside.

[0068] The image decoding operation of the image decoding device according to this embodiment will be described below. In this embodiment, the bitstream generated in the first embodiment is input frame by frame, but a still image bitstream for one frame may also be input. Furthermore, for ease of explanation, only intra-prediction decoding processing will be described in this embodiment, but the present invention is not limited to this and can also be applied to inter-prediction decoding processing.

[0069] 2, a bitstream for one frame input from a terminal 201 is input to a demultiplexing decoder 202. The demultiplexing decoder 202 demultiplexes the bitstream into coded data related to information about the decoding process and coefficients, and decodes the coded data present in the header of the bitstream. More specifically, the demultiplexing decoder 202 decodes the basic block data partition information, tile data partition information, brick data partition information, slice data partition information, basic block row data synchronization information, and basic block row data position information in FIG. 6 to generate partition information and send it to an image regeneration unit 205. Next, the coded data of the picture data is regenerated in units of basic blocks, and output to a decoding unit 203.

[0070] The decoding unit 203 decodes the coded data to reproduce the quantization coefficients and prediction information. The reproduced quantization coefficients are output to the inverse quantization and inverse transform unit 204, and the reproduced prediction information is output to the image reproduction unit 205.

[0071] The inverse quantization and inverse transform unit 204 performs inverse quantization on the input quantized coefficients to generate orthogonal transform coefficients, and then performs inverse orthogonal transform on the quantized coefficients to regenerate prediction errors. The regenerated prediction errors are output to the image regeneration unit 205.

[0072] In the image reproduction unit 205, based on the prediction information input from the decoding unit 203, the frame memory 206 is appropriately referred to, and a predicted image is reproduced. Image data is reproduced from this predicted image and the prediction error input from the inverse quantization / inverse transformation unit 204. The generated reproduced image data identifies, based on the segmentation information input from the separation decoding unit 202, the shapes of tiles, slices, and blocks and their positions in the frame as shown in, for example, FIG. 8, and is input to and stored in the frame memory 206. The stored image data is used for reference during prediction.

[0073] In the in-loop filter unit 207, similar to the in-loop filter unit 109 in FIG. 1, a reproduced image is read from the frame memory 206, and in-loop filter processing such as deblocking filter processing is performed. Then, the filtered image is input to the frame memory 206 again. The reproduced image stored in the frame memory 206 is finally output to the outside from the terminal 208.

[0074] FIG. 4 is a flowchart showing the image decoding process in the image decoding apparatus according to Embodiment 2. First, in step S401, the separation decoding unit 202 separates the bitstream into information related to the decoding process and coded data related to coefficients, and decodes the coded data in the header portion. In the separation decoding unit 202, tile data information, block data information, slice data information, etc. in FIG. 6 are decoded, information for decoding is generated, and sent to the image reproduction unit 205. The segmentation of the image stored in the bitstream in this embodiment is as shown in FIG. 8.

[0075] [[ID=IS]] First, from the values of pic_width_in_luma_samples and pic_height_in_luma_samples of the image size information, it is derived that the image is 1536x768 pixels.

[0076] Next, since the value of log2_ctu_size_minus2 of the basic block data segmentation information is 4, the size of the basic block is derived to be 64×64 pixels from 1<<log2_ctu_size_minus2+

[0077] Next, subpicture division information is obtained. First, since subpics_present_flag is 1, it can be determined that the image has been divided into subpictures. Next, information on the maximum number of subpictures, indicated by max_subpics_minus2, is obtained. In this embodiment, a value of 0 is obtained, and by adding 2 to this value, it is determined that the maximum number of subpictures is 2. Next, basic pixel information for the grids that make up the subpictures, indicated by subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1, is obtained. Then, the vertical and horizontal numbers of the grids that make up the image, indicated by NumSubPicGridRows and NumSubPicGridCols, are derived using a predetermined calculation method. In this embodiment, subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1 can be obtained as 63.

[0078] NumSubPicGridRows and NumSubPicGridCols are calculated using the following formula:

[0079] NumSubPicGridCols= (pic_width_in_luma_samples+ subpic_grid_col_width_minus1*4+3) / (subpic_grid_col_width_minus1*4+4) NumSubPicGridRows= (pic_height_in_luma_samples+ subpic_grid_row_height_minus1*4+3) / (subpic_grid_row_height_minus1*4+4) In this embodiment, NumSubPicGridCols is derived as 6 and NumSubPicGridRows is derived as 3.

[0080] Then, the INDEX information indicating which subpicture each grid indicated by subpic_grid_idx[i][j] belongs to is calculated using the required number indicated by NumSubPicGridRows and NumSubPicGridCols, which is 18 in this embodiment. The values ​​obtained are [0][0], [0][1], [0][2], [1][0], [1][1], [1][2], [2][0], [2][1], and [2][2]. The values ​​obtained are [0][3], [0][4], [0][5], [1][3], [1][4], [1][5], [2][3], [2][4], and [2][5]. From this information, it is possible to restore that the image is divided into two subpictures, and that the two subpictures are divided as shown in FIG. 8. Then, the bitstream generated in the first embodiment, which efficiently encodes the syntax related to the subpictures, can be decoded.

[0081] After obtaining this information, the information necessary for decoding, such as tile data division information and brick data division information, is obtained, and the division information derived from the separation decoding unit 202 is sent to the image reproduction unit 205, where it is used to identify the position within the image of the data being processed in step S404.

[0082] In step S402, the decoding unit 203 decodes the coded data separated in step S401 to reproduce the quantized coefficients and prediction information. In step S403, the inverse quantization and inverse transform unit 204 performs inverse quantization on the quantized coefficients to obtain transform coefficients, and then performs inverse orthogonal transform to reproduce the prediction error.

[0083] In step S404, the image reproduction unit 205 reproduces the prediction information and predicted image generated in step S403. Furthermore, image data is reproduced from the reproduced predicted image and the prediction error generated in step S404. The reproduced image data is composited at an appropriate position in the image based on the division information generated in step S401.

[0084] In step S405, the image decoding device determines whether or not decoding of all basic blocks in the frame has been completed. If so, the process proceeds to step S406; if not, the process returns to step S402 for the next basic block.

[0085] In step S406, the in-loop filter unit 207 performs in-loop filtering on the image data reproduced in step S404 to generate a filtered image, and then the process ends.

[0086] With the above configuration and operation, it is possible to decode the bitstream in which the syntax related to the subpictures generated in the first embodiment is efficiently coded.

[0087] Note that, although the present embodiment is configured to acquire a value obtained by subtracting 2 from the maximum number of subpictures, this is not limiting. Alternatively, max_subpics_minus1, which is obtained by subtracting 1 from the maximum number of subpictures, may be acquired, and the following information may be acquired only when this value is 1 or greater, i.e., when the maximum number of subpictures is 2 or greater. That is, a configuration may be adopted in which subpic_grid_col_width_minus1, subpic_grid_row_height_minus1, and subpic_grid_idx[i][j] are acquired. In other words, if the maximum number is 1, this information may not be acquired. This makes it possible to decode a bitstream in which redundant subpicture information has been reduced.

[0088] In the above explanation, the following information is acquired only when the value of max_subpics_minus1 is 1 or greater, i.e., when the maximum number of subpictures is 2 or greater. That is, the information subpic_grid_col_width_minus1, subpic_grid_row_height_minus1, and subpic_grid_idx[i][j] is acquired when the maximum number of subpictures is 2 or greater. Furthermore, the following two pieces of information may also be acquired when the maximum number of subpictures is 2 or greater. That is, the two values ​​of subpic_treated_as_flag and loop_filter_across_subpic_enabled_flag may also be acquired only when the value of max_subpics_minus1 is 1 or greater. This makes it possible to decode a bitstream with even less redundant subpicture information.

[0089] Furthermore, although the present embodiment has been described with reference to an example in which a picture is divided into two sub-pictures, this is not limiting. For example, when dividing a picture into three sub-pictures as shown in Fig. 10, it is also possible to decode a bitstream created using an encoding method in which the maximum value of subpic_grid_idx is equal to or less than max_subpics_minus1.

[0090] Furthermore, although in this embodiment grids are defined in units of four pixels, they may be defined in units of CTU rows and columns. In this case, when restoring the values ​​of subpic_grid_col_width_minus1 and subpic_grid_row_height_minus1, calculation is performed using the basic number of pixels in the CTU, making it possible to decode a bitstream with a reduced amount of coding.

[0091] Note that, in the present embodiment, subpic_grid_idx, an ID indicating the position of a rectangular area of ​​a subpicture, is acquired from a sequence parameter set, but this is not limiting. FIG. 11 shows the structure of a bitstream in which subpic_idx, the ID of the corresponding subpicture, is instead coded in the slice header, which is the header of each slice. FIG. 11 also shows the structure of a bitstream in which subpic_idx, the ID of the corresponding subpicture, is instead coded in the slice header, which is the header of each slice. By adopting such a structure, the amount of information indicating the position of a rectangular area of ​​a subpicture can be reduced from the number of grids to the number of slices. In the example shown in FIG. 8, the number of grids is 18, and since the number of subpictures = the number of slices, there are two slices. Therefore, decoding can be performed by acquiring two subpic_idx and calculating information equivalent to 18 subpic_grid_idx.

[0092] Also, although the embodiment in which subpic_idx is placed in the slice header has been described, the present invention is not limited to this, and information equivalent to subpic_grid_idx can be decoded even if it is placed in brick coded data.

[0093] (Embodiment 3) In the above embodiment, each processing unit shown in Figures 1 and 2 has been described as being configured by hardware. However, the processing performed by each processing unit shown in these figures may be configured by a computer program.

[0094] FIG. 5 is a block diagram showing an example of the hardware configuration of a computer that can be applied to the image display device according to each of the above embodiments.

[0095] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and the ROM 503, and also executes the processes described above as being performed by the image processing device according to each of the above embodiments. That is, the CPU 501 functions as each processing unit shown in FIGS. 1 and 2.

[0096] The RAM 502 has an area for temporarily storing computer programs and data loaded from an external storage device 506, data acquired from the outside via an I / F (interface) 507, etc. The RAM 502 also has a work area used when the CPU 501 executes various processes. That is, the RAM 502 can be allocated as a frame memory, for example, or can provide various other areas as needed.

[0097] The ROM 503 stores setting data for the computer, a boot program, etc. The operation unit 504 is made up of a keyboard, a mouse, etc., and can be operated by a user of the computer to input various instructions to the CPU 501. The display unit 505 displays the results of processing by the CPU 501. The output unit 505 is made up of, for example, a liquid crystal display.

[0098] The external storage device 506 is a large-capacity information storage device, such as a hard disk drive. The external storage device 506 stores an operating system (OS) and computer programs for causing the CPU 501 to implement the functions of the various units shown in Figures 1 and 2. Furthermore, the external storage device 506 may also store image data to be processed.

[0099] Computer programs and data stored in the external storage device 506 are loaded into the RAM 502 as appropriate under the control of the CPU 501 and become the subject of processing by the CPU 501. An I / F 507 can be connected to a network such as a LAN or the Internet, or other devices such as a projector or display device, and the computer can obtain and send various information via this I / F 507. 508 is a bus that connects the above-mentioned components.

[0100] The operation of the above-described configuration is controlled mainly by the CPU 501 as explained in the flow chart above.

[0101] (Other Examples) The object of the present invention can also be achieved by providing a storage medium on which computer program code that realizes the above-described functions is recorded to a system, and the system reading and executing the computer program code. In this case, the computer program code read from the storage medium itself realizes the functions of the above-described embodiments, and the storage medium on which the computer program code is stored constitutes the present invention. It also includes cases where an operating system (OS) running on a computer performs some or all of the actual processing based on the instructions of the program code, thereby realizing the above-described functions.

[0102] Furthermore, the present invention may be realized in the following form: That is, computer program code read from a storage medium is written to memory in a function expansion card inserted into a computer or in a function expansion unit connected to the computer, and the CPU in the function expansion card or function expansion unit then performs some or all of the actual processing based on the instructions of the computer program code, thereby realizing the above-mentioned functions.

[0103] When the present invention is applied to the storage medium, the storage medium stores computer program code corresponding to the flowcharts described above. [Explanation of symbols]

[0104] 101, 112, 201, 208 terminals 102 Image division unit 103 Block division section 104 Prediction Department 105 Transformation and Quantization Unit 106, 204 Inverse quantization and inverse transform unit 107, 205 Image playback unit 108, 206 frame memory 109, 207 In-loop filter section 110 Encoding section 111 Integrated coding unit 202 Separate Decoding Unit 203 Decoding Unit

Claims

1. An image coding apparatus for coding an image including one or more sub-pictures, each of which is composed of one or more slices, into a bitstream, comprising: a first encoding means for encoding a flag relating to the presence of sub-picture information into a sequence parameter set of the bitstream; second encoding means for encoding, when the flag has a value indicating the presence of sub-picture information, a syntax element relating to the number of sub-pictures included in the image and first information relating to IDs of the sub-pictures included in the image into a sequence parameter set of the bitstream; third encoding means for encoding, into the sequence parameter set of the bitstream, second information, which is information of a predetermined size used to specify an arrangement of sub-pictures included in the image, and which is a value equal to a result of subtracting 1 from a result of dividing the predetermined size by 64; a fourth encoding means for encoding a loop_filter_across_subpic_enabled_flag relating to a filtering process into the sequence parameter set of the bitstream, The second information and the loop_filter_across_subpic_enabled_flag are coded into the sequence parameter set of the bitstream only when the value of the syntax element coded into the sequence parameter set of the bitstream is 1 or greater; a state in which the value of the syntax element is 1 or more corresponds to the number of sub-pictures included in the image being 2 or more; When the flag does not have a value indicating the presence of sub-picture information, the syntax element and the first information are not coded into the sequence parameter set; When the second information and the loop_filter_across_subpic_enabled_flag are coded, the second information is coded before the loop_filter_across_subpic_enabled_flag in the sequence parameter set. An image encoding device comprising:

2. The image encoding device according to claim 1 , wherein the syntax element corresponds to a maximum number of sub-pictures included in the image.

3. 2. The image encoding device according to claim 1, wherein the sub-picture included in the image can include one tile.

4. An image decoding device that decodes a bitstream obtained by encoding an image including one or more subpictures, each of which is composed of one or more slices, comprising: a first decoding means for decoding a flag relating to the presence of sub-picture information from a sequence parameter set of the bitstream; second decoding means for decoding, when the flag has a value indicating the presence of sub-picture information, a syntax element relating to the number of sub-pictures included in the image and first information relating to IDs of the sub-pictures included in the image from a sequence parameter set of the bitstream; third decoding means for decoding, from the sequence parameter set of the bitstream, second information, which is information of a predetermined size used to identify an arrangement of sub-pictures included in the image, and which is a value equal to a result of subtracting 1 from a result of dividing the predetermined size by 64; a fourth decoding means for decoding a loop_filter_across_subpic_enabled_flag related to a filtering process from the sequence parameter set of the bitstream; the second information and the loop_filter_across_subpic_enabled_flag are decoded from the sequence parameter set of the bitstream only if the value of the syntax element decoded from the sequence parameter set of the bitstream is greater than or equal to 1; a state in which the value of the syntax element is 1 or more corresponds to the number of sub-pictures included in the image being 2 or more; if the flag does not have a value indicating the presence of sub-picture information, the syntax element and the first information are not decoded from the sequence parameter set; When the second information and the loop_filter_across_subpic_enabled_flag are decoded, the second information is decoded from the sequence parameter set before the loop_filter_across_subpic_enabled_flag. An image decoding device comprising:

5. The image decoding device according to claim 4 , wherein the syntax element corresponds to a maximum number of sub-pictures included in the image.

6. 5. The image decoding device according to claim 4, wherein the sub-picture included in the image can include one tile.

7. A method of encoding an image comprising one or more sub-pictures, each sub-picture consisting of one or more slices, into a bitstream, comprising: a first encoding step of encoding a flag relating to the presence of sub-picture information in a sequence parameter set of said bitstream; a second encoding step of encoding, if the flag has a value indicating the presence of sub-picture information, a syntax element relating to the number of sub-pictures included in the image and first information relating to IDs of the sub-pictures included in the image into a sequence parameter set of the bitstream; a third encoding step of encoding, into the sequence parameter set of the bitstream, second information, which is information of a predetermined size used to specify the arrangement of sub-pictures included in the image, and which is a value equal to a result of subtracting 1 from a result of dividing the predetermined size by 64; a fourth encoding step of encoding a loop_filter_across_subpic_enabled_flag related to a filtering process into the sequence parameter set of the bitstream, The second information and the loop_filter_across_subpic_enabled_flag are coded into the sequence parameter set of the bitstream only when the value of the syntax element coded into the sequence parameter set of the bitstream is 1 or greater; a state in which the value of the syntax element is 1 or more corresponds to the number of sub-pictures included in the image being 2 or more; When the flag does not have a value indicating the presence of sub-picture information, the syntax element and the first information are not coded into the sequence parameter set; When the second information and the loop_filter_across_subpic_enabled_flag are coded, the second information is coded before the loop_filter_across_subpic_enabled_flag in the sequence parameter set.

1. An image encoding method comprising:

8. An image decoding method for decoding a bitstream obtained by encoding an image including one or more subpictures, each of which is composed of one or more slices, comprising: a first decoding step of decoding a flag relating to the presence of sub-picture information from a sequence parameter set of said bitstream; a second decoding step of decoding, if the flag has a value indicating the presence of sub-picture information, a syntax element relating to the number of sub-pictures included in the image and first information relating to IDs of the sub-pictures included in the image from a sequence parameter set of the bitstream; a third decoding step of decoding, from the sequence parameter set of the bitstream, second information, which is information of a predetermined size used to identify the arrangement of sub-pictures included in the image, and which is a value equal to a result of subtracting 1 from a result of dividing the predetermined size by 64; a fourth decoding means for decoding a loop_filter_across_subpic_enabled_flag related to a filtering process from the sequence parameter set of the bitstream; the second information and the loop_filter_across_subpic_enabled_flag are decoded from the sequence parameter set of the bitstream only if the value of the syntax element decoded from the sequence parameter set of the bitstream is greater than or equal to 1; a state in which the value of the syntax element is 1 or more corresponds to the number of sub-pictures included in the image being 2 or more; if the flag does not have a value indicating the presence of sub-picture information, the syntax element and the first information are not decoded from the sequence parameter set; When the second information and the loop_filter_across_subpic_enabled_flag are decoded, the second information is decoded from the sequence parameter set before the loop_filter_across_subpic_enabled_flag.

1. An image decoding method comprising:

Images