Information processing device and information processing method

Abstract

The present invention suppresses a reduction in bit efficiency. An information processing device is provided, the information processing device including: an encoding unit (101) that encodes a picture including two or more sub-pictures to generate encoded data; a metadata generation unit (102) that generates sub-picture mapping information, the sub-picture mapping information including first size information of a first sub-picture among the two or more sub-pictures and second size information of each of one or more second sub-pictures other than the first sub-picture, the second size information of the second sub-pictures being considered to be the same as the first size information of the first sub-picture; and a bitstream generation unit (103) that generates a bitstream including the encoded data and the sub-picture mapping information.

Description

Technical Field

[0001] The present disclosure relates to an information processing device and an information processing method. Background Art

[0002] In Versatile Video Coding (VVC), which is established as a standard specification for image coding technology, each image (picture) is divided into one or more slices (for example, see Non-Patent Document 1). In addition, in VVC, each picture can be divided into one or more sub-pictures.

[0003] In VVC, a sub-picture is rectangular and is divided into one or more slices. It is assumed that the sub-picture function is used for, for example, partial decoding of a full-spherical video, merging of multiple pictures into one picture, etc.

[0004] Sub-picture mapping information such as the position and size of each sub-picture in a picture is stored in a sequence parameter set (SPS) as parameters in units of a coding tree unit (CTU), which is a minimum unit of encoding.

[0005] Reference List

[0006] Non-patent literature

[0007] Non-Patent Document 1: Benjamin Bross, Jianle Chen, Shan Liu, and Ye-Kui Wang, “Versatile Video Coding (Draft 8)”, JVET-Q2001-vE, 17th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11: Brussels, Belgium, January 7-17, 2020 Summary of the Invention

[0008] Technical issues

[0009] However, in the method described in Non-Patent Document 1, since sub-picture mapping information is in units of CTU based on resolution, redundant bits may be generated, resulting in a problem of reduced bit efficiency.

[0010] Therefore, the present disclosure has been made in view of such circumstances, and proposes an information processing apparatus and an information processing method capable of suppressing a decrease in bit efficiency.

[0011] Solution to the problem

[0012] An information processing device as one aspect of the present technology includes: an encoding unit that encodes a picture including two or more sub-pictures to generate encoded data; a metadata generation unit that generates sub-picture mapping information, the sub-picture mapping information including first size information of a first sub-picture among the two or more sub-pictures and information that regards second size information of each of one or more second sub-pictures other than the first sub-picture as being the same as the first size information of the first sub-picture; and a bitstream generation unit that generates a bitstream including the encoded data and the sub-picture mapping information.

[0013] As another aspect of the present technology, an information processing device includes: an extraction unit that obtains encoded data and sub-picture mapping information from a bit stream; a decoding unit that decodes the encoded data to restore a picture, and divides the restored picture into two or more sub-pictures based on the sub-picture mapping information; and a presentation unit that presents the picture based on the two or more sub-pictures, wherein the sub-picture mapping information includes first size information of a first sub-picture among the two or more sub-pictures and information that regards second size information of each of one or more second sub-pictures other than the first sub-picture as being the same as the first size information of the first sub-picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a diagram showing a configuration example of a bit stream.

[0015] Figure 2 is a diagram illustrating an example of a sequence parameter set in which sub-picture mapping information is signaled.

[0016] Figure 3 is a diagram showing an example of position information and size information of a reference pixel in each partial area indicated by sub-picture mapping information.

[0017] Figure 4 A diagram illustrating a case where a picture having a size of 3072×2048 pixels is divided into six sub-pictures having equal sizes.

[0018] Figure 5 is a diagram showing an example of the bit length of each parameter in the semantics of sub-picture mapping information.

[0019] Figure 6 is a diagram illustrating an example of a sequence parameter set according to one embodiment of the present disclosure, in which sub-picture mapping information is signaled.

[0020] Figure 7 is a diagram illustrating an example of a portion of a sequence parameter set according to a modification example of one embodiment of the present disclosure, in which sub-picture mapping information is signaled.

[0021] Figure 8 : is a block diagram illustrating an example of the configuration of an image encoding device which is one aspect of an information processing device to which the technology according to one embodiment of the present disclosure is applied.

[0022] Figure 9 : is a flowchart illustrating an example of the flow of an encoding process performed by an image encoding device which is one aspect of an information processing device to which the technology according to one embodiment of the present disclosure is applied.

[0023] Figure 10 : is a block diagram illustrating an example of the configuration of an image decoding device which is one aspect of an information processing device to which the technology according to one embodiment of the present disclosure is applied.

[0024] Figure 11 is a flowchart illustrating an example of the flow of a decoding process performed by an image decoding device which is one aspect of an information processing device to which the technology according to one embodiment of the present disclosure is applied.

[0025] Figure 12 : is a block diagram illustrating a configuration example of hardware of a computer that executes a program for the information processing apparatus according to an embodiment of the present disclosure to perform a series of processes. DETAILED DESCRIPTION

[0026] Hereinafter, a method for carrying out the present disclosure will be described. Note that the description will be given in the following order.

[0027] 1. Introduction

[0028] 2. One Implementation Method

[0029] 2.1 Syntax of Sub-Image Mapping Information

[0030] 2.2 Improving Bit Efficiency by Omitting Parameters Indicating the Reference Position of Each Sub-Picture (Modified Example)

[0031] 2.3 Example of Configuration of Image Encoding Device

[0032] 2.3.1 Coding Process

[0033] 2.4 Configuration Example of Image Decoding Device

[0034] 2.4.1 Decoding Process

[0035] 3. Appendix

[0036] 1. Introduction

[0037] The scope of the present technology disclosed herein includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that have been disclosed at the time of filing this application, the contents of other documents cited in the following non-patent documents, and the like.

[0038] Non-Patent Document 1: (as mentioned above)

[0039] That is, the content described in the aforementioned non-patent literature can also serve as a basis for determining whether support requirements are met. For example, regarding the sequence parameter set described in non-patent literature 1, although its syntax and decoding process are not directly defined in the description of the present disclosure, the syntax and decoding process fall within the scope of the present disclosure and meet the support requirements of the claims. In addition, for example, although technical terms such as parsing, syntax, and semantics are not directly defined in the description of the present disclosure, these terms also fall within the scope of the present disclosure and meet the support requirements of the claims.

[0040] In addition, in this specification, unless otherwise specified, the term "block" (which is not a block indicating a processing unit) used to describe a partial area or processing unit of an image (picture) refers to any partial area in the image, and the size, shape, characteristics, etc. of the "block" are not limited. For example, the "block" includes the transform block (TB), transform unit (TU), prediction block (PB), prediction unit (PU), smallest coding unit (SCU), coding unit (CU), largest coding unit (LCU), coding tree block (CTB), coding tree unit (CTU), subblock, macroblock, tile, or slice described in the above-mentioned non-patent literature.

[0041] In addition, when specifying the size of such a block, the block size can be specified indirectly as well as directly. For example, the block size can be specified using identification information for identifying the size. Alternatively, for example, the block size can be specified based on a ratio or difference relative to the size of a reference block (e.g., LCU or SCU). For example, in the case where information for specifying the block size is transmitted as a syntax element, the information for indirectly specifying the size as described above can be used as the information. By doing so, the amount of information can be reduced, thereby improving coding efficiency. In addition, the specification of the block size includes specifying a block size range (e.g., specifying an allowable block size range).

[0042] <sub-image>

[0043] In the VVC described in Non-Patent Document 1, a function called sub-picture is implemented so that an image area corresponding to a picture is divided into a plurality of partial areas when used.

[0044] Figure 1: is a diagram showing an example of a main configuration of a VVC bitstream, which is a bitstream generated by encoding an image using the VVC technique. Figure 1 The VVC bitstream 10 shown is coded data for a moving image, including multiple frame images. The VVC bitstream 10 is composed of a set of coded data 11 for a coded video sequence (CVS). A CVS is a collection of pictures within a certain time period. This certain time period does not need to be fixed and can vary for each piece of coded data 11. A picture is a frame image at a certain time. In other words, the coded data 11 of the CVS is composed of a collection of coded data 12 for pictures at various times within the certain time period.

[0045] The coded data 12 of a picture is configured by a set of coded data of sub-pictures 13. A sub-picture is a partial area into which a picture (ie, an image area corresponding to a picture) is divided.

[0046] In the VVC described in Non-Patent Document 1, pictures and sub-pictures have the following characteristics. Both pictures and sub-pictures are rectangular. There are no pixels in the picture that do not have coded data. There is no overlap between sub-pictures. As a pixel of the picture, there is no pixel that is not included in any sub-picture.

[0047] The sub-picture is a function intended to achieve decoding of each sub-picture (distributed processing) or reduce the number of decoder instances by combining multiple pictures or sub-pictures into one picture.

[0048] For example, by assigning six-sided images of each cube of a full-spherical video (3-degree-of-freedom (DoF) content) projected using a cube mapping technique to each sub-picture, various types of control are facilitated, such as processing the images of each face independently or in a combined manner. Note that since a sub-picture is not a coding unit such as a slice or tile, other sub-pictures can also be referenced during encoding, for example.

[0049] In order to implement such sub-pictures, picture partitioning information (sub-picture mapping information) is signaled (ie, sent from the encoding device to the decoding device).

[0050] The sub-picture mapping information is fixed in the CVS (information that cannot be changed). For example, the sub-picture mapping information is signaled in a sequence parameter set (SPS), which is a set of parameters in units of sequences, such as Figure 2 The syntax is shown.

[0051] exist Figure 2In the semantics shown, the value obtained by adding 1 to "sps_num_subpics_minus1" indicates the number of sub-pictures included in each picture of the coding layer video sequence (CLVS). The value of "sps_num_subpics_minus1" may be included in the range from zero to a value obtained by using the sealing function "Ceil(pic_width_max_in_luma_samples÷CtbSizeY)×Ceil(pic_height_max_in_luma_samples÷CtbSizeY)-1". If "sps_num_subpics_minus1" is not included in the semantics, its value is estimated to be zero.

[0052] "subpic_ctu_top_left_x[i]" indicates the horizontal position of the top left end of the i-th sub-picture in CTU units in units of CtbSizeY. The bit length of the corresponding syntax element in the semantics is the bit length obtained by "Ceil(Log2((pic_width_max_in_luma_samples+CtbSizeY-1)>>CtbLog2SizeY))". If "subpic_ctu_top_left_x[i]" is not included in the semantics, its value is estimated to be zero.

[0053] "subpic_ctu_top_left_y[i]" indicates the vertical position of the top left end of the i-th sub-picture in CTU units in units of CtbSizeY. The bit length of the corresponding syntax element in the semantics is the bit length obtained by "Ceil(Log2((pic_height_max_in_luma_samples+CtbSizeY-1)>>CtbLog2Size Y))". If "subpic_ctu_top_left_y[i]" is not included in the semantics, its value is estimated to be zero.

[0054] The value obtained by adding 1 to "subpic_width_minus1[i]" indicates the width of the i-th sub-picture in units of CtbSizeY. The bit length of the corresponding syntax element in the semantics is the bit length obtained by "Ceil(Log2((pic_width_max_in_luma_samples+CtbSizeY-1)>>CtbLog2Size Y))". If "subpic_width_minus1[i]" is not included in the semantics, its value is estimated to be equal to "((pic_width_max_in_luma_samples+CtbSizeY-1)>>CtbLog2SizeY)-subpic_ctu_top_left_x[i]-1".

[0055] The value obtained by adding 1 to "subpic_height_minus1[i]" indicates the height of the i-th sub-picture in units of CtbSizeY. The bit length of the corresponding syntax element in the semantics is the bit length obtained by "Ceil(Log2((pic_height_max_in_luma_samples+CtbSizeY-1)>>CtbLog2Size Y))". If "subpic_height_minus1[i]" is not included in the semantics, its value is estimated to be equal to "((pic_height_max_in_luma_samples+CtbSizeY-1)>>CtbLog2SizeY)-subpic_ctu_top_left_y[i]-1".

[0056] Note that in Figure 2 In the illustrated semantics, “CtbSizeY” indicates the CTB size of the CTU (unit: sample), “CtbLog2SizeY” indicates the value of Log2 of CtbSizeY, “pic_width_max_in_luma_samples” indicates the width of the picture (unit: sample), and “pic_height_max_in_luma_samples” indicates the height of the picture (unit: sample).

[0057] Such sub-picture mapping information is information indicating the layout of each partial area as a sub-picture. Figure 3 As shown, the sub-picture mapping information expresses each sub-picture based on the position information (e.g., X and Y coordinates, hereinafter also referred to as the reference position) and size information of the reference pixel (e.g., the pixel at the upper left end) of each partial area. Figure 2 and Figure 3In the example of , the horizontal position (subpic_ctu_top_left_x) and vertical position (subpic_ctu_top_left_y) of the pixel at the top left end of each partitioned region are indicated in units of CTU as the position information of the reference pixel of each sub-picture. In addition, the width (subpic_width_minus1) and height (subpic_height_minus1) of each partitioned region are indicated in units of CTU as the size information of each sub-picture.

[0058] The bit length of the parameters used to specify the partial area corresponding to each sub-picture (e.g., the position and size information of the reference pixel) can be calculated based on the width and height of the picture and the size of the CTBs that constitute the CTU. The size of the CTBs that constitute the CTU (hereinafter also referred to as the CTU size) is generally any one of 32 samples, 64 samples, and 128 samples.

[0059] Here, the corresponding bit lengths in the semantics of the parameters "subpic_ctu_top_left_x[i]", "subpic_ctu_top_left_y[i]", "subpic_width_minus1[i]", and "subpic_height_minus1[i]" will be described in the case of dividing a picture having a size of 3072×2048 pixels into six sub-pictures of the same size of 1024×1024 pixels. For example, this can be the case of assuming that each of the six planes of a full-sphere video is used for a sub-picture on a cub map. Figure 4 1 is a diagram showing a case where a picture having a size of 3072×2048 pixels is divided into six sub-pictures having equal sizes. Figure 5 It is shown in Figure 4 The diagram shows the bit length of each parameter in the semantics of the sub-picture mapping information in the example case. Figure 4 and Figure 5 In , the CTU size is 128.

[0060] exist Figure 5 In the semantics shown, the bit length of "subpic_ctu_top_left_x[i]" is 5 bits, the bit length of "subpic_ctu_top_left_y[i]" is 4 bits, the bit length of "subpic_width_minus1[i]" is 5 bits, and the bit length of "subpic_height_minus1[i]" is 4 bits. For example, when the CTU size is 128, the number of bits required to express "subpic_ctu_top_left_x[i]" to 3072 is 5 bits.

[0061] Therefore, if Figure 4 As shown, when a picture having a size of 3072×2048 pixels is divided into six sub-pictures of the same size of 1024×1024 pixels, the total bit length of the parameters of each sub-picture is 18 bits (= 5 bits + 4 bits + 5 bits + 4 bits). However, with respect to the sub-picture P1 located on the upper left side, since the parameters "subpic_ctu_top_left_x[0]" and "subpic_ctu_top_left_y[0]" can be omitted, the total bit length of the parameters of the sub-picture P1 is 9 bits. Similarly, with respect to the sub-picture P6 located on the lower right side, since the parameters "subpic_width_minus1[5]" and "subpic_height_minus1[5]" can be omitted, the total bit length of the parameters of the sub-picture P6 is 9 bits. As a result, the sum of the bit lengths of the parameters of the entire picture is 90 bits (= 9 bits + 18 bits + 18 bits + 18 bits + 18 bits + 9 bits).

[0062] As described above, in the sub-picture mapping information, the parameters of the upper left coordinate position and size information (width and height) of each sub-picture are signaled in units of CTUs. Here, since a CTU is a unit area defined based on the resolution of a picture, a CTU is much smaller than a picture or sub-picture. Therefore, when the parameters of each sub-picture are defined in units of CTUs, the bit length required to describe the parameters may be large, thereby reducing bit efficiency.

[0063] In this regard, in the present disclosure, a method capable of omitting parameters for a sub-picture under the premise of suppressing a reduction in bit efficiency will be described by way of example.

[0064] 2. One Implementation Method

[0065] In one embodiment of the present disclosure, a precondition is that the partial areas of the plurality of sub-pictures into which a picture is divided have equal widths and equal heights. In other words, the sub-pictures are defined by dividing the picture into equal parts. When this precondition is satisfied, for the second or subsequent (i is an integer of 1 or greater) sub-picture, the subpic_width_minus1[0] and subpic_height_minus1[0] of the previous sub-picture are reused. That is, the sub-picture mapping information in the sequence parameter set (SPS) defines that the width and height defined for the first sub-picture (i is 0) located on the upper left side of the picture are reused. As a result, when the sub-pictures have equal widths and equal heights, the parameters defining the width and height of the second and subsequent sub-pictures in the sequence parameter set (SPS) (i.e., subpic_width_minus1[i] and subpic_height_minus1[i]) can be omitted, thereby reducing the number of bits of the parameters in the sub-picture mapping information and thus suppressing a reduction in bit efficiency.

[0066] 2.1 Syntax of Sub-Image Mapping Information

[0067] Figure 6 : is a diagram showing an example of a sequence parameter set (SPS) in which sub-picture mapping information is signaled according to the present embodiment. Figure 6 As shown, with reference Figure 2 Compared to the SPS described above, the SPS according to the present embodiment additionally includes “fixed_subpic_width_height_flag” in the 5th line, “&&(!fixed_subpic_width_height_flag||(fixed_subpic_width_height_flag&&i==0))” in the if clause in the 11th line, and “&&(!fixed_subpic_width_height_flag||(fixed_subpic_width_height_flag&&i==0))” in the if clause in the 13th line.

[0068] "fixed_subpic_width_height_flag" is a flag indicating whether all sub-pictures have equal width and equal height. For example, when "fixed_subpic_width_height_flag" is '1', this indicates that all sub-pictures have equal width and equal height, and when "fixed_subpic_width_height_flag" is '0', this indicates that at least one sub-picture has a width and height different from those of other sub-pictures.

[0069] “&&(!fixed_subpic_width_height_flag||(fixed_subpic_width_height_flag&&i==0))” added to the if clause in line 11 and line 12 define that when “fixed_subpic_width_height_flag” is ‘1’, it is considered that the value of “subpic_width_minus1[i]” is equal to the value of “subpic_width_minus1[0]” when i is greater than 0.

[0070] Similarly, “&&(!fixed_subpic_width_height_flag||(fixed_subpic_width_height_flag&&i==0))” added to the if clause in line 13 and line 14 define that when “fixed_subpic_width_height_flag” is ‘1’, the value of “subpic_height_minus1[i]” is considered to be equal to the value of “subpic_height_minus1[0]” when i is greater than 0.

[0071] By adopting the above syntax, when sub-pictures have equal width and equal height, subpic_width_minus1[i] and subpic_height_minus1[i] for defining the width and height of the second and subsequent sub-pictures can be omitted, thereby reducing the number of bits of parameters in the sub-picture mapping information. As a result, it is possible to suppress a decrease in bit efficiency.

[0072] Note that, for example, the case where all sub-pictures have equal width and equal height is assumed to be a case where six-sided images of a full-sphere video (3-degree-of-freedom (DoF) content) are assigned to each sub-picture. However, the present disclosure is not limited thereto, and the above method can be applied to various forms in which sub-pictures have equal width and equal height.

[0073] 2.2 Improving Bit Efficiency by Omitting Parameters Indicating the Reference Position of Each Sub-Picture (Modified Example)

[0074] In addition, in the case where all sub-pictures have equal width and equal height, the reference position of each sub-picture (for example, the X and Y coordinates of the reference pixel located at the upper left end of each sub-picture) can be easily obtained by calculating according to the size (width and height) of each sub-picture. Figure 6The syntax for defining the X and Y coordinates of the base pixel of each sub-picture (subpic_ctu_top_left_x[i] and subpic_ctu_top_left_y[i]) shown in lines 7 to 10 of Figure 7 The syntax shown in is substituted.

[0075] exist Figure 7 In the if clause in line 7, “&&!fixed_subpic_width_height_flag)” added and line 8 define that when “fixed_subpic_width_height_flag” is ‘1’, the value of “subpic_ctu_top_left_x[i]” when i is greater than 0 is calculated based on the value of “pic_width_max_in_luma_samples” indicating the width of the picture and the value of “subpic_width_minus1[0]” indicating the width of the sub-picture.

[0076] Similarly, “&&!fixed_subpic_width_height_flag)” added in the if clause of line 9 and line 10 define that: when “fixed_subpic_width_height_flag” is ‘1’, the value of “subpic_ctu_top_left_y[i]” when i is greater than 0 is calculated based on the value of “pic_height_max_in_luma_samples” indicating the height of the picture and the value of “subpic_height_minus1[0]” indicating the height of the sub-picture after obtaining the number of sub-pictures in the horizontal direction based on the value of “pic_width_max_in_luma_samples” indicating the width of the picture and the value of “subpic_width_minus1[0]” indicating the width of the sub-picture.

[0077] By adopting the above syntax, when sub-pictures have equal widths and equal heights, subpic_ctu_top_left_x[i] and subpic_ctu_top_left_y[i], which define the reference positions of the second and subsequent sub-pictures and the first sub-picture, can be omitted. The definition of the reference positions is omitted in the original syntax, thereby reducing the number of bits of parameters in the sub-picture mapping information. As a result, it is possible to suppress a decrease in bit efficiency.

[0078] 2.3 Configuration Example of Image Coding Device

[0079] The above method for suppressing bit efficiency reduction can be applied to any device. For example, the above method for suppressing bit efficiency reduction can be applied to an encoding device. Figure 8 : is a block diagram showing an example of the configuration of an image encoding device which is one aspect of an information processing device to which the technology according to the present embodiment is applied. Figure 8 The image encoding device 100 shown in FIG is a device for encoding an image, and is an example of an encoding device. The image encoding device 100 performs encoding by applying, for example, an encoding technique based on VVC described in Non-Patent Document 1. At this time, the image encoding device 100 performs encoding by applying a reference to Figures 6 to 9 The encoding is performed using the described method.

[0080] Notice, Figure 8 Only the main processing units, main data flows, etc. are shown, and Figure 8 The contents shown in FIG. 1 do not include all configurations of the image encoding device. That is, the image encoding device 100 may include Figure 8 Other processing units not shown as blocks, or other processing units may exist in the image encoding device 100. Figure 8 Other processing or data flows not shown by arrows etc.

[0081] like Figure 8 As shown, the image encoding device 100 includes an encoding unit 101, a metadata generating unit 102, and a bit stream generating unit 103.

[0082] The encoding unit 101 performs processing related to image encoding. For example, the encoding unit 101 obtains each picture of the moving image input to the image encoding device 100. The encoding unit 101 encodes the obtained picture by applying, for example, an encoding technique based on VVC described in Non-Patent Document 1.

[0083] The encoding unit 101 supplies the encoded data generated by encoding the image to the bitstream generating unit 103. In addition, the encoding unit 101 can appropriately transmit and receive specific information to and from the metadata generating unit 102 at the time of encoding.

[0084] The metadata generation unit 102 performs processing related to metadata generation. For example, the metadata generation unit 102 sends and receives specific information to and from the encoding unit 101 to generate metadata. For example, the metadata generation unit 102 may generate metadata as sub-picture mapping information, which includes information regarding whether all sub-pictures have the same resolution, the location of the sub-picture's reference pixel, the sub-picture's resolution, and various other types of information. Other types of information may include, for example, information indicating whether all sub-pictures can be processed independently, sub-picture ID mapping information, and the like. The metadata generation unit 102 provides the generated metadata to the bitstream generation unit 103.

[0085] The bitstream generation unit 103 performs processing related to bitstream generation. For example, the bitstream generation unit 103 obtains the encoded data provided by the encoding unit 101. Furthermore, the bitstream generation unit 103 obtains metadata provided by the metadata generation unit 102. The bitstream generation unit 103 generates a bitstream including the obtained encoded data and metadata. The bitstream generation unit 103 outputs the bitstream to the outside of the image encoding device 100. The output bitstream is provided to the decoding device via, for example, a storage medium or a communication medium. The decoding device performs decoding processing based on the information signaled in the bitstream.

[0086] 2.3.1 Coding Process

[0087] Next, we will refer to Figure 9 The flowchart describes an example of the flow of the encoding process performed by the image encoding device 100.

[0088] When the encoding process starts, the encoding unit 101 of the image encoding device 100 divides a picture into sub-pictures in step S101.

[0089] In step S102 , the encoding unit 101 encodes the picture according to the sub-pictures divided in step S101 .

[0090] In step S103, the metadata generation unit 102 generates metadata including sub-picture mapping information. At this time, the metadata generation unit 102 generates sub-picture mapping information with improved bit efficiency by using the above-mentioned method.

[0091] In step S104, the bitstream generation unit 103 generates a bitstream by using the encoded data generated in step S102 and the metadata including the sub-picture mapping information generated in step S103. That is, the bitstream generation unit 103 generates a bitstream including these information items.

[0092] When the bit stream is generated, the encoding process ends.

[0093] Meanwhile, the decoding device may perform a decoding process based on information signaled in a bit stream.

[0094] 2.4 Configuration Example of Image Decoding Device

[0095] The technology according to this embodiment can also be applied to a decoding device. Figure 10 : is a block diagram showing an example of the configuration of an image decoding device which is one aspect of an information processing device to which the technology according to the present embodiment is applied. Figure 10 The image decoding device 200 shown is a device that decodes coded data to generate an image. The image decoding device 200 is an example of a decoding device. The image decoding device 200 performs decoding by applying the decoding technology based on VVC described in Non-Patent Document 1, for example. At this time, the image decoding device 200 applies the reference Figures 6 to 9 Decoding is performed using the described method.

[0096] Notice, Figure 10 Only the main processing units, main data flows, etc. are shown, and Figure 10 The contents shown in FIG. 2 do not include all configurations of the image decoding device. That is, the image decoding device 200 may include Figure 10 Other processing units not shown as blocks, or other processing units may exist in the image decoding device 200. Figure 10 Other processing or data flows not shown by arrows etc.

[0097] like Figure 10 As shown, the image decoding apparatus 200 includes an analyzing unit 201 , an extracting unit 202 , a decoding unit 203 and a presenting unit 204 .

[0098] The analysis unit 201 performs processing related to metadata analysis. For example, the analysis unit 201 obtains a bitstream input to the image decoding device 200. The analysis unit 201 analyzes the metadata included in the bitstream. Then, the analysis unit 201 provides the metadata analysis result and the bitstream to the extraction unit 202.

[0099] The extraction unit 202 extracts desired information from the bitstream provided by the analysis unit 201 based on the analysis result provided by the analysis unit 201. For example, the extraction unit 202 extracts the encoded data of the image, metadata including sub-picture mapping information, etc. from the bitstream. Then, the extraction unit 202 provides the information extracted from the bitstream, etc. to the decoding unit 203.

[0100] The decoding unit 203 performs processing related to decoding. For example, the decoding unit 203 obtains information provided from the extraction unit 202. The decoding unit 203 decodes the obtained coded data based on the obtained metadata to generate a picture. At this time, the decoding unit 203 appropriately applies the reference Figures 6 to 9 The generated picture is divided into sub-pictures by the described method, etc. Then, the decoding unit 203 provides the generated picture (an image of each sub-picture) to the presentation unit 204. In addition, the decoding unit 203 can provide the presentation unit 204 with sub-picture presentation information.

[0101] Rendering unit 204 performs processing related to presentation. For example, rendering unit 204 obtains the picture and sub-picture presentation information provided by decoding unit 203. Based on the sub-picture presentation information, rendering unit 204 renders the desired sub-picture within the picture to generate a display image. Rendering unit 204 then outputs the generated display image to the outside of image decoding device 200. The display image is provided to an image display device (not shown) via a specific storage medium, communication medium, etc., and is displayed on the image display device.

[0102] As described above, the image decoding device 200 analyzes various types of information including sub-picture mapping information signaled from the encoding device, and performs a decoding process based on the information.

[0103] 2.4.1 Decoding Process

[0104] Next, we will refer to Figure 11 An example of the flow of the decoding process performed by the image decoding device 200 is described with reference to a flowchart of FIG.

[0105] When the decoding process starts, the analyzing unit 201 of the image decoding device 200 analyzes metadata included in a bitstream in step S201 .

[0106] In step S202 , the extraction unit 202 extracts the encoded data and metadata including sub-picture mapping information from the bitstream based on the analysis result in step S201 .

[0107] In step S203 , the decoding unit 203 decodes the encoded data extracted from the bit stream in step S202 based on the metadata extracted from the bit stream in step S202 to generate a picture (each sub-picture included in the picture).

[0108] In step S204 , the decoding unit 203 divides the picture decoded in step S203 into a plurality of sub-pictures based on the sub-picture mapping information extracted in step S202 .

[0109] In step S205 , the presentation unit 204 presents the picture divided into sub-pictures in step S204 to generate a display image.

[0110] When the display image is generated, the decoding process ends.

[0111] By performing the decoding process as described above, decoding and presentation are performed based on metadata including the signaled sub-picture mapping information.

[0112] 3. Appendix

[0113] <Computer>

[0114] The above series of processes can be performed by hardware or software. In the case of performing the series of processes by software, the program configuring the software is installed in the computer. Here, the computer includes a computer incorporated into dedicated hardware, a computer capable of performing various functions by installing various programs therein, such as a general-purpose personal computer, etc.

[0115] Figure 12 : is a block diagram showing a configuration example of hardware of a computer that executes a program for the information processing apparatus according to the above-described embodiment to perform a series of processes.

[0116] exist Figure 12 In the illustrated computer 900 , a central processing unit (CPU) 901 , a read-only memory (ROM) 902 , and a random access memory (RAM) 903 are connected to one another via a bus 904 .

[0117] An input / output interface 910 is also connected to the bus 904. An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input / output interface 910.

[0118] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a nonvolatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0119] In the computer configured as described above, the CPU 901 loads a program stored in, for example, the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executes the program, thereby executing the above-described series of processes. The RAM 903 also appropriately stores data and the like necessary for the CPU 901 to execute various types of processes.

[0120] The program executed by the computer can be applied after being recorded in the removable medium 921 as a package medium, etc. In this case, by attaching the removable medium 921 to the drive 915 , the program can be installed in the storage unit 913 via the input / output interface 910 .

[0121] Alternatively, the program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0122] Alternatively, the program may be installed in advance in the ROM 902 or the storage unit 913 .

[0123] <Application of this technology>

[0124] This technology can be applied to any image encoding / decoding technology. That is, unless otherwise inconsistent with the above-mentioned technology, various types of processing related to image encoding / decoding, such as transformation (inverse transformation), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the examples described above. In addition, unless otherwise inconsistent with the above-mentioned technology, some of these processes may be omitted.

[0125] In addition, the present technology can be applied to a multi-view image encoding / decoding system that encodes / decodes a multi-view image including images from multiple viewpoints (views). In this case, the present technology can be applied to encoding and decoding from each viewpoint (view).

[0126] In addition, the present technology can be applied to a hierarchical (tiered) image encoding (scalable encoding) / decoding system that encodes / decodes hierarchical images of multiple layers (multiple levels) to have a scalable function for predetermined parameters. In this case, the present technology can be applied to the encoding and decoding of each level (layer).

[0127] In addition, although the image encoding device 100 , the image decoding device 200 , and the image processing system 500 (the file generation device 501 and the client device 503 ) have been described above as application examples of the present technology, the present technology can be applied to any configuration.

[0128] For example, the present technology can be applied to various electronic devices, such as transmitters or receivers (e.g., television receivers or mobile phones) in satellite broadcasting, wired broadcasting such as cable television, distribution via the Internet, distribution to terminals via cellular communications, etc.; or devices that record images on media such as optical disks, magnetic disks, and flash memories or reproduce images from these storage media (e.g., hard disk recorders or camera devices).

[0129] In addition, for example, the present technology can also be implemented as a partial configuration of an apparatus such as a processor as a system large-scale integration (LSI) (e.g., a video processor), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a collection obtained by further adding other functions to the unit (e.g., a video collection).

[0130] Furthermore, for example, the present technology can also be applied to a network system comprising multiple devices. For example, the present technology can be implemented as cloud computing, where multiple devices share and jointly process data over a network. For example, the present technology can be implemented in a cloud service for providing image (moving image) related services to specific terminals such as computers, audio-visual (AV) devices, portable information processing terminals, or Internet of Things (IoT) devices.

[0131] Note that in this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected to each other via a network, as well as a single device consisting of multiple modules housed in a single housing, are both referred to as systems.

[0132] <Fields of application of this technology and uses of this technology>

[0133] Systems, devices, processing units, and the like using this technology can be used in any field, such as transportation, medical care, crime prevention, agricultural product processing, animal husbandry, mining, beauty, factories, home appliances, weather and nature monitoring. In addition, this technology can be used for any purpose.

[0134] For example, the present technology can be applied to systems or devices that are provided for providing content for enjoyment, etc. In addition, for example, the present technology can also be applied to systems or devices provided for transportation, such as traffic condition monitoring or autonomous driving control. In addition, for example, the present technology can also be applied to systems or devices provided for safety. In addition, for example, the present technology can also be applied to systems or devices provided for automatic control of machinery, etc. In addition, for example, the present technology can also be applied to systems or devices provided for agricultural product processing and animal husbandry. In addition, for example, the present technology can also be applied to systems or devices that monitor natural conditions such as volcanoes, forests or oceans, wild animals, etc. In addition, for example, the present technology can also be applied to systems or devices provided for sports.

[0135] <Other>

[0136] Note that in this specification, a "flag" is information for identifying a plurality of states, and includes not only information for identifying two states (i.e., true (1) and false (0)), but also information capable of identifying three or more states. Therefore, a "flag" can take a binary value such as 1 / 0, or a value in a ternary or higher base. That is, a "flag" can include any number of bits, for example, one or more bits. In addition, it is assumed that identification information (including a flag) includes not only identification information in a bit stream, but also information about the difference between the identification information and specific reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only information about the flag and identification information, but also information about the difference between the flag and identification information and the reference information.

[0137] In addition, various types of information (metadata, etc.) about the coded data (bitstream) can be transmitted or recorded in any form, as long as the information is associated with the coded data. Here, the term "associated" means that, for example, when processing one piece of data, another piece of data can be used (linked). That is, data associated with each other can be integrated into one piece of data, or can be provided as separate data. For example, information associated with the coded data (image) can be transmitted on a transmission path different from the transmission path of the coded data (image). In addition, for example, information associated with the coded data (image) can be recorded on a recording medium different from the recording medium of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that the data can be "associated" with each other partially rather than as a whole. For example, an image and information corresponding to the image can be associated with each other in any unit, such as multiple frames, a single frame, or a portion of a frame.

[0138] Note that in this specification, terms such as "synthesize", "multiplex", "add", "integrate", "include", "store", "put in", "place" and "insert" all refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to a way of "association" as described above.

[0139] In addition, the embodiment of the present technology is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present technology.

[0140] For example, a configuration described as a single device (or a single processing unit) may be divided so as to be configured as a plurality of devices (or a plurality of processing units). Conversely, a configuration described above as a plurality of devices (or a plurality of processing units) may be integrated so as to be configured as a single device (or a single processing unit). In addition, it goes without saying that a configuration other than the above may be added to the configuration of each device (or each processing unit). Furthermore, a portion of the configuration of a specific device (or a specific processing unit) may be included in the configuration of another device (or other processing unit), as long as the overall configuration and operation of the system remain substantially unchanged.

[0141] In addition, for example, the above program can be executed in any device. In this case, it is only required that the device has necessary functions (functional blocks, etc.) and can obtain necessary information.

[0142] In addition, for example, each step in a flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when multiple types of processing are included in a single step, the multiple types of processing may be executed by a single device, or may be shared and executed by multiple devices. In other words, the multiple types of processing included in a single step may also be executed as processing in multiple steps. Conversely, processing described as multiple steps may also be executed collectively as a single step.

[0143] In addition, for example, regarding a program executed by a computer, the processing of the steps describing the program may be executed in a time series in the order described in this specification, or may be executed in parallel or individually at necessary timings, such as when the program is called. In other words, the processing of the steps may be executed in an order different from the above order as long as no contradiction arises. Furthermore, the processing of the steps describing the program may be executed in parallel with the processing of other programs, or may be executed in combination with the processing of other programs.

[0144] In addition, for example, each of the multiple technologies related to the present technology can be implemented independently and individually, as long as no contradiction arises. It goes without saying that multiple technologies can be implemented in any combination. For example, some or all of the technologies described in one embodiment can be implemented in combination with some or all of the technologies described in other embodiments. In addition, some or all of the technologies described above can be implemented in combination with other technologies not described above.

[0145] In addition, for example, each of the multiple technologies related to the present technology can be implemented independently and individually, as long as no contradiction arises. It goes without saying that multiple technologies can be implemented in any combination. For example, some or all of the technologies described in one embodiment can be implemented in combination with some or all of the technologies described in other embodiments. In addition, some or all of the technologies described above can be implemented in combination with other technologies not described above.

[0146] Note that the present technology can also adopt the following configurations. (1)

[0148] An information processing device, comprising:

[0149] an encoding unit that encodes a picture including two or more sub-pictures to generate encoded data;

[0150] a metadata generating unit that generates sub-picture mapping information, the sub-picture mapping information including first size information of a first sub-picture among the two or more sub-pictures and information regarding second size information of each of one or more second sub-pictures other than the first sub-picture as being the same as the first size information of the first sub-picture; and

[0151] A bitstream generating unit is configured to generate a bitstream including the encoded data and the sub-picture mapping information. (2)

[0153] The information processing device according to (1), wherein

[0154] The first size information is defined in units of coding tree units (CTUs). (3)

[0156] The information processing device according to (1) or (2), wherein

[0157] The first sub-picture is a sub-picture located at the upper left end of the picture. (4)

[0159] The information processing device according to any one of (1) to (3), wherein

[0160] The first size information includes information about the width and height of the first sub-picture. (5)

[0162] The information processing device according to any one of (1) to (4), wherein

[0163] The sub-picture mapping information further includes position information indicating a position of each of the first sub-picture and the one or more second sub-pictures in the picture. (6)

[0165] The information processing device according to any one of (1) to (4), wherein

[0166] The sub-picture mapping information also includes first position information indicating the position of the first sub-picture in the picture and information indicating the position of each of the one or more second sub-pictures in the picture calculated based on the first position information and the first size information. (7)

[0168] An information processing device, comprising:

[0169] an extraction unit, the extraction unit obtaining coded data and sub-picture mapping information from a bitstream;

[0170] a decoding unit that decodes the encoded data to restore a picture, and divides the restored picture into two or more sub-pictures based on the sub-picture mapping information; and

[0171] a rendering unit, the rendering unit rendering the picture based on the two or more sub-pictures,

[0172] The sub-picture mapping information includes first size information of a first sub-picture among the two or more sub-pictures and information that regards second size information of each of one or more second sub-pictures other than the first sub-picture as being the same as the first size information of the first sub-picture. (8)

[0174] The information processing device according to (7), wherein

[0175] The first size information is defined in units of coding tree units (CTUs). (9)

[0177] The information processing device according to (7) or (8), wherein

[0178] The first sub-picture is a sub-picture located at the upper left end of the picture. (10)

[0180] The information processing device according to any one of (7) to (9), wherein

[0181] The first size information includes information about the width and height of the first sub-picture. (11)

[0183] The information processing device according to any one of (7) to (10), wherein

[0184] The sub-picture mapping information further includes position information indicating a position of each of the first sub-picture and the one or more second sub-pictures in the picture. (12)

[0186] The information processing device according to any one of (7) to (10), wherein

[0187] The sub-picture mapping information further includes first position information indicating a position of the first sub-picture in the picture and information calculating second position information indicating a position of each of the one or more second sub-pictures in the picture based on the first position information and the first size information, and

[0188] The decoding unit calculates second position information indicating a position of each of the one or more second sub-pictures in the picture based on the first position information and the first size information. (13)

[0190] An information processing method, comprising:

[0191] encoding a picture including two or more sub-pictures to generate encoded data;

[0192] generating sub-picture mapping information including first size information of a first sub-picture among the two or more sub-pictures and information regarding second size information of each of one or more second sub-pictures other than the first sub-picture as being the same as the first size information of the first sub-picture; and

[0193] A bitstream including the encoded data and the sub-picture mapping information is generated. (14)

[0195] An information processing method, comprising:

[0196] Get coded data and sub-picture mapping information from the bitstream;

[0197] decoding the encoded data to restore a picture, and dividing the restored picture into two or more sub-pictures based on the sub-picture mapping information; and

[0198] presenting the picture according to the two or more sub-pictures,

[0199] The sub-picture mapping information includes first size information of a first sub-picture among the two or more sub-pictures and information that regards second size information of each of one or more second sub-pictures other than the first sub-picture as being the same as the first size information of the first sub-picture.

[0200] Reference Signs List

[0201] 100 Image Coding Device

[0202] 101 coding units

[0203] 102 Metadata Generation Unit

[0204] 103 Bitstream Generation Unit

[0205] 200 Image Decoding Device

[0206] 201 Analysis Unit

[0207] 202 Extraction Unit

[0208] 203 Decoding Unit

[0209] 204 Presentation Unit

Claims

1. An information processing device, comprising: an encoding unit that encodes a picture including two or more sub-pictures to generate encoded data; a metadata generating unit that generates sub-picture mapping information, the sub-picture mapping information including first size information of a first sub-picture among the two or more sub-pictures and a flag indicating that second size information of each of one or more second sub-pictures other than the first sub-picture is regarded as being the same as the first size information of the first sub-picture; and a bitstream generating unit, the bitstream generating unit generating a bitstream including the coded data and the sub-picture mapping information, Wherein, when the sizes of the two or more sub-pictures are the same, the metadata generation unit sets the flag and omits the second size information of the one or more second sub-pictures in the sub-picture mapping information.

2. The information processing device according to claim 1, wherein The first size information is defined in units of coding tree units (CTUs).

3. The information processing device according to claim 1, wherein The first sub-picture is a sub-picture located at the upper left end of the picture.

4. The information processing device according to claim 1, wherein: The first size information includes information about the width and height of the first sub-picture. The information processing device according to claim 1 , wherein: The sub-picture mapping information further includes position information indicating a position of each of the first sub-picture and the one or more second sub-pictures in the picture. The information processing apparatus according to claim 1 , wherein: The sub-picture mapping information also includes first position information indicating the position of the first sub-picture in the picture and information indicating the position of each of the one or more second sub-pictures in the picture calculated based on the first position information and the first size information.

7. An information processing device comprising: an extraction unit, the extraction unit obtaining coded data and sub-picture mapping information from a bitstream; a decoding unit that decodes the encoded data to restore a picture, and divides the restored picture into two or more sub-pictures based on the sub-picture mapping information; as well as a rendering unit, the rendering unit rendering the picture based on the two or more sub-pictures, The sub-picture mapping information includes first size information of a first sub-picture among the two or more sub-pictures and a flag, wherein the flag indicates that second size information of each of one or more second sub-pictures other than the first sub-picture is regarded as the same as the first size information of the first sub-picture, Wherein, when the sizes of the two or more sub-pictures are the same, the flag is set, and the second size information of the one or more second sub-pictures is omitted in the sub-picture mapping information.

8. The information processing apparatus according to claim 7, wherein: The first size information is defined in units of coding tree units (CTUs).

9. The information processing apparatus according to claim 7, wherein: The first sub-picture is a sub-picture located at the upper left end of the picture.

10. The information processing apparatus according to claim 7, wherein: The first size information includes information about the width and height of the first sub-picture.

11. The information processing apparatus according to claim 7, wherein: The sub-picture mapping information further includes position information indicating a position of each of the first sub-picture and the one or more second sub-pictures in the picture.

12. The information processing apparatus according to claim 7, wherein: The sub-picture mapping information further includes first position information indicating a position of the first sub-picture in the picture and information calculating second position information indicating a position of each of the one or more second sub-pictures in the picture based on the first position information and the first size information, and The decoding unit calculates second position information indicating a position of each of the one or more second sub-pictures in the picture based on the first position information and the first size information.

13. An information processing method, comprising: encoding a picture including two or more sub-pictures to generate encoded data; generating sub-picture mapping information, the sub-picture mapping information including first size information of a first sub-picture among the two or more sub-pictures and a flag indicating that second size information of each of one or more second sub-pictures other than the first sub-picture is regarded as being the same as the first size information of the first sub-picture; and generating a bitstream including the coded data and the sub-picture mapping information, Wherein, when the sizes of the two or more sub-pictures are the same, the flag is set, and the second size information of the one or more second sub-pictures is omitted in the sub-picture mapping information.

14. An information processing method, comprising: Get coded data and sub-picture mapping information from the bitstream; decoding the encoded data to restore a picture, and dividing the restored picture into two or more sub-pictures based on the sub-picture mapping information; as well as presenting the picture according to the two or more sub-pictures, The sub-picture mapping information includes first size information of a first sub-picture among the two or more sub-pictures and a flag, wherein the flag indicates that second size information of each of one or more second sub-pictures other than the first sub-picture is regarded as the same as the first size information of the first sub-picture, Wherein, when the sizes of the two or more sub-pictures are the same, the flag is set, and the second size information of the one or more second sub-pictures is omitted in the sub-picture mapping information. 15 . A computer-readable storage medium having a computer-executable program stored thereon, which, when executed by a processor, causes the processor to execute the information processing method according to claim 13 or 14 .

16. A computer program product comprising a computer program / instructions, wherein: When the computer program / instructions are executed by a processor, the steps of the information processing method according to claim 13 or 14 are implemented.

Images