Subpictures and Subpicture Sets with Level Derivation - Patent application

By introducing subpictures and subpicture sets with level derivation, the HEVC standard addresses the high bitrate overhead issue in signaling subregions, enhancing decoding efficiency and interoperability in video coding.

JP7596310B2Active Publication Date: 2024-12-09INTEL CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2021568820
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-06-25
Filing Date
2020-06-23
Publication Date
2024-12-09
Estimated Expiration
2040-06-23

AI Technical Summary

Technical Problem

The High Efficiency Video Coding (HEVC) standard faces challenges with high bitrate overhead when signaling all possible combinations of subregions within a video sequence, particularly when using Temporal Motion Constrained Tile Sets, which affects interoperability and efficiency in video decoding.

Method used

The implementation of subpictures and subpicture sets with level derivation, where subpictures are rectangular regions within a coded picture, allowing for independent decoding and signaling of level indicators through a sequence parameter set (SPS) and optional SEI messages, reducing bitrate overhead and enhancing interoperability.

Benefits of technology

This approach enables easier sub-bitstream extraction and bitstream merging with reduced bitrate overhead, improving decoding efficiency and interoperability across different video coding standards, including Versatile Video Coding (VVC).

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007596310000006
    Figure 0007596310000006
  • Figure 0007596310000007
    Figure 0007596310000007
  • Figure 0007596310000008
    Figure 0007596310000008
Patent Text Reader

Abstract

Embodiments of the video codec may include techniques for deriving conformance points for sub-regions of coded images in a coded video sequence in the video data, grouping any combination of sub-pictures that form rectangular regions into sub-picture sets, and / or deriving level indicators corresponding to sub-pictures based on the level of the coded video sequence and the relative sizes of the coded images and sub-picture sets. Other embodiments are disclosed and claimed.
Need to check novelty before this filing date? Find Prior Art

Description

[Background technology]

[0001] Claiming priority This application claims priority to U.S. Provisional Patent Application No. 62 / 866,156, filed June 25, 2019, entitled “SUB-PICTURES AND SUB-PICTURE SETS WITH LEVEL DERIVATION,” which is incorporated herein in its entirety.

[0002] Video coding and decoding are useful for reducing the amount of data transmitted to process video information. A video codec refers to electronic circuitry or software that compresses or decompresses digital video. For example, a video codec converts uncompressed video to a compressed format or decompresses a compressed format back to uncompressed video. Examples of compressed video formats include MP4, 3GP, OGG, WMV, FLV, AVI, MPEG-2PS, MPEG, VOB, VP9, ​​etc. Examples of video codecs include H.264, High Efficiency Video Coding (HEVC), MPEG-4, QUICKTIME, DV, etc. [Brief description of the drawings]

[0003] The materials described herein are shown by way of example, and not by way of limitation, in the accompanying drawings. For simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, dimensions of some elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference labels have been repeated among the figures to indicate corresponding or analogous elements. [Figure 1A] FIG. 1A is an exemplary illustration of valid sub-picture grid values ​​according to one embodiment. [Figure 1B]FIG. 1B is an exemplary illustration of invalid sub-picture grid values ​​according to one embodiment. [Figure 2A] FIG. 2A is an exemplary illustration of images organized into sub-pictures according to one embodiment. [Figure 2B] FIG. 2B is an exemplary illustration of images organized into sub-pictures and sub-picture sets according to one embodiment. [Figure 2C] FIG. 2C is an exemplary illustration of an image organized as sub-pictures and separate sub-picture sets according to one embodiment. [Figure 2D] FIG. 2D is an exemplary illustration of an image organized as sub-pictures and separate sub-picture sets according to one embodiment. [Diagram 3] FIG. 3 is an exemplary block diagram of an electronic system according to one embodiment; [Figure 4] FIG. 4 is an exemplary block diagram of an electronic device according to one embodiment. [Diagram 5] FIG. 5 is an exemplary flow chart of a method for processing video data according to one embodiment. [Figure 6] FIG. 6 is an illustration of an exemplary system. [Figure 7] FIG. 7 illustrates one example small form factor device, all configured in accordance with at least some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0004] One or more embodiments or implementations will now be described with reference to the accompanying drawings. While particular configurations and arrangements are described, it should be understood that this is done for illustrative purposes only. Those skilled in the art will appreciate that other configurations and arrangements may be employed without departing from the spirit and scope of the present specification. Those skilled in the art will appreciate that the techniques and / or arrangements described herein may be used in a variety of other systems and applications other than those described herein.

[0005] Although the following description illustrates various implementations that may be manifested in architectures such as, for example, system-on-chip (SoC) architectures, implementations of the techniques and / or configurations described herein are not limited to a particular architecture and / or computing system, and may be implemented by any architecture and / or computing system for the same purpose. For example, various architectures using multiple integrated circuit (IC) chips and / or packages, and / or various computing devices and / or consumer electronics (CE) devices such as set-top boxes, smartphones, etc., may implement the techniques and / or configurations described herein. Furthermore, although the following description may specify many specific details, such as logic implementation, types and interrelationships of system components, logical partitioning / integration options, etc., the claimed subject matter may be practiced without such specific details. In other instances, some material, such as, for example, control structures and complete software instruction sequences, may not be shown in detail in order not to obscure the material described herein.

[0006] The material described herein may be implemented in hardware, firmware, software, or any combination thereof. The material described herein may be implemented as instructions stored on a machine-readable medium that may be read and executed by one or more processors. A machine-readable medium may include any medium and / or mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing device). For example, a machine-readable medium may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.), and the like.

[0007] References herein to "one implementation," "an implementation," "an example implementation," and the like indicate that the described implementation may include a particular feature, structure, or characteristic, but all embodiments may not necessarily include the particular feature, structure, or characteristic. Moreover, such references do not necessarily refer to identical implementations. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one of ordinary skill in the art to implement such feature, structure, or characteristic in connection with other embodiments, whether or not explicitly described herein.

[0008] Methods, apparatus, systems, and articles of manufacture are described herein with respect to video systems. More particularly, embodiments relate to sub-pictures and sub-picture sets with level derivation.

[0009] For use in video coding (e.g., including video coding standards), embodiments of the bitrate efficient mechanism may enable derivation of conformance points for sub-regions of a coded picture within a coded video sequence.

[0010] In the High Efficiency Video Coding (HEVC) video coding standard, for each coded video sequence (CVS), profile, tier, and level indicator syntax elements are signaled in the bitstream to define the conformance of the bitstream and enable interoperability of decoders conforming to a particular profile, tier, and level. The signaled profile, tier, and level correspond to a video sequence that contains an entire image.

[0011] HEVC includes a Supplemental Enhancement Information (SEI) message for Temporal Motion Constrained Tile Sets that can indicate sub-regions of images in a sequence, called tile sets, that can be decoded independently. Hierarchical and level information can be optionally transmitted for tiles or tile sets.

[0012] When using the HEVC Temporal Motion Constrained Tile Sets SEI message, the grouping of tiles into tile sets for which conformance point information is signaled must be pre-identified and then explicitly signaled for each identified combination. The bitrate overhead is high to signal all possible combinations, especially when the number of tiles is large.

[0013] Some embodiments may provide techniques that allow any combination of sub-pictures forming a rectangular region to be grouped into a sub-picture set, and a level indicator corresponding to any sub-picture to be derived based on the CVS level and the relative size of the coded picture and the sub-picture set. Advantageously, some embodiments may enable easier sub-bitstream extraction and bitstream merging with clear interoperability and low bitrate overhead. For example, some embodiments may be useful for various video coding standards, including the Versatile Video Coding (VVC) standard.

[0014] Some embodiments may include techniques to support sequence-level sub-pictures and sub-picture sets. Sub-pictures can correspond to rectangular regions and are independently decodable for the entire sequence. Tiles and slices do not cross sub-picture boundaries. Sub-picture sets are derived based on any combination of sub-pictures that form a rectangular region and are determined by external means. Sub-picture parameters are signaled in a sequence parameter set (SPS) using a grid indicating sub-picture size and position. Some embodiments may include an SPS flag indicating that a level may be derived for any sub-picture or sub-picture set based on the scaling of the image size relative to the sample size of the sub-picture or sub-picture set, respectively. Alternatively, an SEI message may be used to signal additional hierarchy and level information for a sub-picture or sub-picture set. A fixed-length sub_pic_id syntax element is signaled early in the slice header.

[0015] Subpicture definition example A subpicture may correspond to a rectangular region of the coded picture. Subpicture size may vary within a picture. Each particular subpicture size and position is consistent for all pictures in a sequence.

[0016] Each sub-picture sequence may be independently decodable. Titles and slices cannot extend beyond sub-picture boundaries. The encoder may ensure that each sub-picture is independently decodable. Semantic constraints on the bitstream may also be useful to impose this constraint.

[0017] The arrangement of tiles, slices, and bricks within a subpicture may vary for pictures in a sequence.

[0018] Sub-picture signaling example Subpicture parameters are signaled in the SPS. Subpicture mapping is signaled to associate sample positions within a picture to subpicture IDs based on a grid. The granularity of the grid is selected by the encoder and is explicitly signaled for the horizontal and vertical dimensions. The syntax in the SPS indicates a sub_pic_id for each grid position, similar to the signaling mechanism proposed in JVET-N0275 for signaling sections of a picture.

[0019] The coded image is divided into a grid, determined by the signaled number of horizontal and vertical grid elements into which the image is divided. For every (i,j)th grid location, sub_pic_id[i][j] is signaled. There is no restriction on the ordering of the sub_pic_id values. However, since subpictures are required to be contiguous rectangular, all grid samples with a particular sub_pic_id value must be contiguous and rectangular. For example, Figure 1A shows valid examples of sub_pic_id values ​​for a grid. Figure 1B shows invalid examples, since subpicture 3 is not rectangular.

[0020] The encoder can determine the grid element size. Using a finer grid granularity requires more signaling overhead but provides more flexibility for varying sizes of sub-pictures. In some embodiments of the syntax, the granularity is 4x4, but other options are possible, including the minimum coding tree block (CTB) size. The total overhead for sub-picture parameter signaling increases with the number of sub-pictures.

[0021] An advantage of the grid signaling approach is that it is very easy to change the sub-picture ID assignments in the coded bitstream, for example for a merging or splitting process.

[0022] Some embodiments may utilize conventional tile signaling, which may be done in a picture parameter set (PPS). It may be beneficial to revise the tile parameter signaling based on sub-picture parameters present in the SPS, but this may violate limitations in the independent analysis of parameter sets.

[0023] Some syntax elements in the SPS are converted to fixed-length code in some embodiments to allow easier access to the sub-picture parameters in the SPS.

[0024] In some embodiments of the syntax, the number of sub-pictures is limited to 256, with the maximum value reserved to allow for expansion if support for more sub-pictures is desired.

[0025] Slice Header Signaling Example Some embodiments may include the slice_sub_pic_id syntax element at the start of the slice header, immediately following the slice PPS. The slice PPS may be modified to be a fixed bit width field to allow easy access to and / or modification of the slice_sub_pic_id syntax element.

[0026] In some embodiments, the signaling and / or calculation of slice_address or brick parameters is based on sub_pic_id[i][j], SubPicTop[slice_sub_pic_id], and may be modified based on the value of SubPicLeft[slice_sub_pic_id].

[0027] Subpicture Set Example A sub-picture set may consist of one or more sub-pictures. Sub-picture set parameters are not explicitly signaled in the SPS, but may be derived based on any combination of sub-pictures that form a rectangular region. External means may be used to determine which pictures are present in a sub-picture set that the decoder is expected to decode.

[0028] Figure 2A shows an image divided into several sub-pictures, while Figures 2B, 2C, and 2D show examples of different sub-picture sets that may be formed from these sub-pictures, with thicker outlines indicating the respective sets of sub-pictures (e.g., the set of sub-pictures 0 and 1 in Figure 2B, the set of sub-pictures 0, 1, and 3 in Figure 2C, and the set of sub-pictures 0, 1, and 2 in Figure 2D).

[0029] An example use case is for 360 video coding using cube maps. For example, each of the faces of a 6 cube can be divided into 4 regions, creating 24 sub-pictures. The decoder can choose to decode and render only the sub-pictures that contain the sub-picture that corresponds to the region being viewed.

[0030] Example Profiles, Tiers, and Levels for Subpicture and Subpicture Set Sequences All sub-picture sequences and sub-picture set sequences conform to the same profile as CVS. Some embodiments may include an SPS flag to indicate that the levels of all sub-picture sequences and sub-picture set sequences for CVS may be derived based on the ratio of the sub-picture or sub-picture set size, respectively, to the coded picture size. When this flag is set, the same layers as CVS are used for the sub-picture sequences and sub-picture set sequences.

[0031] The encoder may ensure that all sub-pictures and all possible sub-picture set sequences fit into the derived hierarchy and level when the flag is set, including any restrictions on bit rate, sample rate, etc.

[0032] Example SEI Message for Explicit Profile, Layer, and Level Signaling for Subpicture Sequences and Subpicture Set Sequences If additional level signaling is desired, some embodiments may utilize SEI messages that allow for layer and level signaling for any desired subpictures and subpicture sets. The syntax embodiment may be different from that used in the HEVC temporal motion-constrained tile sets SEI message. Advantageously, the syntax embodiment is more easily modifiable when the merged bitstream is formed.

[0033] Sequence Parameter Set Syntax Example [Table 1-1] [Table 1-2] sub_pics_present_flag equal to 1 indicates the presence of sub-picture parameters in the syntax structure. sub_pics_present_flag equal to 0 indicates the absence of sub-picture parameters in the syntax structure. sub_pic_derive_tier_level_flag equal to 1 indicates that the tier and level indicators of all subpictures and subpicture sets in the CVS may be derived. max_sub_pics_minus1 plus 1 specifies the maximum number of subpictures that can exist in the CVS. max_sub_pics_minus1 ranges from 0 to 254. The value 255 may be reserved for future use. sub_pic_grid_element_width_minus1 specifies the width of each element of the subpicture identifier grid in units of 8 samples. The length of this syntax element is: Ceil(Log2(PicWidthInMinCbsY)) bits. sub_pic_grid_element_height_minus1 specifies the height of each element of the subpicture identifier grid in units of 8 samples. The length of this syntax element is: Ceil(Log2(PicHeightInMinCbsY)) bits. sub_pic_id[i][j] specifies the subpicture identifier for the (i,j)th grid position. The length of this syntax element is Ceil(Log2(max_sub_pics_minus1+1)) bits.

[0034] The following are bitstream conformance requirements: if(i>0 && j>0) if(sub_pic_id[i][j]==sub_pic_id[i-1][j]&&(sub_pic_id[i][j]== sub_pic_id[i][j-1]) sub_pic_id[i][j]shallequal sub_pic_id[i-1][j-1]

[0035] The variables SubPicTop[i[j]] and SubPicLeft[i][j] are derived as follows. if(i==0) SubPicTop[sub_pic_id[i][j]]=0 Else if(sub_pic_id[i][j]!=sub_pic_id[i-1][j]) SubPicTop[sub_pic_id[i][j]]=i if(j==0) SubPicLeft[sub_pic_id[i][j]]=0 Else if(sub_pic_id[i][j]!=sub_pic_id[i][j-1]) SubPicLeft[sub_pic_id[i][j]]=j

[0036] Slice Header Example [Table 2] slice_sub_pic_id specifies the subpicture identifier of the slice. The length of this syntax element is Ceil(Log2(max_sub_pics_minus1+1)) bits.

[0037] Example of SEI message [Table 3] sp_id_len specifies the number of bits used to signal the sp_sub_ic_id syntax element. sp_tier_present_flag equal to 1 specifies that the sp_tier_idc[sp_sub_pic_id] and sp_set_tier_idc[sp_sub_pic_set_id] syntax elements are present in the syntax structure. sp_tier_present_flag equal to 0 specifies that the sp_tier_idc[sp_sub_pic_id] and sp_set_tier_idc[sp_sub_pic_set_id] syntax elements are not present in the syntax structure. num_sub_pic_params_minus1 +1 specifies the number of subpictures for which parameters exist in the syntax structure. sp_sub_pic_id specifies the subpicture identifier for the signaled parameters. sp_tier_idc[sp_sub_pic_id] specifies the level context for the interpretation of sp_level_idc[sp_sub_pic_id]. If not present, the value of sp_tier_idc[sp_sub_pic_id] is inferred to be equal to the tier_idc for the active SPS. sp_level_idc[sp_sub_pic_id] specifies the level to which the sp_sub_pic_id-th sub-bitstream sequence conforms. If not present, the value of sp_level_idc[sp_sub_pic_id] is inferred to be equal to the level_idc for the active SPS. num_sub_pic_sets_params specifies the number of sub-picture sets for which parameters exist in the syntax structure. sp_sub_pic_set_id specifies the sub-picture set identifier for the signaled parameters. sp_sub_pic_id_in_set[sp_sub_pic_set_id][j] equal to 1 specifies that the jth subpicture is included in the sp_sub_pic_set_idth subpicture set. sp_sub_pic_id_in_set[sp_sub_pic_set_id][j] equal to 0 specifies that the jth subpicture is not included in the sp_sub_pic_set_idth subpicture set. sp_set_tier_idc[sp_sub_pic_set_id] specifies the level context for interpretation of sp_level_idc[sp_sub_pic_id]. If not present, the value of sp_tier_idc[sp_sub_pic_set_id] is inferred to be equal to the tier_idc of the active SPS. sp_set_level_idc[sp_sub_pic_set_id] specifies the level to which the sp_sub_pic_id-th sub-bitstream sequence conforms. If not present, the value of sp_set_level_idc[sp_sub_pic_set_id] is inferred to be equal to the level_idc of the active SPS.

[0038] Example of the Subpicture Extraction Process An external means is expected to provide a list of subpicture ID values ​​contained in a subpicture set applicable across the CVS to be used in the subpicture extraction process. The subpicture set must form a rectangle.

[0039] The sub-bitstream extraction process discards all NAL units that are not in the list of sub-picture ID values ​​contained in the sub-picture set.

[0040] The sub-picture set extraction process imposes the constraint that the extracted sub-picture set sequence is a valid decodable bitstream.

[0041] In alternative embodiments, the sub-picture sets need not be rectangular in shape.

[0042] Level derivation example Some embodiments may derive the level value for the sub-picture set (e.g., based on the level limits of HEVC Annex A). For example, in HEVC Annex A, the value of the general_level_idc syntax element is 30 times (30*) the level value shown in Table A.6.

[0043] The level for an individual subpicture is derived by deriving the level for the subpicture set that contains only the individual subpicture.

[0044] The derivation is based on scaling the maximum luma sample rate of the CVS level_idc by the ratio of the sub-picture set size to the picture size, and is based on the assumption that the frame rate for sub-pictures in a sequence is the same as for pictures in the sequence.

[0045] The level restrictions in CVSe, including bitrate restrictions, also apply these constraints to the extracted sub-picture set sequence, and it is the encoder's responsibility to ensure that all sub-pictures and all possible sub-picture sets obey these constraints.

[0046] The set of sub-pictures contained in a sub-picture set is provided by external means.

[0047] The size of the nth subpicture in units of grid samples, SubPicNumGridSamples[n], for n in 0..max_sub_pics_minus1, is derived as follows: SubPicNumGridSamples[n]=0 for(i=0; i <SubPicGridRows; i++) for(j=0; i <SubPicGridCols; j++) if(n=sub_pic_id[i][j]) SubPicNumGridSamples[n]++

[0048] The size of the subpicture set in units of grid samples, SubPicSetNumGridSamples, is derived as follows: SubPicSetNumGridSamples=0 for(n=0; n<=max_sub_pics_minus1; n++) if(sub-picture n is in the sub-picture set) SubPicSetNumGridSamples += SubPicNumGridSamples[n]

[0049] SubPicSetSizeY, the set subpicture set size in units of luma samples, SubPicSetNumGridSamples*(sub_pic_grid_element_width_minus1*4+1)* It is set equal to (sub_pic_grid_element_height_minus*4+1).

[0050] The values ​​of MaxLumarSr[LevelIdcl] and MaxLumaSr[LevelIdc] are from Table AX. For values ​​of LevelIdc not in Table AX, MaxLumaPs[LevelIdc]=0 and MaxLumaSr[LevelIdc]=0. Column A corresponds to “Level;”, column B corresponds to “Level Idc;”, column C corresponds to “Max Luma picture size MaxLumaPs[LevelIdc](samples);”, and column D corresponds to Corresponds to "Max Luma sample rate MaxLumaSr[LevelIdc](samples / sec);". [Table 4] Table AX Level Limit

[0051] The sub-picture set sequence level indicator value, SubPicSetLevelIdc, is derived as follows: SubPicSetLevelIdc=level_idc for(k=level_idc; k>0; k --) if(SubPicSetSizeY<=MaxLumaPs[k] && MaxLumaSr[level_idc]*SubPicSetSizeY / PicSizeInSamplesY<=MaxLumaSr[k]) SubPicSetLevelIdc=k

[0052] In an alternative embodiment, a sub-picture set need not be rectangular in shape: instead, the derivation level for a sub-picture set is based on the size in samples of a bounding rectangle that contains the sub-picture set, rather than the size in samples of the sub-picture set itself.

[0053] Various components of the systems described herein may be implemented in software, firmware, and / or hardware, and / or any combination thereof. For example, various components of the systems or devices described herein may be provided, at least in part, by hardware in a computing system-on-chip (SoC), such as found in a computing system, such as a smartphone. Those skilled in the art will recognize that the systems described herein may include additional components not shown in the corresponding figures. For example, the systems described herein may include additional components not shown for clarity, such as bitstream multiplexer or de-multiplexer modules, etc.

[0054] Although implementations of the example processes described herein may include the performance of all of the operations shown in the order illustrated, the disclosure is not limited in this respect and, in various examples, implementations of the example processes described herein may include only a subset of the operations illustrated, operations performed in an order different from those illustrated, or additional operations.

[0055] Additionally, any one or more of the operations described herein may be performed in response to instructions provided by one or more computer program products. Such program products may include, for example, a signal-bearing medium that provides instructions that, when executed by a processor, can provide the functionality described herein. A computer program product may be provided in any form of one or more machine-readable media. Thus, for example, a processor including one or more graphics processing units or processor cores may perform one or more blocks of the example processes herein in response to program code and / or instructions or instruction sets carried to the processor by one or more machine-readable media. In general, a machine-readable medium may carry software in the form of program code and / or instruction sets that cause any of the devices and / or systems described herein to perform at least a portion of the operations described herein and / or the devices, systems, or any modules or components described herein.

[0056] As used in any implementation described herein, the term "module" refers to any combination of software logic, firmware logic, hardware logic, and / or circuitry configured to provide the functionality described herein. Software may be embodied as a software package, code, and / or instruction set. And "hardware" refers to any software, firmware, or hardware that is embodied in a software package, code, and / or instruction set. As used in any implementation described herein, may include, for example, alone or in any combination, hardwired circuitry, programmable circuitry, state machine circuitry, fixed function circuitry, execution unit circuitry, and / or firmware that stores instructions executed by the programmable circuitry. These modules, collectively or individually, may be embodied as circuitry that forms part of a larger system, such as, for example, an integrated circuit (IC), a system on a chip (SoC), etc.

[0057] 3, one embodiment of electronic system 510 may include a memory 512 for storing video data, a processor 511 coupled to memory 512, and logic 513 coupled to processor 511 and memory 512. Logic 513 may be configured to derive a fit point for a sub-region of a coded image in a coded video sequence in the video data. For example, logic 513 may be configured to group a combination of one or more sub-pictures that form a rectangular region in the video data into a sub-picture set, and / or to derive a level indicator corresponding to the sub-picture based on a level of the coded video sequence and the relative size of the coded image and the sub-picture set. Logic 513 may also be configured to implement one or more aspects of the sequence parameter set syntax that may be described herein, to implement one or more aspects of the signaling described herein, and / or to implement one or more aspects of the sub-picture extraction and level derivation described herein. In some embodiments, logic 513 may be configured to signal one or more layer and level information for one or more of the subpictures and subpictures with a message in the video data. In some embodiments, logic 513 may be configured to signal subpicture parameters in a parameter set and indicate a size and position of the subpicture based on a grid. Logic 513 may also be configured to provide an indication in a message in the video data that a level of the subpicture or subpicture set should be derived based on a scaling of the picture size to one of the sample sizes of the subpicture and subpicture set. In any of the embodiments herein, logic 513 may be further configured to provide a VVC codec to utilize the subpicture and the set of subpictures for one or more of encoding and decoding the video data.

[0058] Each of the above-described embodiments of the processor 511, memory 512, logic 513, and other system components may be implemented in hardware, software, or any suitable combination thereof. For example, a hardware implementation may include configurable logic, such as a programmable logic array (PLA), a field programmable gate array (FPGS), a complex programmable logic device (CPLD), or fixed functionality logic hardware using circuit technologies, such as an application specific integrated circuit (ASIC), complementary metal oxide semiconductor (CMOS), or transistor-transistor logic (TTL), or any combination thereof. Embodiments of the processor 511 may include a general purpose processor, a special purpose processor, a central processing unit (CPU), a graphics processor, a general purpose controller, an execution unit, a special purpose controller, a general purpose controller, a microcontroller, and the like. In some embodiments, the logic 513 may be located within or co-located with (e.g., on the same die) various components, including the processor 511.

[0059] Alternatively or additionally, all or a portion of these components may be implemented in one or more modules as a set of logic instructions stored in a machine or computer readable storage medium, such as a random access memory (RAM), a read only memory (ROM), a programmable ROM (PROM), firmware, flash memory, etc., for execution by a processor or computing device. For example, computer program code for carrying out operations of the components may be written in any combination of programming languages ​​applicable / suitable for one or more operating systems (OS), including object-oriented programming languages, such as PYTHON, PERL, JAVA®, SMALLTALK®, C+, C#, etc., and conventional procedural programming languages, such as the “C” programming language or similar programming languages. For example, memory 512, firmware memory, persistent storage medium, or other system memory may store a set of instructions that, when executed by processor 511, cause system 510 to perform one or more components, functions, or aspects of system 510 (e.g., logic 513, grouping sub-pictures into sub-picture sets, deriving level indicators, etc.).

[0060] 4, one embodiment of a video codec device 520 may include one or more substrates 521 and logic 522 coupled to the one or more substrates 521. The logic 522 may be configured to derive matching points for sub-regions of coded images in a coded video sequence in the video data. For example, the logic 522 may be configured to group any combination of sub-pictures that form a rectangular region into a sub-picture set and / or to derive matching points for sub-regions of coded images in a coded video sequence at a level and coding level. Logic 522 may be configured to derive a level indicator corresponding to a sub-picture based on the relative sizes of the picture and the sub-picture set. Logic 522 may also be configured to implement one or more aspects of the sequence parameter set syntax described herein, to implement one or more aspects of the signaling described herein, and / or to implement one or more aspects of the sub-picture extraction and level derivation described herein. In some embodiments, logic 522 may be configured to signal one or more hierarchical and level information for one or more of the sub-pictures and sub-pictures by messages in the video data. In some embodiments, logic 522 may be configured to signal sub-picture parameters in a parameter set and indicate the size and position of the sub-picture based on a grid. Logic 522 may also be configured to provide an indication in a message in the video data that the level of the sub-picture or sub-picture set should be derived based on a scaling of the picture size to one of the sample sizes of the sub-picture and the sub-picture set. In any of the embodiments herein, the logic 522 may be further configured to provide a VVC codec to utilize sub-pictures and sets of sub-pictures for one or more of encoding and decoding video data.

[0061] Some embodiments of logic 522 may additionally or alternatively be configured with complementary decoding logic. For example, logic 522 may be configured to receive a bitstream of video data, identify a sub-picture set to be decoded that corresponds to a group of one or more sub-pictures, derive a sub-picture set level based on a size of the identified sub-picture set, and determine whether the derived level is threshold level decoding. The group of one or more sub-pictures may form a rectangular region in the video data. In some embodiments, logic 522 may be further configured to decode one or more of the layer and level information of the sub-picture set based on a message in the video data. In some embodiments, logic 522 may be further configured to determine sub-picture parameters from a parameter set in the video data and determine a size and position of the sub-picture based on a grid. For example, logic 522 may be configured to determine, based on an indication provided in a message in the video data, that a level for any sub-picture or sub-picture set should be derived based on a scaling of the picture size for one of the sub-pictures and a sample size of the sub-picture set.

[0062] For example, logic 522 may be implemented on a semiconductor device that may include one or more substrates 521 to which logic 522 is coupled. In some embodiments, logic 522 may be implemented at least partially in one or more of configurable logic and fixed functionality hardware logic on a semiconductor substrate 521 (e.g., silicon, sapphire, gallium arsenide, etc.). For example, logic 522 may include a transistor array and / or other integrated circuit components coupled to substrate 521 having transistor channel regions disposed within substrate 521. The interface between logic 522 and substrate 521 need not be an abrupt junction. Logic 522 may also be considered to include an epitaxial layer grown on an initial wafer of substrate 521.

[0063] 5, one embodiment of a method 530 for processing video data may include storing the video data at block 531 and deriving a fit point for a sub-region of a coded picture in a coded video sequence in the video data at block 532. For example, the method 530 may include grouping a combination of one or more sub-pictures forming a rectangular region in the video data into a sub-picture set at block 533 and / or deriving a level indicator corresponding to the sub-picture based on a level of the coded video sequence and the relative size of the coded picture and the sub-picture at block 534. Some embodiments of the method 530 may further include signaling one or more hierarchical and level information for the sub-picture and one or more of the sub-pictures in a message in the video data at block 535. The method 530 may also include signaling sub-picture parameters in a parameter set at block 536 and indicating a size and a position of the sub-picture based on a grid at block 537. For example, the method 530 may further include, at block 538, providing an indication in a message in the video data that the level for any sub-picture or sub-picture set should be derived based on a scaling of the picture size for one of the picture sizes of the sub-picture and the sub-picture set. In any of the embodiments herein, the method 530 may further include, at block 539, providing a VVC codec to utilize the sub-picture and the set of sub-pictures for one or more of encoding and decoding the video data.

[0064] An embodiment of the method 530 may be implemented in, for example, a system, apparatus, computer, device, etc., as described herein. More particularly, a hardware implementation of the method 530 may include configurable logic, such as, for example, PLA, FPGA, CPLD, or fixed functionality logic hardware using circuit technologies, such as, for example, ASIC, CMOS, or TTL technology, or any combination thereof. Alternatively or additionally, the method 530 may be implemented in one or more modules as a set of logic instructions stored in a machine or computer readable storage medium, such as RAM, ROM, PROM, firmware, flash memory, etc., for execution by a processor or computing device. For example, computer program code for carrying out the operations of the components may be written in any combination of programming languages ​​applicable / suitable for one or more operating systems (OS), including object-oriented programming languages, such as PYTHON, PERL, JAVA®, SMALLTALK®, C+, C#, etc., and conventional procedural programming languages, such as the “C” programming language or similar programming languages.

[0065] For example, the method 530 may be implemented on a computer-readable medium as described in connection with Examples 14-19 below. An embodiment or portion of the method 530 may be implemented in firmware, an application (e.g., via an application programming interface (API)), or driver software running on an operating system (OS). In addition, the logic instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, state setting data, configuration data for integrated circuits, state information that personalizes electronic circuits and / or other structural components specific to hardware (e.g., a host processor, a central processing unit / CPU, a microcontroller, etc.).

[0066] 6 is an illustration of an example system 1000 arranged in accordance with at least some implementations of the present disclosure. In various implementations, the system 1000 may be a mobile system, although the system 1000 is not limited in this context. For example, the system 1000 may be incorporated into a personal computer (PC), a laptop computer, an ultra laptop computer, a tablet, a touchpad, a portable computer, a handheld computer, a personal digital assistant (PDA), a mobile phone, a mobile phone / PDA combination, a television, a smart device (e.g., a smartphone, a smart tablet, or a smart television), a mobile internet device (MID), a messaging device, a data communication device, a camera (e.g., a point-and-shoot camera, a super zoom camera, a digital single-lens reflex camera), and the like.

[0067] In various implementations, the system 1000 includes a platform 1002 coupled to a display 1020. The platform 1002 can receive content from a content device, such as a content services device 1030, or a content delivery device 1040, or other similar content source. A navigation controller 1050 including one or more navigation features can be used, for example, to interact with the platform 1002 and / or the display 1020. Each of these components is described in more detail below.

[0068] In various implementations, the platform 1002 may include any combination of a chipset 1005, a processor 1010, a memory 1012, an antenna 1013, storage 1014, a graphics subsystem 1015, applications 1016, and / or wireless devices 1018. The chipset 1005 may provide intercommunication between the processor 1010, the memory 1012, the storage 1014, the graphics subsystem 1015, applications 1016, and / or wireless devices 1018. For example, the chipset 1005 may include a storage adapter (not shown) that may provide intercommunication with the storage 1014.

[0069] The processor 1010 may be implemented as a complex instruction set computer (CISC) or reduced instruction set computer processor (RISC), an x86 instruction set compatible processor, a multi-core, or any other microprocessor or central processing unit (CPU). In various implementations, the processor 1010 may be a dual-core processor, a dual-core mobile processor, etc.

[0070] The memory 1012 may be implemented as a volatile memory device, such as, but not limited to, a random access memory (RAM), a dynamic random access memory (DRAM), or a static RAM (SRAM).

[0071] Storage 1014 may be implemented as a non-volatile storage device, such as, but not limited to, a magnetic disk drive, an optical disk drive, a tape drive, an internal storage device, an attached storage device, flash memory, battery-backed SDRAM (synchronous DRAM), and / or a network-accessible storage device. In various implementations, storage 1014 may include technology that increases storage performance enhancing protection for valuable digital media, for example when multiple hard drives are included.

[0072] The graphics subsystem 1015 may perform processing of images, such as still images or video for display. The graphics subsystem 1015 may be, for example, a graphics processing unit (GPU) or a visual processing unit (VPU). An analog or digital interface may be used to communicatively couple the graphics subsystem 1015 and the display 1020. For example, the interface may be any of High-Definition Multimedia Interface, DisplayPort, Wireless HDMI, and / or Wireless HD compliant technologies. The graphics subsystem 1015 may be integrated into the processor 1010 or the chipset 1005. In some implementations, the graphics subsystem 1015 may be a standalone device communicatively coupled to the chipset 1005.

[0073] The graphics and / or video processing techniques described herein may be implemented in a variety of hardware architectures. For example, graphics and / or video capabilities may be integrated within a chipset. Alternatively, discrete graphics and / or video processors may be used. In yet another implementation, graphics and / or video capabilities may be provided by a general-purpose processor, including a multi-core processor. In a further embodiment, the capabilities may be implemented in a consumer electronics product.

[0074] The wireless device 1018 may include one or more radios capable of transmitting and receiving signals using various suitable wireless communication technologies. Such technologies may include communication over one or more wireless networks. Examples of wireless networks include, but are not limited to, wireless local area networks (WLANs), wireless personal area networks (WPANs), wireless metropolitan area networks (WMANs), cellular networks, and satellite networks. When communicating over such networks, the wireless device 1018 may operate according to one or more applicable standards in any version.

[0075] In various implementations, the display 1020 may include any television-type monitor or display. The display 1020 may include, for example, a computer display screen, a touch screen display, a video monitor, a television-like device, and / or a television. The display 1020 may be digital and / or analog. In various implementations, the display 1020 may be a holographic display. The display 1020 may also be a transparent surface capable of receiving visual projections. Such projections may convey various forms of information, images, and / or objects. For example, such projections may be visual overlays for mobile augmented reality (MAR) applications. Under the control of one or more software applications 1016, the platform 1002 may display a user interface 1022 on the display 1020.

[0076] In various implementations, the content services devices 1030 may be hosted by any national, international, and / or independent service and thus accessible to the platform 1002 via, for example, the Internet. The content services devices 1030 may be coupled to the platform 1002 and / or the display 1020. The platform 1002 and / or the content services devices 1030 may be coupled to the network 1060 for communicating (e.g., sending and / or receiving) media information to or from the network 1060. The content delivery devices 1040 may also be coupled to the platform 1002 and / or the display 1020.

[0077] In various implementations, the content services device 1030 may include a cable television box, a personal computer, a network, a telephone, an Internet-enabled device or other device capable of delivering digital information and / or content, and other similar devices capable of communicating content unidirectionally or bidirectionally between a content provider and the platform 1002 and / or the display 1020, either over the network 1060 or directly. It will be understood that content may be communicated unidirectionally and / or bidirectionally to or from any one of the components and content providers in the system 1000 over the network 1060. Examples of content may include any media information including, for example, video, music, medical, and gaming information, etc.

[0078] The content services device 1030 can receive content, such as cable television programming, including media information, digital information, and / or other content. Examples of content providers can include any cable or satellite television, or radio, or Internet content provider. The examples provided are not intended to limit in any way implementations in accordance with the present disclosure.

[0079] In various embodiments, the platform 1002 can receive control signals from a navigation controller 1050 having one or more navigation functions. The navigation functions can be used, for example, to interact with the user interface 1022. In various embodiments, the navigation can be a pointing device, which can be a computer hardware component (particularly a human interface device) that allows a user to input spatial (e.g., continuous and multi-dimensional) data into a computer. Many systems, such as graphical user interfaces (GUIs) and televisions and monitors, allow a user to control a computer or television and provide data using physical gestures.

[0080] Movement of the navigation feature may be replicated on a display (e.g., display 1020) by movement of a pointer, cursor, focus ring, or other visual indicator displayed on the display. For example, navigation features placed on a navigation device under control of software application 1016 may be mapped to virtual navigation features displayed on user interface 1022, for example. In various embodiments, the navigation features may not be separate components, but may be integrated into platform 1002 and / or display 1020. However, this disclosure is not limited to the elements or in the context shown or described herein.

[0081] In various implementations, the drivers (not shown) may include technology that, when enabled, allows a user to instantly turn the platform 1002 on and off like a television, for example, with the touch of a button after initial boot-up. Program logic allows the platform 1002 to stream content to a media adapter or other content service apparatus 1030, or content delivery device 1040, even when the platform is turned "off." In addition, the chipset 1005 may include hardware and / or software support for, for example, 5.1 surround sound audio and / or high definition 7.1 surround sound audio. The drivers may include graphics drivers for an integrated graphics platform. In various embodiments, the graphics drivers may include a peripheral component interconnect (PCI) graphics card.

[0082] In various embodiments, any one or more of the components shown in system 1000 may be integrated. For example, platform 1002 and content services device 1030 may be integrated, or platform 1002 and content delivery device 1040 may be integrated, or platform 1002, content services device 1030, and content delivery device 1040 may be integrated. In various embodiments, platform 1002 and display 1020 may be an integrated unit. For example, display 1020 and content services device 1030 may be integrated, or display 1020 and content delivery device 1040 may be integrated. These examples are not intended to limit the disclosure.

[0083] In various embodiments, the system 1000 may be implemented as a wireless system, a wired system, or a combination of both. When implemented as a wireless system, the system 1000 may include components and interfaces suitable for communicating over a wireless shared media, such as one or more antennas, transmitters, receivers, transceivers, amplifiers, filters, control logic, etc. An example of a wireless shared media may include a portion of a wireless spectrum, such as the RF spectrum. When implemented as a wired system, the system 1000 may include components and interfaces suitable for communicating over a wired communication medium, such as input / output (I / O) adapters, physical connectors for connecting the I / O adapters to a corresponding wired communication medium, network interface cards (NICs), disk controllers, video controllers, audio controllers, etc. Examples of wired communication media may include wires, cables, metal leads, printed circuit boards (PCBs), backplanes, switch fabrics, semiconductor materials, twisted pair wires, coaxial cables, optical fibers, etc.

[0084] The platform 1002 can establish one or more logical or physical channels to communicate information. The information can include media information and control information. Media information can refer to any data that represents content intended for a user. Examples of content can include, for example, data from a voice conversation, video conferencing, streaming video, electronic mail (“email”) messages, voicemail messages, alphanumeric symbols, graphics, images, video, text, etc. Data from a voice conversation can be, for example, speech information, silent periods, background noise, comfort noise, tones, etc. Control information can refer to any data that represents commands, instructions, or control words intended for an automated system. For example, control information can be used to route media information through a system or to instruct a node to process the media information in a predetermined manner. The embodiments, however, are not limited to the elements or in the context shown or described in FIG. 6 .

[0085] As mentioned above, the system 1000 may be embodied in a variety of physical styles or form factors. FIG. 7 illustrates an exemplary small form factor device 1100 arranged in accordance with at least some implementations of the present disclosure. In some examples, the system 100 may be implemented via the device 1100. In other examples, the system 100 or portions thereof may be implemented via the device 1100. In various embodiments, for example, the device 1100 may be implemented as a mobile computing device having wireless capabilities. A mobile computing device may refer to any device having a processing system and a mobile power source or power source, such as, for example, one or more batteries.

[0086] Examples of mobile computing devices may include personal computers (PCs), laptop computers, ultra laptop computers, tablets, touchpads, portable computers, handheld computers, personal digital assistants (PDAs), mobile phones, mobile phone / PDA combinations, smart devices (e.g., smartphones, smart tablets, or smart mobile TVs), mobile Internet devices (MIDs), messaging devices, data communication devices, cameras, and the like.

[0087] Examples of mobile computing devices may also include computers configured to be worn by a person, such as wrist computers, finger computers, ring computers, eyeglass computers, belt clip computers, armband computers, shoe computers, clothing computers, and other wearable computers. In various embodiments, for example, a mobile computing device may be implemented as a smartphone capable of executing computer applications as well as voice and / or data communications. Some embodiments may be described using a mobile computing device implemented as a smartphone as an example, but it will be understood that other embodiments may be implemented using other wireless mobile computing devices as well. The embodiments are not limited in this context.

[0088] As shown in FIG. 7, device 1100 may include a housing having a front surface 1101 and a back surface 1102. Device 1100 includes a display 1104, input / output (I / O) devices 1106, and an integrated antenna 1108. Device 1100 may also include navigation capabilities 1112. I / O devices 1106 may include any suitable I / O devices for inputting information into a mobile computing device. Examples of I / O devices 1106 may include an alphanumeric keyboard, a numeric keypad, a touchpad, input keys, buttons, switches, a microphone, a speaker, a voice recognition device and software, and the like. Information may also be input into device 1100 via a microphone (not shown) or digitized by a voice recognition device. As shown, the device 1100 may include a camera 1105 (e.g., including a lens, aperture, and imaging sensor) and a flash 1110 integrated into the back 1102 (or elsewhere) of the device 1100. In other examples, the camera 1105 and flash 1110 may be integrated into the front 1101 of the device 1100, or both front and rear cameras may be provided. The camera 1105 and flash 1110 may be components of a camera module for generating image data that is processed into streaming video that is output to the display 1104 and / or communicated remotely from the device 1100 via, for example, an antenna 1108.

[0089] System 1000 and / or device 1100 may include one or more features or aspects of various embodiments described herein, including deriving a fit point for a sub-region of a coded picture in a coded video sequence in video data, grouping any combination of sub-pictures to form rectangular regions into a sub-picture set, deriving a level indicator corresponding to a sub-picture based on a level of the coded video sequence and the relative sizes of the coded picture and the sub-picture set, sequence parameter set syntax described herein, signaling described herein, sub-picture extraction and level derivation described herein, providing a VCC codec for utilizing sub-pictures and sets of sub-pictures for one or more of encoding and decoding video data, or any embodiment described herein.

[0090] Additional Notes and Examples Example 1 is an electronic system including a memory for storing video data, a processor coupled to the memory, and logic coupled to the processor and the memory, the logic deriving match points for sub-regions of coded images in a coded video sequence in the video data.

[0091] Example 2 includes the system of example 1, in which the logic further groups combinations of one or more sub-pictures that form rectangular regions in the video data into a sub-picture set.

[0092] Example 3 includes the system of example 2, wherein the logic is further configured to derive a level indicator corresponding to a sub-picture based on a level of the coded video sequence and a relative size of the coded picture and the sub-picture set.

[0093] Example 4 includes the system of any one of Examples 2 to 3, wherein the logic further comprises: One or more layers and level information for one or more subpictures and subpictures are signaled with messages in the video data.

[0094] Example 5 includes the system of any of examples 2-3, wherein the logic is further configured to signal sub-picture parameters in a parameter set to indicate a size and position of the sub-picture based on a grid.

[0095] Example 6 includes the system of Example 5, wherein the logic further provides an indication within a message in the video data that a level of any sub-picture or sub-picture set should be derived based on scaling the picture size to one of the sample sizes of the sub-picture and sub-picture set.

[0096] Example 7 includes the system of any of examples 1-6, wherein the logic further comprises: A versatile video coding (VVC) codec is provided for utilizing sub-pictures and sets of sub-pictures for one or more of encoding and decoding the video data.

[0097] Example 8 is a method for processing video data, comprising the steps of storing the video data and deriving matching points in the video data for sub-regions of coded images in a coded video sequence.

[0098] Example 9 includes the method of example 8, further including grouping a combination of one or more sub-pictures that form a rectangular region in the video data into a sub-picture set.

[0099] Example 10 includes the method of example 9, further including the step of deriving a level indicator corresponding to a sub-picture based on a level of the coded video sequence and a relative size of the coded picture and the sub-picture set.

[0100] Example 11 includes the method of any of examples 9-10, further including signaling one or more layer and level information for one or more subpictures and subpictures with a message in the video data.

[0101] Example 12 includes the method of any of Examples 9-10, further including the steps of signaling sub-picture parameters in a parameter set and indicating a size and a position of the sub-picture based on a grid.

[0102] Example 13 includes the method of Example 12, and further includes a step of providing an indication within a message in the video data that the level of any sub-picture or sub-picture set should be derived based on scaling the picture size to one of the sample sizes of the sub-picture and sub-picture set.

[0103] Example 14 is at least one non-transitory machine-readable medium including a plurality of instructions that, in response to being executed in a computing device, cause the computing device to store video data and derive matching points for subregions of coded images in a coded video sequence in the video data.

[0104] Example 15 includes at least one non-transitory machine-readable medium of example 14, further comprising, in response to being executed on a computing device, causing the computing device to group combinations of one or more sub-pictures that form rectangular regions in the video data into a sub-picture set. The instruction includes a number of commands that cause the

[0105] Example 16 includes at least one non-transitory machine-readable medium of Example 15, including a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further derive a level indicator corresponding to a sub-picture based on a level of the coded video sequence and a relative size of the coded image and the sub-picture set.

[0106] Example 17 includes at least one non-transitory machine-readable medium of any of Examples 15 to 16, including a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further signal one or more hierarchical and level information for one or more subpictures and subpictures together with a message in the video data.

[0107] Example 18 includes at least one non-transitory machine-readable medium of any of Examples 15 to 16, including a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further signal sub-picture parameters in a parameter set and indicate a size and position of the sub-picture based on a grid.

[0108] Example 19 includes at least one non-transitory machine-readable medium of Example 18, including a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further provide an indication within a message in the video data that a level of any sub-picture or sub-picture set should be derived based on scaling the picture size to one of the sample sizes of the sub-picture and sub-picture set.

[0109] Example 20 is an electronic device including one or more substrates and logic coupled to the one or more substrates, the logic deriving a match point for a sub-region of a coded image in a coded video sequence in the video data.

[0110] Example 21 includes the apparatus of example 20, in which the logic is further configured to group combinations of one or more sub-pictures that form rectangular regions in the video data into a sub-picture set.

[0111] Example 22 includes the apparatus of example 21, wherein the logic is further configured to derive a level indicator corresponding to a sub-picture based on a level of the coded video sequence and a relative size of the coded picture and the sub-picture set.

[0112] Example 23 includes the apparatus of any of examples 21-22, wherein the logic is further configured to signal one or more layer and level information for one or more subpictures and subpictures with the message in the video data.

[0113] Example 24 includes the apparatus of any of Examples 21-22, wherein the logic is further configured to signal sub-picture parameters in a parameter set and indicate a size and a position of the sub-picture based on a grid.

[0114] Example 25 includes the apparatus of Example 24, wherein the logic further provides an indication within a message in the video data that a level of any sub-picture or sub-picture set should be derived based on scaling the picture size to one of the sample sizes of the sub-picture and sub-picture set.

[0115] Example 26 includes the device of any of Examples 20 to 25, wherein the logic further provides a VVC codec to utilize sub-pictures and sets of sub-pictures for one or more of encoding and decoding the video data.

[0116] Example 27 is a video coding apparatus including means for storing video data and means for deriving matching points in the video data for sub-regions of coded images in a coded video sequence.

[0117] Example 28 includes the apparatus of example 27, further including means for grouping combinations of one or more sub-pictures that form a rectangular region in the video data into a sub-picture set.

[0118] Example 29 includes the apparatus of example 28, further including means for deriving a level indicator corresponding to a sub-picture based on a level of the coded video sequence and a relative size of the coded picture and the sub-picture set.

[0119] Example 30 includes the apparatus of any of Examples 28-29, further including means for signaling one or more layer and level information for one or more subpictures and subpictures with a message in the video data.

[0120] Example 31 includes the apparatus of any of examples 28-29, further including means for signaling sub-picture parameters in a parameter set, and means for indicating a size and a position of the sub-picture based on a grid.

[0121] Example 32 includes the apparatus of Example 31, and further includes means for providing an indication within a message in the video data that a level of any sub-picture or sub-picture set should be derived based on scaling the picture size to one of the sample sizes of the sub-picture and sub-picture set.

[0122] Example 33 includes the device of any of Examples 27 to 32, and further includes means for providing a VVC codec to utilize sub-pictures and sets of sub-pictures for one or more of encoding and decoding the video data.

[0123] Example 34 is an electronic system including a memory for storing video data, a processor coupled to the memory, and logic coupled to the processor and the memory, the logic deriving match points for sub-regions of coded images in a coded video sequence in the video data and grouping any combination of sub-pictures that form rectangular regions into a sub-picture set.

[0124] Example 35 includes the system of example 34, in which the logic is further configured to derive a level indicator corresponding to a sub-picture based on a level of the coded video sequence and a relative size of the coded picture and the sub-picture set.

[0125] Example 36 includes the system of any of examples 34-35, wherein the logic further implements one or more aspects of a sequence parameter set syntax described herein.

[0126] Example 37 includes the system of any of examples 34-36, wherein the logic further implements one or more aspects related to signaling described herein.

[0127] Example 38 includes the system of any of Examples 34-37, wherein the logic further performs one or more aspects related to sub-picture extraction and level derivation described herein.

[0128] Example 39 includes the system of any of Examples 34 to 38, wherein the logic further provides a VVC codec to utilize subpictures and sets of subpictures for one or more of encoding and decoding the video data.

[0129] Example 40 is a method for encoding / decoding video data, comprising the steps of storing video data, deriving matching points for sub-regions of coded images in a coded video sequence in the video data, and grouping any combination of sub-pictures forming a rectangular region into a sub-picture set.

[0130] Example 41 includes the method of example 40, further including the step of deriving a level indicator corresponding to a sub-picture based on a level of the coded video sequence and a relative size of the coded image and the sub-picture set.

[0131] Example 42 includes the method of any of Examples 40-41, and further includes implementing one or more aspects of the sequence parameter set syntax described herein.

[0132] Example 43 includes the method of any of Examples 40-42, further including performing one or more aspects of signaling described herein.

[0133] Example 44 includes the method of any of Examples 40-43, further including implementing one or more aspects of sub-picture extraction and level derivation described herein.

[0134] Example 45 includes any of the methods of Examples 40 to 44, and further includes providing a VVC codec to utilize sub-pictures and sets of sub-pictures for one or more of encoding and decoding the video data.

[0135] Example 46 is at least one non-transitory machine-readable medium including a plurality of instructions that, in response to being executed on a computing device, cause the computing device to derive matching points for sub-regions of coded images in a coded video sequence in video data, and group any combination of sub-pictures that form rectangular regions into a sub-picture set.

[0136] Example 47 includes the machine-readable medium of Example 46, and includes a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further derive a level indicator corresponding to a sub-picture based on a level of the coded video sequence and a relative size of the coded image and the sub-picture set.

[0137] Example 48 includes the machine-readable medium of any of Examples 46-47, including a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further implement one or more aspects relating to the sequence parameter set syntax described herein.

[0138] Example 49 includes the machine-readable medium of any of Examples 46-48, including a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further perform one or more aspects relating to signaling as described herein.

[0139] Example 50 includes the machine-readable medium of any of Examples 46-49, and includes a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further perform one or more aspects relating to sub-picture extraction and level derivation described herein.

[0140] Example 51 includes the machine-readable medium of any of Examples 46 to 50, and includes a plurality of instructions that, in response to being executed on a computing device, cause the computing device to further provide a VVC codec to utilize sub-pictures and sets of sub-pictures for one or more of encoding and decoding the video data.

[0141] Example 52 is an electronic device including one or more substrates and logic coupled to the one or more substrates, the logic receiving a bitstream of video data, identifying a sub-picture set to be decoded corresponding to a group of one or more sub-pictures, deriving a sub-picture set level based on a size of the identified sub-picture set, and determining whether the derived level is less than or equal to a threshold level decode.

[0142] Example 53 includes the apparatus of example 52, wherein the group of one or more sub-pictures forms a rectangular region in the video data.

[0143] Example 54 includes the device of any of Examples 52-53, wherein the logic is further configured to decode one or more layer and level information for the sub-picture based on a message in the video data.

[0144] Example 55 includes the device of any of Examples 52-53, wherein the logic is further configured to determine sub-picture parameters from a parameter set in the video data, and determine a size and position of the sub-picture based on a grid.

[0145] Example 56 includes the apparatus of Example 55, wherein the logic further determines, based on an instruction provided in a message in the video data, that a level of any sub-picture or sub-picture set should be derived based on scaling the picture size to one of the sample sizes of the sub-picture and sub-picture set.

[0146] Various embodiments may be implemented using a combination of hardware elements, software elements, or both. Examples of hardware elements may include a processor, a microprocessor, a circuit, a circuit element (e.g., a transistor, a resistor, a capacitor, an inductor, etc.), an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a digital signal processor (DSP), a field programmable gate array (FPGA), a logic gate, a register, a semiconductor device, a chip, a microchip, a chipset, etc. Examples of software may include a software component, a program, an application, a computer program, an application program, a system program, a machine program, an operating system software, a middleware, a firmware, a software module, a routine, a subroutine, a function, a method, a procedure, a software interface, an application program interface (API), an instruction set, a computational code, a computer code, a code segment, a word, a value, a symbol, or any combination thereof. Determining which embodiment to implement using hardware and / or software elements may vary according to any number of factors, such as desired computational speed, power levels, thermal tolerances, processing cycle budgets, input data rates, output data rates, memory resources, data bus speeds, and other design or performance constraints.

[0147] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium that represent various logic within a processor, which, when read by a machine, produces logic to cause the machine to perform the techniques described herein. Such representations, known as IP cores, are stored on tangible machine-readable media and supplied to various customers or manufacturing facilities for loading into manufacturing machines that actually create the logic or processor.

[0148] While certain features described herein have been described with reference to various embodiments, this description is not intended to be construed in a limiting sense. Various modifications of the embodiments described herein, as well as other embodiments, that are apparent to those skilled in the art to which this disclosure pertains, are deemed to be within the spirit and scope of the present disclosure.

[0149] It will be recognized that these embodiments are not limited to the above embodiments, but can be practiced with modification and alteration without departing from the scope of the appended claims. For example, the above embodiments may include certain combinations of features. However, the above embodiments are not limited in this respect, and in various embodiments, the above embodiments may include taking on only a subset of such features, taking on a different order of such features, taking on different combinations of such features, and / or taking on additional features other than those expressly recited. The scope of the embodiments should therefore be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. An apparatus comprising: a processor circuit for executing instructions; A sub-picture of a coded video sequence acquires a matching level indicator, the level indicator indicating a decoder interoperability, the sub-picture corresponding to a rectangular region of an image in the coded video sequence, and including a first supplemental enhancement information (SEI) message in the video bitstream, the first SEI message including the level indicator; a second SEI message in the video bitstream, the level indicator being estimated for a portion of the video bitstream corresponding to the second SEI message based on the first SEI message in response to a level indicator not present in the second SEI message; Processor circuitry, 13. An apparatus comprising:

2. The level indicator corresponds to a processing load of a decoder.

2. The apparatus of claim 1.

3. the processor circuit designates a number of the sub-pictures in an image of the coded video sequence; 3. Apparatus according to claim 1 or 2.

4. the processor circuit populates a field of the first SEI message with a value indicating the number of the sub-pictures in an image of the coded video sequence; 4. Apparatus according to any one of claims 1 to 3.

5. the processor circuit obtains the level indicator by determining the level indicator; 4. Apparatus according to any one of claims 1 to 3.

6. further comprising the instructions:

4. Apparatus according to any one of claims 1 to 3.

7. At least one memory; a processor circuit, The processor circuit includes: at least one of a central processing unit, a graphics processing unit, or a digital signal processor, the at least one of the central processing unit, the graphics processing unit, or the digital signal processor having control circuitry for controlling the movement of data within the processor circuitry, arithmetic and logic circuitry for performing one or more first operations corresponding to instructions, and one or more registers for storing results of the one or more first operations; a field programmable gate array (FPGA) including a first logic gate circuit, a plurality of configurable interconnects, and a storage circuit, the first logic gate circuit and the interconnects performing one or more second operations, and the storage circuit storing results of the one or more second operations; or an application specific integrated circuit including a second logic gate circuit for performing one or more third operations; One or more of the following may be included: The processor circuitry for performing at least one of the first operation, the second operation, or the third operation comprises: obtaining a first level indicator that a sub-picture of a coded video sequence matches, the sub-picture corresponding to a rectangular region of an image in the coded video sequence; and including a first supplemental enhancement information (SEI) message in the video bitstream, the first SEI message including the first level indicator; a second SEI message in the video bitstream, a second level indicator being estimated for a portion of the video bitstream corresponding to the second SEI message since the second level indicator is not present in the second SEI message, and the second level indicator being estimated based on the second level indicator in a third message of the video bitstream. Device.

8. the first level indicator corresponds to a processing load of a decoder; 8. The apparatus of claim 7.

9. the processor circuit designates a number of the sub-pictures in an image of the coded video sequence; 9. Apparatus according to claim 7 or 8.

10. the processor circuit populates a field of the first SEI message with a value indicating the number of the sub-pictures in an image of the coded video sequence; 10. Apparatus according to any one of claims 7 to 9.

11. the processor circuit obtains the first level indicator by determining the first level indicator; 10. Apparatus according to any one of claims 7 to 9.

12. A computer readable storage device or storage disk containing instructions for at least one processor, comprising: obtaining a matching level indicator for a sub-picture of a coded video sequence, the sub-picture corresponding to a rectangular region of an image in the coded video sequence; including a first supplemental enhancement information (SEI) message in the video bitstream, the first SEI message including the level indicator; including a second SEI message in the video bitstream, and a subsequent level indicator is estimated for a portion of the video bitstream responsive to the second SEI message when the subsequent level indicator is not present in the second SEI message; A computer readable storage device or storage disk that causes the

13. The level indicator corresponds to a processing load of a decoder.

13. A computer readable storage device or storage disk according to claim 12.

14. The instructions to the at least one processor specify a number of the sub-pictures in an image of the coded video sequence.

14. A computer readable storage device or storage disk according to claim 12 or 13, causing the computer to

15. The instructions cause the at least one processor to populate a field of the first SEI message with a value indicative of a number of the sub-pictures in an image of the coded video sequence.

15. A computer readable storage device or storage disk according to any one of claims 12 to 14, causing the computer to

16. The instructions cause the at least one processor to obtain the level indicator by determining the level indicator.

15. A computer readable storage device or storage disk according to any one of claims 12 to 14, causing the computer to

17. An apparatus comprising: At least one memory, instructions in the apparatus; and a processor circuit having control circuitry for controlling movement of data within said processor circuit, arithmetic and logic circuitry for performing one or more operations on said data, and one or more registers for storing results of said one or more operations; Including, The processor circuit includes: A sub-picture of a coded video sequence obtains a level of conformance, the sub-picture corresponding to a rectangular region of an image in the coded video sequence; and a first supplemental enhancement information (SEI) message in the video bitstream, the first SEI message including a level indicator indicating the level; a second SEI message in the video bitstream, the level indicator being estimated for a portion of the video bitstream corresponding to the second SEI message based on the first SEI message in response to a level indicator not present in the second SEI message; Device.

18. The level corresponds to a processing load of a decoder.

20. The apparatus of claim 17.

19. The processor circuit executes the instructions to specify a number of the sub-pictures in an image of the coded video sequence.

19. Apparatus according to claim 17 or 18.

20. the processor circuit executes the instructions to populate a field of the first SEI message with a value indicative of a number of the sub-pictures in an image of the coded video sequence.

20. Apparatus according to any one of claims 17 to 19.

21. the processor circuit executes the instructions to obtain the level by determining the level; 20. Apparatus according to any one of claims 17 to 19.