Method and apparatus for signaling CMAF switching sets in ISOBMFF

Abstract

Methods, devices, and systems for encoding or decoding media are provided. The process may include obtaining a track group entry box from an ISO Base Media File Format (ISOBMFF) track associated with video data. The process may further include obtaining a first track of one or more tracks associated with the video data in a CMAF switching set, the first track having a first alternate group parameter, and obtaining a second track associated with the video data, the second alternate group parameter. The second alternate group parameter and the first alternate group parameter may have the same value. The process may further include decoding the video data based on the first track or the second track in the CMAF switching set.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims priority based on U.S. Provisional Application No. 63 / 417,114, filed on October 18, 2022, and U.S. Application No. 18 / 487,596, filed on October 16, 2023, the disclosures of which are hereby incorporated by reference in their entirety.

[0002] This disclosure relates to a series of advanced video encoding techniques. More specifically, this disclosure relates to the encoding and decoding of switch sets, the attributes of switch sets, and related tracks in an ISO (International Organization for Standardization) - based media file format (ISOBMFF).

Background Art

[0003] ISOBMFF is a file format widely used for media content. The Common Media Application Format (CMAF) standard defines common media format tracks that can be grouped into switch sets. CMAF switch sets are used to convey media with alternative tracks. Alternative tracks represent the same content but have different attributes such as, for example, bitrate, resolution, frame rate, and other possible characteristics.

Summary of the Invention

Problems to be Solved by the Invention

[0004] While the CMAF standard uses ISOBMFF as its track format, it does not include any specific method for signaling the presence of CMAF switch sets. Previously, such signaling relied on out-of-band signaling, such as DASH (Dynamic Adaptive Streaming over HTTP) manifests. Therefore, it is necessary to utilize the CMAF standard to signal the presence of CMAF switch sets within an ISOBMFF file. [Means for solving the problem]

[0005] According to the embodiment, a method for encoding or decoding video data can be provided. The method includes the steps of: obtaining a track group entry box from an ISO-based media file format (ISOBMFF) track associated with video data, wherein the track group entry box represents one or more tracks associated with the video data in a Common Media Application Format (CMAF) switch set, the CMAF switch set represents one or more tracks containing alternative representations of the video data, and the track group entry box includes one or more CMAF switch set characteristics associated with the video data; and obtaining a first track from one or more tracks associated with the video data in the CMAF switch set. A step of obtaining a video data, comprising: a first track being one or more tracks in a CMAF switching set, the first track including a first alternate group parameter for indicating the CMAF switching set to which the first track belongs; a step of obtaining a second track associated with video data, the second track including a second alternate group parameter, the second track being one or more tracks in a CMAF switching set, and the second alternate group parameter and the first alternate group parameter having the same value; and a step of decoding the video data based on the first or second track in the CMAF switching set.

[0006] According to the embodiment, an apparatus for encoding or decoding video data can be provided. The apparatus includes at least one memory arranged to store program code, and at least one processor arranged to read the program code and operate according to the instructions of the program code. The program code includes a first acquisition code arranged to cause at least one processor to acquire a track group entry box from an ISO-based media file format (ISOBMFF) track associated with video data, wherein the track group entry box indicates one or more tracks associated with the video data in a Common Media Application Format (CMAF) switch set, the CMAF switch set indicates one or more tracks containing alternative representations of the video data, and the track group entry box includes one or more CMAF switch set characteristics associated with the video data, and the first acquisition code arranged to cause at least one processor to acquire a first track from one or more tracks associated with the video data in the CMAF switch set. The system includes: a second acquisition code wherein the first track is one or more tracks in a CMAF switching set, and the first track includes a first alternate group parameter for indicating the CMAF switching set to which the first track belongs; a third acquisition code configured to cause at least one processor to acquire a second track associated with video data, wherein the second track includes a second alternate group parameter, and the second track is one or more tracks in a CMAF switching set, and the second alternate group parameter and the first alternate group parameter have the same value; and a first decoding code configured to cause at least one processor to decode video data based on the first or second track in a CMAF switching set.

[0007] According to the embodiment, a non-temporary computer-readable medium storing computer instructions can be provided. The instructions include one or more instructions, which, when executed by one or more processors of a device for encoding or decoding video data, include the steps of: obtaining a track group entry box from an ISO-based media file format (ISOBMFF) track associated with video data; the track group entry box indicates one or more tracks associated with video data in a Common Media Application Format (CMAF) switching set; the CMAF switching set indicates one or more tracks containing alternative representations of video data; and the track group entry box includes one or more CMAF switching set characteristics associated with video data. The process involves: obtaining a first track from one or more tracks associated with video data in a CMAF switch set, wherein the first track is one or more tracks in the CMAF switch set, and the first track includes a first alternate group parameter indicating the CMAF switch set to which the first track belongs; obtaining a second track associated with the video data, wherein the second track includes a second alternate group parameter, the second track is one or more tracks in the CMAF switch set, and the second alternate group parameter and the first alternate group parameter have the same value; and decoding the video data based on the first or second track in the CMAF switch set.

[0008] Further features, properties, and various advantages of the disclosed subject matter will become clearer from the following detailed description and accompanying drawings. In the drawings, [Brief explanation of the drawing]

[0009] [Figure 1]This is a schematic diagram of a simplified block diagram of a communication system according to the embodiment of this disclosure. [Figure 2] This is a schematic diagram of a simplified block diagram of a streaming system according to an embodiment of the present disclosure. [Figure 3] This is a simplified diagram of a CMAF track and CMAF switching set according to an embodiment of the present disclosure. [Figure 4] This is an illustrative diagram of the configuration of the CMAF switching set in ISOBMFF according to the embodiment of this disclosure. [Figure 5] This is an illustrative diagram of the configuration of the CMAF switching set in ISOBMFF according to the embodiment of this disclosure. [Figure 6] This is an exemplary flowchart of the process for signaling a CMAF switching set according to the embodiment of this disclosure. [Figure 7] This is a diagram of a computer system suitable for realizing an embodiment. [Modes for carrying out the invention]

[0010] One aspect of this disclosure provides a method, system, and non-temporary storage medium for parallel processing of dynamic mesh compression. Embodiments of this disclosure can also be applied to static meshes.

[0011] An embodiment of the present disclosure for realizing the encoding and decoding configurations of the present disclosure will be described with reference to Figures 1 and 2.

[0012] Figure 1 shows a simplified block diagram of a communication system 100 according to an embodiment of the present disclosure. The system 100 may include at least two terminals 110, 120 connected to each other via a network 150. In the case of one-way data transmission, the first terminal 110 can encode video data, which may include mesh data at its local location, and transmit it to the other terminal 120 via the network 150. The second terminal 120 can receive the encoded video data from the other terminal via the network 150, decode the encoded data, and display the restored video data. One-way data transmission is common in media service applications and the like.

[0013] Figure 1 shows a second set of terminals 130, 140, which are provided to support the bidirectional transmission of encoded video that may occur, for example, during a video conference. In the case of bidirectional data transmission, each terminal 130, 140 can encode video data captured at its local location and transmit it to another terminal over the network 150. Each terminal 130, 140 may receive encoded video data transmitted by another terminal, decode the encoded data, and display the restored video data on a local display device.

[0014] In Figure 1, terminals 110 to 140 may be, for example, servers, personal computers, and smartphones, and / or any other type of terminal. For example, terminals (110 to 140) may be laptop computers, tablet computers, media players, and / or dedicated video conferencing devices. Network 150 represents any number of networks that transmit encoded video data between terminals 110 to 140, including, for example, wired communication networks and / or wireless communication networks. Communication network 150 can exchange data over circuit-switched channels and / or packet-switched channels. Typical networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, the architecture and topology of network 150 may not be important to the operation of this disclosure unless described below.

[0015] As an example of the application of the disclosed subject matter, Figure 2 shows the arrangement of a video encoder and decoder in a streaming environment. The disclosed subject matter can be used in conjunction with other applications that support video, such as video conferencing and digital TV that store compressed video on digital media such as CDs (Compact Discs), DVDs (Digital Video Disks), and Memory Sticks.

[0016] As shown in Figure 2, the streaming system 200 includes a capture subsystem 213 which includes a video source 201 and an encoder 203. The streaming system 200 may further include at least one streaming server 205 and / or at least one streaming client 206.

[0017] The video source 201 can create a stream 202 containing, for example, a 3D (Three Dimensional, 3D) mesh and metadata associated with the 3D mesh. The video source 201 may include, for example, a 3D sensor (e.g., a depth sensor) or 3D imaging technology (e.g., a digital camera), and a computing device arranged to generate a 3D mesh using data received from the 3D sensor or 3D imaging technology. A sample stream 202, having a higher data volume compared to the encoded video bitstream, can be processed by an encoder 203 coupled to the video source 20. The encoder 203 may include hardware, software, or a combination thereof, to realize or implement each aspect of the disclosed subject matter, as will be described in detail below. The encoder 203 can generate an encoded video bitstream 204. The encoded video bitstream 204, having a lower data volume compared to the uncompressed stream 202, can be stored in the streaming server 205 for future use. One or more streaming clients 206 can access the streaming server 205 to retrieve a video bitstream 209, which may be a copy of the encoded video bitstream 204.

[0018] The streaming client 206 may include a video decoder 210 and a display 212. The video decoder 210 can, for example, decode a video bitstream 209 which is an input copy of an encoded video bitstream 204, and generate an output video sample stream 211 that can be rendered on a display 212 or another rendering device (not shown). In some streaming systems, the video bitstreams 204 and 209 can be encoded according to some video encoding / compression standards.

[0019] Embodiments of this disclosure relate to signaling the presence of a CMAF switch set in an ISOBMFF file using the CMAF standard.

[0020] FIG. 3 is an exemplary diagram 300 of a CMAF track and a CMAF switching set.

[0021] According to an embodiment, the CMAF switching set may be a set of CMAF tracks. Each of the CMAF tracks is an alternative encoding of the same source content that is constrained to enable seamless track switching and decoding. The CMAF selection set may be a set of CMAF switching sets, and each switching set encodes alternative renditions of the same rendering at the same time period so that only one of, for example, different languages or codecs at a time is played.

[0022] According to an embodiment, a CMAF track is a sequence of relevant CMAF headers (CMAF head) sufficient to initialize playback and CMAF fragments in presentation order of the same media stream, and the CMAF fragments are ISOBMFFs that conform to CMAF encoding and packaging constraints.

[0023] According to an embodiment, the CMAF header is a sequence of ISOBMFF boxes starting with a file type box and including a movie box, and includes initialization information regarding the CMAF track. A CMAF segment is a resource including one or more consecutive CMAF fragments from the same CMAF track. The CMAF chunk resource includes a single ISOBMFF segment, and the single ISOBMFF segment includes a consecutive subset in the order of CMAF fragment samples. A CMAF track file is a CMAF track stored in a single ISOBMFF file, and the CMAF track file includes all CMAF segments arranged in order from the CMAF header and the time of zero decoding time.

[0024] As shown in Figure 3, a CMAF switch is a collection of CMAF tracks that share several common constraints. The main purpose of a CMAF switch set is to provide alternative representations of the same content across multiple tracks, allowing the player to switch tracks to adapt to changes in network bandwidth and other changing attributes during streaming or playback.

[0025] As mentioned above, the CMAF standard uses ISOBMFF as the track format, but the CMAF standard does not provide a way to signal the presence of CMAF switch sets within an ISOBMFF file.

[0026] Embodiments of this disclosure relate to an alternate_group flag in an ISOBMFF tkhd box (also known as the “track header box”) for signaling a track set belonging to a CMAF switching set. This flag can be used to signal an alternative representation, but it does not indicate whether or not the set is a CMAF switching set, or its attributes.

[0027] Embodiments of this disclosure limit a new track group box type for CMAF switching sets. A track group entry box may be a special type of track box that limits the track requirements of a common media format. In embodiments, such a track group entry box may be identified by a unique four-character code, for example, "cmsg" (just an example), and the track group box may include: 1. track_group_id may have a value equal to the alternate_group of the CMAF track in the switching set; 2. num_tracks: Specifies the number of tracks in the switch set; 3. `aligned`: Specifies whether the tracks are aligned in time; 4. single_init: You can specify whether a track has a single initialization; 5. principal_id can indicate the track ID (identification, ID) including the principal header of the switch set; 6. structural_brand can indicate the CMAF structure type to which all tracks in a particular switching set conform; or 7. mediaprofile_brand can indicate the CMAF file type of the media profile that all tracks in the switching set conform to.

[0028] In this embodiment, each track that is part of a CMAF switching set may have an alternate_group that is set to be equal to the value of track_group_id in the CMAF track group entry box.

[0029] In some embodiments, CMAF switching set characteristics can be signaled using a track group box, similar to the pre-selection method. In such embodiments, the CMAF switching set track group entry box (e.g., "cmsg") which may contain the parameters mentioned above, may be limited.

[0030] Figure 4 shows an exemplary CMAF switching set with two tracks, Figure 400.

[0031] As shown in Figure 400, arrow 445 indicates the association of tracks and track group boxes (also called "track group entry boxes") in a CMAF switching set using alternate_group and track_group_id. For example, tracks 410, 420, and 430 are associated with each other. The value of track_group_id for ISOBMFF track 410 is equal to the value of alternate_group for tracks 420 and 430. The fact that the value of track_group_id for ISOBMFF track 410 is equal to the value of alternate_group for tracks 420 and 430 indicates that tracks 420 and 430 are associated with each other and are alternate representations of the same media content but have different parameters. When decoding media data, the decoder can use either track 420 or track 430 based on network, user equipment, or decoder parameters.

[0032] Figure 5 shows an exemplary CMAF switching set with two tracks and an exemplary Figure 500 of a pre-selection set with two tracks.

[0033] As shown in Figure 500, arrows 555 and 565 indicate the association between the CMAF switching set and the CMAF auxiliary selection set and the ISOBMFF track 510. As shown in Figure 500, the CMAF switching set and the auxiliary selection set can be used together.

[0034] As shown in Figure 500, tracks 530 and 540 are in one CMAF switching set, and tracks 520 and 530 are in one auxiliary selection set. According to the embodiment, since tracks 530 and 540 are in one CMAF switching set, track 540 can be used in the same auxiliary selection set as track 520, rather than track 530.

[0035] Box type: "cmsg"

[0036] Container: Truck Group Description Box

[0037] Compulsory: No

[0038] Number: Zero or greater

[0039] A CMAF transition group defines a set of tracks that conform to a CMAF transition set in ISO / IEC (International Organization for Standardization / International Electrotechnical Commission, ISO / IEC) 23000-19. A CMAF transition set is limited by its attributes.

[0040] A CMAF switching group entry box can only describe track groups identified by a track_group_type equal to "cmsg". All CMAF switching groups having at least one contributing track with the track_in_movie flag set to 1 can be limited by a CMAF switching group entry box. Otherwise, the presence of a CMAF switching group entry box is optional. All attributes that uniquely limit a given pre-selection must be present in that pre-selected CMAF switching group entry box.

[0041] grammar

number

[0042] Semantics

[0043] The following is an example of the information that may be included in a box for a CMAF switching group.

[0044] num_tracks can specify the number of alternate tracks grouped by the CMAF switching group or set. Tracks grouped by the CMAF switching track group may have an alternate_group equal to the ID of the switching group. Aligned, when set to 1, can indicate an aligned CMAF switching set. Otherwise, it can be set to 0, with other values ​​reserved. Single_init, when set to 1, can indicate that the CMAF switching set has a single initialization segment for all tracks. Otherwise, it can be set to 0, with other values ​​reserved. Principal can specify the track_id of a track, and the initialization segment of that track can be used to initialize any track in the switching group. Structural_brand can specify the CMAF structure type for all tracks in the switching group. Mediaprofile_brand can specify the media profile type to which all tracks in the switching group conform.

[0045] Embodiments of this disclosure use a track group box to provide signaling for a CMAF switch set in a manner that does not interfere with the grouping of other tracks. Thus, the association between a CMAF switch set and other track groups can be easily signaled.

[0046] According to the embodiment, a method can be provided for signaling the presence, track associations, and attributes of one or more CMAF switching sets within an ISOBMFF file. Each track may have an alternative_group for signaling its association, and at the same time, a new type of track group entry box can be defined for a CMAF switching set such that the track_group_id of the track group entry box is the same as the alternative_group of the associated track. In addition, attributes of a CMAF switching set, such as alignment, initialization segment attributes, CMAF structure type, and CMAF media profile type, may also be signaled in the same track group entry, and such signaling can coexist with other track group signaling, such as pre-selection signaling, so that signaling CMAF switching sets and pre-selection signaling can be combined.

[0047] Figure 6 shows an exemplary process 600 for signaling a CMAF switching set according to an embodiment of the present disclosure.

[0048] As shown in process 600, in operation 605, the track group entry box can be obtained from the ISO-based media file format (ISOBMFF) track associated with the video data.

[0049] In embodiments, the track group entry box indicates one or more tracks associated with video data in a Common Media Application Format (CMAF) switch set. In embodiments, the CMAF switch set may indicate one or more tracks containing alternative representations of video data. In embodiments, the track group entry box includes one or more CMAF switch set characteristics associated with video data.

[0050] One or more CMAF switching set characteristics in a track group entry box may include a first track group ID parameter indicating the ID of the track group entry box, which is used to determine one or more tracks in the CMAF switching set. In embodiments, the value of the first track group ID parameter may be the same as the value of the corresponding alternate group parameter for one or more tracks in the CMAF switching set.

[0051] The track group entry box may include a track count parameter, which indicates the total number of tracks in the CMAF switch set. The track group entry box may include a principal ID, which indicates the track ID containing the principal header of the CMAF switch set. The track group entry box may include an alignment parameter, which indicates whether one or more tracks in the CMAF switch set are time-aligned. The track group entry box may include a structure band parameter, which indicates the CMAF structure band to which one or more tracks in the CMAF switch set belong. The track group entry box may include a media profile band parameter, which indicates the CMAF file band of the media profile to which one or more tracks in the CMAF switch set belong.

[0052] In this embodiment, each of the one or more tracks in the CMAF switching set may have a corresponding alternate group parameter, which is equal to the first track group id parameter of the CMAF switching set to which the one or more tracks belong.

[0053] Operation 610 allows for the acquisition of a first track from among one or more tracks associated with video data in a CMAF switching set. In some embodiments, a first alternate group parameter may be acquired, which indicates the CMAF switching set to which the first track belongs. The first track may be one or more tracks in the CMAF switching set.

[0054] In operation 615, the second track of one or more tracks associated with the video data in the CMAF switching set can be obtained. In some embodiments, a second alternate group parameter may be obtained. In some embodiments, the second alternate group parameter and the first alternate group parameter have the same value, indicating that the second track and the first track are one or more tracks in the CMAF switching set.

[0055] In the embodiment, a third track associated with video data can be obtained. In the embodiment, the second and third tracks are tracks in a set of pre-selected tracks. The third track (and the second track) may include a pre-selection box, which indicates that the third track is one of a plurality of pre-selected tracks.

[0056] In operation 620, video data can be decoded based on a first or second track in a CMAF switching set. In embodiments, video data is decoded based on multiple pre-selected tracks and a CMAF switching set. Decoding video data based on multiple pre-selected tracks and a CMAF switching set may include decoding based on a first and a third track, or a second and a third track. In embodiments, each of the pre-selected tracks may include a pre-selected track box, and each of the pre-selected tracks may have the same value for the second track group id parameter.

[0057] The techniques described above can be implemented as computer software using computer-readable instructions and can be stored physically on one or more computer-readable media. For example, Figure 7 shows a computer system 700 suitable for implementing some embodiments of the present disclosure.

[0058] Computer software can be encoded using any suitable machine code or computer language, which can be directly executed by a computer's central processing unit (CPU), graphics processing unit (GPU), etc., or interpreted, to create code containing instructions that are executed via microcode, through assembly, compilation, linking, or similar mechanisms.

[0059] This directive can be executed on various types of computers or their components (including, for example, personal computers, tablet computers, servers, smartphones, game consoles, and Internet of Things devices).

[0060] The components of the computer system 700 shown in Figure 7 are illustrative and are not intended to imply any limitations on the scope or functionality of the computer software used to implement embodiments of this disclosure. The arrangement of the components should not be construed as having any dependencies or requirements on any one or combination of components shown in any non-limiting embodiment of the computer system 700.

[0061] The computer system 700 may include several human-machine interface input devices. Such human-machine interface input devices can respond to input from one or more human users, for example, tactile input (e.g., keystrokes, swipes, data glove movements), audio input (e.g., voice, tapping), visual input (e.g., gestures), and olfactory input (not shown). The human-machine interface devices may also be used to capture several media that are not necessarily directly related to conscious human input, such as audio (e.g., voices, music, ambient sounds), images (e.g., scanned images, photographic images acquired from still image cameras), and videos (e.g., two-dimensional videos including stereoscopic videos, three-dimensional videos).

[0062] The input human-machine interface device may include one or more of the following: keyboard 701, mouse 702, touchpad 703, touchscreen 710, data glove, joystick 705, microphone 706, scanner 707, and imaging device 708 (only one of each is shown).

[0063] The computer system 700 may further include several human-machine interface output devices. Such human-machine interface output devices can stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such human-machine interface output devices include tactile output devices (e.g., touchscreen 710, tactile feedback via data glove or joystick 705, but there may also be tactile feedback devices that are not used as input devices). For example, such devices may include audio output devices (e.g., speaker 709, head-mounted headphones (not shown)), visual output devices (e.g., screen 710 including CRT screens, LCD screens, plasma screens, OLED screens, etc., with or without touchscreen input capability, with or without tactile feedback capability, some of which can output two-dimensional visual output or three-dimensional or more output in a stereoscopic output manner, for example, virtual reality glasses (not shown), holographic displays and smoke tanks (not shown)), and printers (not shown).

[0064] The computer system 700 may further include human-accessible storage devices and associated media, such as optical media including a CD / DVD ROM / RW 720 having a CD / DVD media 721, a thumb drive 722, a removable hard drive or solid-state drive 723, traditional magnetic media such as magnetic tape or floppy disks (not shown), and devices such as security dongles (not shown) using dedicated ROM / ASIC / PLDs.

[0065] Those skilled in the art should understand that the term “computer-readable medium” as used in combination with the currently disclosed subject matter does not include a transmission medium, carrier wave, or other transient signal.

[0066] The computer system 700 may include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, or optical networks. The networks may be local, wide-area, metropolitan, vehicle, industrial, real-time, or latency-tolerant networks. Examples of networks include, for example, local networks such as Ethernet, cellular networks including wireless LAN (Local Area Network, LAN), GSM (Global System for Mobile Communications, GSM), 3G (the Third Generation, 3G), 4G (the Fourth Generation, 4G), 5G (the Fifth Generation, 5G), and LTE (Long Term Evolution, LTE), wide-area digital networks including cable TV, satellite TV, and terrestrial TV, and vehicle and industrial networks including CANBus. Some networks typically require an external network interface adapter connected to a general-purpose data port or peripheral bus 749 (e.g., a USB port on computer system 700), while others are integrated into the core of computer system 70 by connecting to the system bus via the network infrastructure described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, computer system 700 can communicate with other entities. Such communication may be one-way reception only (e.g., broadcast television), one-way transmission only (e.g., a CAN bus to some CAN (Controller Area Network, CAN) bus devices), or bidirectional to other computer systems via a local area or wide area digital network, for example. Such communication may include communication to a cloud computing environment 755.Certain protocols and protocol stacks can be used on each of these networks and network interfaces mentioned above.

[0067] The aforementioned human-machine interface device, human-accessible storage device, and network interface 754 can be attached to the core 740 of the computer system 700.

[0068] The core 740 may include one or more central processing units (CPUs) 741, graphics processing units (GPUs) 742, dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) 743, and hardware accelerators 744 for certain specific tasks. These devices are connected via a system bus 748, along with read-only memory (ROM) 745, random access memory 746, and internal mass storage devices 747 such as internal hard disk drives or SSDs that are not accessible to the user. In some computer systems, expansion with additional CPUs, GPUs, etc., can be enabled by accessing the system bus 748 in the form of one or more physical plugs. Peripheral devices are connected to the core's system bus 748 directly or via a peripheral bus 749. Peripheral bus architectures include PCI (Peripheral Component Interconnect / Interface) and USB (Universal Serial Bus). A graphics adapter 750 may be included in the core 740.

[0069] The CPU 741, GPU 742, FPGA 743, and accelerator 744 can execute certain commands, and by combining these commands, the above-mentioned computer code can be constructed. This computer code can be stored in ROM 745 and RAM (Random Access Memory, RAM) 746. Transient data can also be stored in RAM 746, and persistent data can be stored, for example, in the internal mass storage device 747. High-speed storage and retrieval to any memory device can be achieved by using cache memory, and this cache memory is closely related to one or more CPUs 741, GPUs 742, mass storage devices 747, ROMs 745, RAM 746, etc.

[0070] A computer-readable medium may have computer code thereon for performing various operations that a computer can perform. The medium and computer code may be specially designed and constructed for the purposes of this disclosure, or they may be of a type known and available to those skilled in the computer software field.

[0071] For example, but not limited to, a computer system having the architecture of computer system 700, in particular core 740, can provide functionality by having a processor (including a CPU, GPU, FPGA, accelerator, etc.) execute software embodied in one or more tangible computer-readable media. Such computer-readable media may be media relating to user-accessible mass storage devices, as described above, and media relating to several storage devices having a non-transient nature in core 740 (e.g., internal mass storage device 747 or ROM 745). Software of various embodiments for realizing this disclosure can be stored in such devices and executed by the core. Depending on the specific requirements, the computer-readable media may include one or more storage devices or chips. The software can cause the core, in particular the processor (including a CPU, GPU, FPGA, etc.) therein, to execute certain processes or specific parts of certain processes, including defining data structures stored in RAM 746 as described herein, and modifying such data structures according to processes defined by the software. In addition, or alternatively, a computer system may provide functionality as a result of logic being embodied in circuitry (e.g., accelerator 744) in hardwired or otherwise, such circuitry may operate in place of or with software to perform specific processes or specific parts of specific processes described herein. Where appropriate, references to software may include logic, and references to logic may include software. Where appropriate, references to computer-readable media may include circuitry that stores software for execution (e.g., integrated circuit, IC), circuitry that implements logic for execution, or both. This disclosure includes any appropriate combination of hardware and software.

[0072] While this disclosure has already described several non-limiting embodiments, there are various modifications, substitutions, and alternative equivalents that fall within the scope of this disclosure. Accordingly, various systems and methods that embody the principles of this disclosure but are not expressly shown or described herein, and that fall within the spirit and scope of this disclosure, can be devised by those skilled in the art. [Explanation of Symbols]

[0073] 20 Video Sources 70 Computer Systems 100 Communication Systems 100 Systems 110 to 140 devices 150 Communication Networks 200 Streaming Systems 201 Video Sources 202 Sample Streams 203 Encoder 204 Encoded video bitstream 205 Streaming Servers 206 Streaming Clients 209 video bitstreams 210 Video Decoders 211 Output video sample stream 212 displays 213 Capture Subsystem 410 trucks 410 ISOBMFF Truck 420 trucks 430 trucks 510 ISOBMFF Truck 520 trucks 530 trucks 540 trucks 700 Computer Systems 701 Keyboard 702 Mouse 703 Touchpad 705 Joystick 706 Mike 707 Scanner 708 Imaging device 709 Speaker 710 Touchscreen 710 screens 720 RW 721 Medium 722 Thumb Drive 723 Solid State Drive 740 cores 741 Central Processing Unit (CPU) 742 Graphics Processing Units (GPUs) 743 FPGA 744 Hardware Accelerators 745 Dedicated memory (Read-Only memory, ROM) 746 RAM (Random Access Memory, RAM) 747 Mass Storage Device 748 System Bus 749 Local Buses 750 Graphics Adapter 754 Network Interfaces 755 Cloud Computing Environment

Claims

1. A method for decrypting media, which is performed by at least one processor, The aforementioned method, A step of obtaining a track group entry box from an ISO-based media file format (ISOBMFF) track associated with video data, wherein the track group entry box represents one or more tracks associated with the video data in a Common Media Application Format (CMAF) switch set, the CMAF switch set represents one or more tracks containing alternative representations of the video data, and the track group entry box includes one or more CMAF switch set characteristics associated with the video data. A step of obtaining a first track from among the one or more tracks associated with the video data in the CMAF switching set, wherein the first track includes a first alternate group parameter for indicating which CMAF switching set the first track belongs to. A step of acquiring a second track from among the one or more tracks associated with the video data in the CMAF switching set, wherein the second track includes a second alternate group parameter, and the second alternate group parameter and the first alternate group parameter have the same value. The steps include decoding the video data based on the first track or the second track in the CMAF switching set, A method that includes this.

2. The one or more CMAF switching set characteristics in the track group entry box are: A first track group ID parameter indicating the ID of the track group entry box, wherein the first track group ID parameter is used to determine the one or more tracks in the CMAF switching set, and the value of the first track group ID parameter is the same as the value of the corresponding alternate group parameter for the one or more tracks in the CMAF switching set, A track count parameter indicating the total number of tracks in the CMAF switching set, A principal ID indicating the track ID including the principal header of the CMAF switching set, The method according to claim 1, including the following:

3. The one or more CMAF switching set characteristics in the track group entry box are further: Alignment parameters indicating whether one or more tracks in the CMAF switching set are time-aligned, A structure type parameter indicating the CMAF structure type to which one or more tracks in the CMAF switching set belong, A media profile type parameter indicating the CMAF file type of the media profile to which one or more tracks in the CMAF switching set belong, The method according to claim 2, including the method described in claim 2.

4. Each of the one or more tracks in the CMAF switching set has a corresponding alternate group parameter, the alternate group parameter being equal to the first track group id parameter of the CMAF switching set to which the one or more tracks belong. The method according to claim 1.

5. The second track includes a preliminary selection box, and the method is, A step of obtaining a third track associated with the video data, wherein the third track is a track in a set of pre-selected tracks, and the third track includes a pre-selection box indicating that the third track is one of a plurality of pre-selected tracks; The steps include decoding the video data based on the plurality of pre-selected tracks and the CMAF switching set, The method according to claim 1, further comprising:

6. The step of decoding the video data based on the plurality of pre-selected tracks and the CMAF switching set is: A step of decoding based on the first track and the third track, or the second track and the third track, The method according to claim 5, including the method described in claim 5.

7. Each of the plurality of pre-selected tracks includes a pre-selected track box, and each of the plurality of pre-selected tracks has the same value as the second track group id parameter. The method according to claim 6.

8. A device for decoding media, At least one memory in which computer code is stored, At least one processor configured to access the computer code and operate according to the instructions of the computer code, Includes, The aforementioned computer code is A first acquisition code, configured to cause at least one processor to acquire a track group entry box from an ISO-based media file format (ISOBMFF) track associated with video data, wherein the track group entry box indicates one or more tracks associated with the video data in a Common Media Application Format (CMAF) switch set, the CMAF switch set indicates one or more tracks containing alternative representations of the video data, and the track group entry box includes one or more CMAF switch set characteristics associated with the video data, A second acquisition code is configured to cause at least one processor to acquire a first track of the one or more tracks associated with the video data in the CMAF switching set, wherein the first track includes a second acquisition code that includes a first alternate group parameter indicating which CMAF switching set the first track belongs to. A third acquisition code configured to cause at least one processor to acquire a second track of the one or more tracks associated with the video data in the CMAF switching set, wherein the second track includes a second alternate group parameter, and the third acquisition code has the same value as the second alternate group parameter and the first alternate group parameter. A first decoding code is provided which at least one processor is configured to decode the video data based on the first track or the second track in the CMAF switching set, A device that includes this.

9. The one or more CMAF switching set characteristics in the track group entry box are: A first track group ID parameter indicating the ID of the track group entry box, wherein the first track group ID parameter is used to determine the one or more tracks in the CMAF switching set, and the value of the first track group ID parameter is the same as the value of the corresponding alternate group parameter for the one or more tracks in the CMAF switching set. A track count parameter indicating the total number of tracks in the CMAF switching set, A principal ID indicating the track ID including the principal header of the CMAF switching set, The apparatus according to claim 8, including the following:

10. The one or more CMAF switching set characteristics in the track group entry box are further: Alignment parameters indicating whether one or more tracks in the CMAF switching set are time-aligned, A structure type parameter indicating the CMAF structure type to which one or more tracks in the CMAF switching set belong, A media profile type parameter indicating the CMAF file type of the media profile to which one or more tracks in the CMAF switching set belong, The apparatus according to claim 9, including the following:

11. Each of the one or more tracks in the CMAF switching set has a corresponding alternate group parameter, the alternate group parameter being equal to the first track group id parameter of the CMAF switching set to which the one or more tracks belong. The apparatus according to claim 8.

12. The second track includes a preliminary selection box, and the computer code further includes: The code includes a fourth acquisition code configured to cause at least one processor to acquire a third track associated with the video data, The third track is a track in the set of spare selected tracks, The third track includes the pre-selection box indicating that the third track is one of a plurality of pre-selection tracks, The first decoding code includes decoding the video data based on the plurality of pre-selected tracks and the CMAF switching set, The apparatus according to claim 9.

13. The first decoding code includes decoding the video data based on the first track and the third track, or the second track and the third track. The apparatus according to claim 12.

14. Each of the plurality of pre-selected tracks includes a pre-selected track box, and each of the plurality of pre-selected tracks has the same value as the second track group id parameter. The apparatus according to claim 13.

15. A computer executable program containing instructions, When the instruction is executed by at least one processor, the at least one processor is instructed to: A program that causes the method described in any one of claims 1 to 7 to be performed.

Images