Method, apparatus and computer program for performing step-based processing in media processing on a cloud platform

Abstract

A method and apparatus for performing step-based processing in a network-based media processing (NBMP) workflow are provided, the method including the steps of obtaining a workflow including an input media stream having a plurality of segments, identifying boundaries between the plurality of segments in the input media stream based on one of metadata-based segment boundary signaling or code-based segment boundary signaling, and processing the plurality of segments based on the identified boundaries between the plurality of segments.

Description

[Technical field]

[0001] This application is based on and claims priority to U.S. Provisional Patent Application No. 63 / 218,817, filed July 6, 2021, the disclosure of which is incorporated herein by reference in its entirety.

[0002] SUMMARY OF THE DISCLOSURE Embodiments of the present disclosure relate to an apparatus and method for performing step-based processing in media processing on a cloud platform. [Background technology]

[0003] Networks and cloud platforms may be used to execute various applications. The Network-Based Media Processing (NBMP) standard provides specifications for defining, instantiating, and executing workflows on cloud platforms. The NBMP standard also includes a set of tools for independently processing multiple media segments of equal duration. For example, the NBMP step descriptor provides information about the functionality step-based processing support, where the NBMP step descriptor may define, among other parameters, the segment maximum duration / size and processing units. However, the step descriptor functionality of the NBMP standard does not define segment boundaries. One or more example embodiments of the present disclosure may solve this problem and provide the functionality to properly process inputs and outputs in the step descriptor by defining segment boundaries. Summary of the Invention

[0004] One or more example embodiments of the present disclosure provide a method and apparatus for providing step descriptor functionality by defining segment boundaries.

[0005] According to one aspect of the present disclosure, there is provided a method for performing step-based processing in media processing on a cloud platform, the method comprising: obtaining a workflow including an input media stream having a plurality of segments; identifying boundaries between the plurality of segments in the input media stream based on one of metadata-based segment boundary signaling or code-based segment boundary signaling; and processing the plurality of segments based on the identified boundaries between the plurality of segments.

[0006] According to another aspect of the present disclosure, an apparatus is provided for performing step-based processing in media processing on a cloud platform, the apparatus including: at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code including: acquisition code configured to cause the at least one processor to acquire a workflow including an input media stream, the input media stream including a plurality of segments; identification code configured to cause the at least one processor to identify boundaries between the plurality of segments in the input media stream based on one of metadata-based segment boundary signaling or code-based segment boundary signaling; and processing code configured to cause the at least one processor to process the plurality of segments based on the identified boundaries between the plurality of segments.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable medium is provided having instructions stored thereon that, when executed by at least one processor of an apparatus that performs step-based processing in media processing on a cloud platform, cause the at least one processor to obtain a workflow including an input media stream, the input media stream having a plurality of segments, identify boundaries between the plurality of segments in the input media stream based on one of metadata-based segment boundary signaling or code-based segment boundary signaling, and process the plurality of segments based on the identified boundaries between the plurality of segments.

[0008] Additional aspects will be set forth in part in the description that follows, and in part will be apparent from the description, or may be realized by practice of illustrative embodiments of the present disclosure. [Brief description of the drawings]

[0009] The above and other aspects, features and embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings, in which: [Figure 1] 1 is a schematic diagram of a communication system according to one or more embodiments. [Diagram 2] 1 is a simplified illustration of a streaming environment in accordance with one or more embodiments. [Diagram 3] FIG. 1 is a block diagram of an NBMP system according to one or more embodiments. [Figure 4] FIG. 1 is an illustration of an NBMP splitter and merge process according to one or more embodiments. [Diagram 5] FIG. 1 is a block diagram of an example process for performing step-based processing in media processing on a cloud platform in accordance with one or more embodiments. [Figure 6] FIG. 2 is a block diagram of an example of computer code for performing step-based processing in media processing on a cloud platform in accordance with one or more embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] SUMMARY OF THE DISCLOSURE Embodiments of the present disclosure relate to methods and apparatus for signaling changes in a network-based media processing (NBMP) workflow, and in particular to an apparatus and method for performing step-based processing in media processing on a cloud platform.

[0011] The embodiments of the present disclosure are exemplary in nature and will be comprehensively described with reference to the accompanying drawings. However, the implementation examples can be implemented in various forms, and the present disclosure should not be construed as being limited to the examples described herein. On the contrary, the implementation examples are provided to make the technical solutions of the present disclosure more comprehensive and complete, and to comprehensively convey the ideas of the implementation examples to those skilled in the art. The accompanying drawings are merely illustrative of the present disclosure, and are not necessarily drawn to scale. The same reference numerals in the accompanying drawings represent the same or similar components, and therefore repeated description of those components will be omitted.

[0012] The proposed features described below may be used separately or combined in any order. Some block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. Also, the embodiments may be implemented in the form of hardware, which may include processing circuits (e.g., one or more processors or one or more integrated circuits) or microcontroller devices, in the form of software, in different networks and / or processor devices, or by a combination of hardware and software components. In one example, one or more processors execute computer program code stored on one or more non-transitory computer-readable media.

[0013] Figure 1 is a diagram of an environment 100 in which the methods, devices, and systems described herein may be implemented, according to an embodiment. As shown in Figure 1, environment 100 may include a user device 110, a platform 120, and a network 130. The devices of environment 100 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0014] User device 110 may include one or more devices that can receive, generate, store, process, and / or provide information related to platform 120. For example, user device 110 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smartphone, a wireless phone, etc.), a wearable device (e.g., a pair of smart glasses or a smart watch), or a similar device. In some implementations, user device 110 may receive information from platform 120 and / or transmit information to platform 120.

[0015] Platform 120 may include one or more devices as described elsewhere herein. In some implementations, platform 120 may include a cloud server or a collection of cloud servers. In some implementations, platform 120 may be designed to be modular such that software components may be swapped in or out depending on particular needs. As such, platform 120 may be easily and / or quickly reconfigured for different uses.

[0016] In some implementations, as shown, platform 120 may be hosted within a cloud computing environment 122. In particular, although the implementations described herein describe platform 120 as being hosted within a cloud computing environment 122, in some implementations platform 120 may not be cloud-based (i.e., may be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0017] Cloud computing environment 122 may include an environment that hosts platform 120. Cloud computing environment 122 may provide services, such as computing, software, data access, storage, etc., that do not require end-user (e.g., user device 110) knowledge of the physical location and configuration of the systems and / or devices that host platform 120. As shown, cloud computing environment 122 may include a collection of computing resources 124 (collectively referred to as “computing resources 124” and individually referred to as “computing resource 124”).

[0018] Computational resources 124 may include one or more personal computers, workstation computers, server devices, or other types of computing and / or communication devices. In some implementations, computational resources 124 may host platform 120. Cloud resources may include computational instances running on computational resources 124, storage devices provided within computational resources 124, data transfer devices provided by computational resources 124, etc. In some implementations, computational resources 124 may communicate with other computational resources 124 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0019] 1, the computational resources 124 may include a collection of cloud resources, such as, for example, one or more applications (“APP”) 124-1, one or more virtual machines (“VM”) 124-2, virtualized storage (“VS”) 124-3, one or more hypervisors (“HYP”) 124-4, or the like. However, this disclosure is not so limited, and thus, according to other example embodiments, the computational resources 124 may include other types of cloud resources.

[0020] Application 124-1 may include one or more software applications that may be provided to or accessed by user device 110 and / or platform 120. Application 124-1 may eliminate the need to install and run a software application on user device 110. For example, application 124-1 may include software associated with platform 120 and / or any other software that may be provided via cloud computing environment 122. In some implementations, one application 124-1 may send / receive information to / from one or more other applications 124-1 via virtual machine 124-2.

[0021] Virtual machine 124-2 may include a software implementation of a machine (e.g., a computer) that executes programs like a physical machine. Virtual machine 124-2 may be either a system virtual machine or a process virtual machine, depending on the extent to which virtual machine 124-2 corresponds to any real machine and the intended use. A system virtual machine may provide a complete system platform that supports the execution of a complete operating system ("OS"). A process virtual machine may execute a single program and support a single process. In some implementations, virtual machine 124-2 may execute on behalf of a user (e.g., user device 110) and manage the infrastructure of cloud computing environment 122, such as data management, synchronization, or long-term data transfer.

[0022] Virtualized storage 124-3 may include one or more storage systems and / or one or more devices that use virtualization techniques among the storage systems or devices of the computing resources 124. In some implementations, within the context of a storage system, types of virtualization may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage, allowing the storage system to be accessed without regard to the physical storage or heterogeneity. This separation may allow administrators of storage systems flexibility in how they manage storage for end users. File virtualization may eliminate the dependency between data accessed at the file level and where the file is physically stored. This may enable optimization of storage usage, server consolidation, and / or performing uninterrupted file movement.

[0023] The hypervisor 124-4 may provide hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to run simultaneously on a host computer, such as the computing resource 124. The hypervisor 124-4 may present a virtual operating platform to the guest operating systems and may manage the execution of the guest operating systems. Multiple instances of various operating systems may share virtualized hardware resources.

[0024] Network 130 may include one or more wired and / or wireless networks. For example, network 130 may include a cellular network (e.g., a fifth generation (5G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad-hoc network, an intranet, the Internet, an optical fiber-based network, or the like, and / or a combination of these or other types of networks.

[0025] The number and configuration of devices and networks shown in Figure 1 are provided as an example. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or different configurations of devices and / or networks than those shown in Figure 1. Also, two or more devices shown in Figure 1 may be implemented within a single device, or a single device shown in Figure 1 may be implemented as multiple distributed devices. Additionally, or instead, a set of devices (e.g., one or more devices) of environment 100 may perform one or more functions described as being performed by another set of devices of environment 100.

[0026] Figure 2 is a block diagram of example components of one or more devices of Figure 1. Device 200 may correspond to user device 110 and / or platform 120. As shown in Figure 2, device 200 may include a bus 210, a processor 220, a memory 230, a storage component 240, an input component 250, an output component 260, and a communication interface 270.

[0027] The bus 210 includes components that enable communication between the components of the device 200. The processor 220 may be implemented in hardware, firmware, or a combination of hardware and software. The processor 220 may be a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or other type of processing component. In some implementations, the processor 220 may include one or more processors that can be programmed to perform functions. The memory 230 may include random access memory (RAM), read only memory (ROM), and / or other types of dynamic or static storage devices (e.g., flash memory, magnetic memory, and / or optical memory) that store information and / or instructions for use by the processor 220.

[0028] Storage component 240 stores information and / or software related to the operation and use of device 200. For example, storage component 240 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid state disks), compact disks (CDs), digital versatile disks (DVDs), floppy disks, cartridges, magnetic tapes, and / or other types of non-transitory computer-readable media along with corresponding drives.

[0029] Input components 250 may include components that enable device 200 to receive information, such as via user input (e.g., a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, and / or a microphone). Additionally or alternatively, input components 250 may include sensors that sense information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and / or an actuator). Output components 260 include components that provide output information from device 200 (e.g., a display, a speaker, and / or one or more light emitting diodes (LEDs)).

[0030] Communications interface 270 may include transceiver-like components (e.g., a transceiver and / or separate receivers and transmitters) that enable device 200 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communications interface 270 may enable device 200 to receive information from and / or provide information to other devices. For example, communications interface 270 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, or the like.

[0031] The device 200 may perform one or more of the processes described herein in response to the processor 220 executing software instructions stored by a non-transitory computer-readable medium, such as the memory 230 and / or the storage component 240. A computer-readable medium is defined herein as a non-transitory memory device. A memory device may include storage space within a single physical storage device or may include storage space spread across multiple physical storage devices.

[0032] Software instructions may be loaded into memory 230 and / or storage component 240 from other computer-readable media or from other devices via communication interface 270. The software instructions stored in memory 230 and / or storage component 240, when executed, may cause processor 220 to perform one or more processes described herein. Additionally, or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

[0033] The number and arrangement of components shown in Figure 2 are provided as an example. In practice, device 200 may include additional, fewer, different, or different arrangements of components than those shown in Figure 2. Additionally or alternatively, a set of components (e.g., one or more components) of device 200 may perform one or more functions described as being performed by another set of components of device 200.

[0034] In one embodiment of the present disclosure, a network-based media processing (NBMP) system is provided. FIG. 3 illustrates an NBMP architecture 300 according to an embodiment herein, which may be implemented for cloud processing. The NBMP system 300 includes an NBMP source 310, an NBMP workflow manager 320, a function repository 330, one or more media processing entities (MPEs) 340, a media source 350, and a media sink 360. The NBMP source 310, the NBMP workflow manager 320, the function repository 330, the MPE 340, the media source 350, and the media sink 360 may include or be implemented by at least one or more processors and a memory having stored therein code for causing the at least one or more processors to perform functions of the NBMP source 310, the NBMP workflow manager 320, the function repository 330, the MPE 340, the media source 350, and the media sink 360, respectively.

[0035] The NBMP source 310 may communicate workflow descriptions with the NBMP workflow manager 320 via the NBMP workflow API 311. The NBMP source 310 may also communicate feature descriptions with the feature repository 330 via the feature discovery API 313. For example, the NBMP source 310 may send a workflow description document (WDD) (one or more) to the NBMP workflow manager 320 and may read feature descriptions of features stored in the feature repository 330, such as media processing features stored in the memory of the feature repository 330, such as media decoding, feature point extraction, camera parameter extraction, projection method, seam information extraction, blending, post-processing, and encoding features. The NBMP workflow manager 320 may communicate with the feature repository 330 via a feature discovery API 312, which may be the same as or different from the feature discovery API 313, and may communicate with one or more of the MPEs 340 via an API 314 (e.g., an MPE API).

[0036] The media processing entity (MPE) 340 may include one or more tasks 341. The NBMP workflow manager 320 may also communicate with the tasks 341 via an API 315 (e.g., an NBMP task API). The NBMP workflow manager 320 may use the API 315 to set up, configure, manage, and monitor one or more tasks 341 of a workflow that may be executed by one or more MPEs 340. To configure, manage, and monitor the tasks 341 of a workflow, the NBMP workflow manager 320 may send messages, e.g., requests, to the MPEs 340 and / or one or more of the tasks 341, where each message may have several descriptors, each of which has several parameters. Additionally, the communication between the NBMP source 310, the NBMP workflow manager 320, the capability repository 330, and the MPEs 340 may be considered a control flow.

[0037] Each task 341 may include media processing functions 343 and configurations 342 for those media processing functions 343. Each task 341 in each media processing entity 340 may also communicate with each other to facilitate data flow between tasks. In one embodiment, the NBMP workflow manager 320 may select a task based on the task's description in the WDD and search the capability repository 330 via the capability discovery API 312 to find an appropriate function to perform as the task 341 for the current workflow. The one or more MPEs 340 may be configured to receive media content from a media source 350, process the media content according to a workflow including the tasks 341 created by the NBMP workflow manager 320, and output the processed media content to a media sink 360. In one embodiment, the one or more MPEs 340 may be provided in parallel for multiple media flows 316 and 317 between the media source 350 and the media sink 360, respectively.

[0038] The media source 350 may include memory for storing media and may be integrated with or separate from the NBMP source 310. In one embodiment, the NBMP workflow manager 320 may notify the NBMP source 310 when a workflow is prepared, and the media source 350 may send media content to the one or more MPEs 340 based on the notification that a workflow is prepared, which may send the media content to the one or more MPEs 340. The communication between the media source 350, the MPE 340, and the media sink 360 may be considered a data flow.

[0039] 4 illustrates an example of NBMP splitting and merging multiple segments of a media stream 440 of a task T for parallel processing in accordance with one embodiment of the present disclosure. As shown in FIG. 4, an NBMP splitter and merger process 400 includes a splitter 420 and a merger 430. In FIG. 4, an input media stream of task T is split into n instances of task T (i.e., T 0 , …, T N-1 ), with these n instances running in parallel. The input media stream 410 may be continuous. A splitter 420 converts the media stream into N media sub-streams. Each sub-stream is processed by an instance of T. A merger 430 then interleaves these sub-streams together to produce an output 450, which corresponds to the output stream of task T (i.e., media stream 440).

[0040] The 1:N splitter 420 and N:1 merger 430 functions as shown in FIG. 4 operate on segment boundaries. Each segment has start, duration, and length metadata associated with it. The segments are independent, and as a result, the substreams are independent of each other with respect to being processed by task T. In an embodiment, task T 0 , …, T N-1are instances of a task T, which may process segments simultaneously or separately. Since the segments and the substreams are independent, each instance of a task T (i.e., T 0 , …, T N-1 ) can run at its own speed.

[0041] According to embodiments of the present disclosure, methods for signaling segment boundaries are described. These methods may include metadata and start code methods. According to embodiments, for better support of step-based processing in media processing on cloud platforms, signaling of segment boundaries is required in NBMP step descriptors. According to embodiments, NBMP step descriptors may provide information about function step-based processing support, where a function may have information, e.g., a descriptor indicating whether the function can be processed in a step mode, where independent and / or parallel processing may take place based on defined segments. The descriptor may define, among other parameters, the maximum duration / size of a segment and the processing unit.

[0042] [Metadata method] According to one embodiment of the present disclosure, in a method for signaling segment boundaries using metadata, metadata may be provided to define a segment start, a segment duration, and a segment size. Such definitions may be provided for any of multiple segments of a task T. For example, a metadata definition may be as follows: S: Start of the segment in T scale units D: Duration of the segment in T scale units L: The length of the segment in bytes. · T: Timescale (number of ticks in seconds).

[0043] According to the metadata-based signaling method, any media stream input or output of a function shall have a corresponding metadata input or output carrying segment boundary metadata, so that splitting and merging segments of the media stream 440 of task T for parallel processing is properly performed when step-based processing is performed in media processing on the cloud platform.

[0044] [Start Code Method] According to an embodiment of the present disclosure, instead of a metadata method, a start code based method for signaling segment boundaries may be provided. According to the start code signaling method, a start code may be provided at the beginning of each segment. In this case, the media stream input may indicate the start code provided for each segment. Also, the media stream output may indicate the start code provided for each segment.

[0045] According to the start code based signaling method, a media stream input or output of any function shall have a corresponding metadata input or output carrying start codes identifying segment boundaries, so that start codes can be used appropriately to split and merge segments of the media stream 440 of task T for parallel processing when step-based processing is performed in media processing on the cloud platform.

[0046] According to one embodiment, by providing each of the input segments with the same start pattern (i.e., start code), the segment splitting and merging process can easily identify segment boundaries, and thus, information about the start time and duration of each segment (as in metadata-based approaches) is not required to signal segment boundaries.

[0047] According to one embodiment, start codes are provided to indicate the beginning of a segment, although the disclosure is not limited to start codes. Thus, according to other embodiments, patterns or codes may be provided in the input and output streams at different positions to indicate segment boundaries. For example, codes, such as end codes, may be provided at the end of a segment to define boundaries between segments.

[0048] According to an example embodiment, the step descriptor of the NBMP standard may be extended as follows: [Table 1]

[0049] For example, a segment metadata descriptor and a segment start code descriptor may be added to the step descriptor of the NBMP standard.

[0050] Additionally, according to an example embodiment, the step descriptor parameters of the NBMP standard may be extended as follows: [Table 2] TIFF0007639257000003.tif231170TIFF0007639257000004.tif49170

[0051] For example, the parameters of the segment metadata descriptor and the segment start code descriptor may be added to the step descriptor parameters of the NBMP standard.

[0052] According to an example embodiment, the segment metadata descriptor may be of type Boolean and may have the following definition: If 'TRUE', the function supports receiving / providing segment metadata for each media input / output for detecting segment boundaries. The default value may be 'FALSE'.

[0053] According to an example embodiment, the segment start code descriptor may be of type Boolean and may have the following definition: If 'TRUE', the function supports receiving / providing start codes for each media input / output for detecting segment boundaries.

[0054] According to an example embodiment, a method of identifying segment boundaries may include determining whether either the segment metadata or the segment start code description is true, and determining that the segment boundaries are identified using a metadata method based on a determination that the segment metadata is TRUE, and determining that the segment boundaries are identified using a start code method based on a determination that the segment start code is TRUE.

[0055] FIG. 5 is a flowchart of an example method 500 for performing step-based processing in a network-based media processing (NBMP) workflow in accordance with one or more embodiments.

[0056] 5 may be performed by platform 120. In some implementations, one or more of the process blocks in FIG. 5 may be performed by a device or devices separate from or including platform 120, such as, for example, user device 110.

[0057] 5, at operation 510, the method 500 includes obtaining a workflow, where the workflow may include an input media stream, where the input media stream may include a number of segments.

[0058] At operation 520, the method 500 includes identifying boundaries between the plurality of segments in the input media stream based on one of metadata-based segment boundary signaling or code-based segment boundary signaling. According to one embodiment, the code-based segment boundary signaling may include a start code provided at the beginning of each segment to indicate a boundary between the plurality of segments.

[0059] At operation 530, the method 500 includes processing the plurality of segments based on the identified boundaries between the plurality of segments.

[0060] According to one embodiment, the method 500 may further include generating a change workflow by dividing the plurality of segments into one or more tasks for independent processing of the change workflow, the one or more tasks of the change workflow being processed in parallel.

[0061] According to one embodiment, the method 500 may further include, in the code-based segment boundary signaling, receiving a code corresponding to the plurality of segments in the input media stream, and determining, each time the code occurs in the input media stream, that the code indicates a boundary between adjacent segments.

[0062] According to one embodiment, the method 500 may further include determining whether the input media stream supports one of metadata-based segment boundary signaling or code-based segment boundary signaling.

[0063] According to one embodiment, the method 500 may further include providing a first flag corresponding to the metadata based segment boundary signaling and providing a second flag corresponding to the code based segment boundary signaling.

[0064] According to one embodiment, the method 500 may further include merging one or more outputs from processing one or more tasks of the modified workflow to generate an output media stream.

[0065] According to one embodiment, the method 500 may further include applying one of metadata-based segment boundary signaling or code-based segment boundary signaling to the output media stream.

[0066] Although Figure 5 illustrates example blocks of the method, in some implementations the method may include additional, fewer, different, or differently organized blocks than those illustrated in Figure 5. Additionally or alternatively, two or more of the method blocks may be performed in parallel.

[0067] FIG. 6 is a block diagram of an example of computer code for performing step-based processing in a network-based media processing (NBMP) workflow in accordance with one or more embodiments.

[0068] According to an embodiment of the present disclosure, there may be provided at least one processor having a memory storing computer code, which when executed by the at least one processor may be configured to perform any number of aspects of the present disclosure.

[0069] For example, referring to FIG. 6, computer code 600 may be implemented within the NBMP system 300.

[0070] As shown in FIG. 6, computer code 600 may include acquisition code 610, identification code 620, and processing code 630.

[0071] The acquisition code 610 may be configured to cause the at least one processor to acquire a workflow. The workflow may include an input media stream, which may include a plurality of segments.

[0072] The identification code 620 may be configured to cause the at least one processor to identify boundaries between the plurality of segments in the input media stream based on one of metadata-based segment boundary signaling or code-based segment boundary signaling.

[0073] Processing code 630 may be configured to cause the at least one processor to process the plurality of segments based on identified boundaries between the plurality of segments.

[0074] Although Figure 6 illustrates example blocks of computer code 600 for a device or apparatus according to an embodiment, in some implementations the device may include additional, fewer, different, or differently organized blocks than those illustrated in Figure 6. Additionally or alternatively, two or more of the device's blocks may be executed in parallel.

[0075] The techniques for signaling segment boundaries of step-based processing in a network-based media processing (NBMP) workflow described above may be used separately or combined in any order, and each of the methods (or embodiments) may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors may execute a program stored on a non-transitory computer-readable medium.

[0076] This disclosure provides illustration and description, and is not intended to be exhaustive or to limit the implementations to the precise form disclosed. Modifications and variations are possible in light of the disclosure or may be acquired from practice of the implementations.

[0077] As used herein, the term component is intended to be interpreted broadly as hardware, firmware, or a combination of hardware and software.

[0078] It will be apparent that the systems and / or methods described herein may be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is not intended to be a limitation of the implementation. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code, with the understanding that software and hardware may be designed to implement the systems and / or methods based on the description herein.

[0079] Although combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. Indeed, many of the features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set.

[0080] No element, act, or instruction used herein should be construed as critical or essential unless expressly stated as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and may be used interchangeably with "one or more." Additionally, as used herein, the term "set" is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with "one or more." When only one item is intended, the term "a" or similar language is used. Also, as used herein, the terms "have," "having," "having," and the like are intended to be open-ended terms. Additionally, the phrase "based on" is intended to mean "based at least in part on" unless expressly stated otherwise.

Claims

1. 1. A processor-executed method for performing step-based processing in media processing on a cloud platform, the method comprising: obtaining a workflow including an input media stream, the input media stream having a plurality of segments; determining whether the input media stream supports metadata-based segment boundary signaling or code-based segment boundary signaling based on a first flag indicating whether metadata-based segment boundary signaling is supported and a second flag indicating whether code-based segment boundary signaling is supported; identifying boundaries between the plurality of segments in the input media stream based on one of the metadata-based segment boundary signaling or the code-based segment boundary signaling, the determined one being supported; processing the plurality of segments based on the identified boundaries between the plurality of segments; having The method further comprises: receiving codes corresponding to the plurality of segments in the input media stream in the code-based segment boundary signaling; determining, for each occurrence of said code in said input media stream, that said code indicates a boundary between adjacent segments; having method.

2. generating a change workflow by dividing the plurality of segments into one or more tasks for independent processing of the change workflow; The method of claim 1 further comprising:

3. The method of claim 1 , wherein the codes include a start code provided at the beginning of each segment to indicate the boundaries between the multiple segments.

4. The method of claim 2 , wherein the one or more tasks of the change workflow are processed in parallel.

5. merging one or more outputs from processing one or more tasks of the modified workflow to generate an output media stream; The method of claim 2 further comprising:

6. applying one of said metadata-based segment boundary signaling or said code-based segment boundary signaling to said output media stream; The method of claim 5 further comprising:

7. An apparatus for performing step-based processing in media processing on a cloud platform, the apparatus comprising: one or more processors; one or more memories storing a computer program; having The computer program causes the one or more processors to carry out a method according to any one of claims 1 to 6. Device.

8. A computer program causing a computer to carry out the method according to any one of claims 1 to 6.

Images