Method and apparatus for parallel transcoding of media content on the cloud

Abstract

A system and method for parallel transcoding of media content is provided that splits media content into parallel media sub-streams, performs transcoding on each media sub-stream using a transcoder, and then merges the parallel media sub-streams into a single transcoded stream. The parallel transcoding may be performed by one or more media processing entities as part of a workflow that includes splitter tasks, transcoder tasks, and merger tasks, and the one or more media processing entities may be controlled to report the status of the tasks.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 253,053, filed October 6, 2021. The disclosure of this provisional application is incorporated by reference in its entirety herein.

[0002] Embodiments of the present disclosure are directed to a system and method for parallel transcoding of media content, which splits media content into parallel media sub-streams, transcodes each media sub-stream using a transcoder, and merges the parallel media sub-streams into a single transcoded stream. [Background technology]

[0003] Network cloud platforms are used to run a variety of applications. The Network-Based Media Processing (NBMP) standard includes specifications for the definition, instantiation, and execution of workflows on cloud platforms. The standard also defines splitter and merger function templates that use metadata to signal segment boundaries. Summary of the Invention [Problem to be solved by the invention]

[0004] According to an embodiment, a cloud service is provided to operate multiple transcoders, which can increase the transcoding speed, for example by increasing the number of parallel sub-streams. [Means for solving the problem]

[0005] According to an embodiment, there is provided a method executed by at least one processor implementing a network-based media processing (NBMP) workflow manager. The method includes creating an NBMP workflow including a diverter task for diverting a compressed video stream into compressed sub-streams, a transcoder task for transcoding each of the compressed sub-streams to become a transcoded sub-stream, and a merger task for merging the transcoded sub-streams into one transcoded sub-stream. The method further includes controlling at least one media processing entity to execute the NBMP workflow, and controlling the at least one media processing entity executing the NBMP workflow to report to another entity at least one of a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a merger state of the merger task.

[0006] According to one or more embodiments, at least one media processing entity is controlled to report a diverter status of a diverter task.

[0007] According to one or more embodiments, the at least one media processing entity is controlled to report a transcoder status of at least one of the transcoder tasks.

[0008] According to one or more embodiments, at least one media processing entity is controlled to report a junction state of a junction task.

[0009] According to one or more embodiments, at least one media processing entity is controlled to report a diverter state of a diverter task, a transcoder state of at least one of the transcoder tasks, and a merger state of the merger task.

[0010] According to one or more embodiments, controlling the at least one media processing entity to report includes controlling the at least one media processing entity to report to a reporting server a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a junction state of the junction task.

[0011] According to one or more embodiments, controlling the at least one media processing entity to report includes controlling the at least one media processing entity to report at least one of a diverter state of a diverter task, a transcoder state of at least one of a transcoder task, and a junction state of a junction task based on information in a workflow description document (WDD) received by the NBMP workflow manager.

[0012] According to one or more embodiments, controlling the at least one media processing entity to report includes controlling the at least one media processing entity to report at least one of a diverter state of a diverter task, a transcoder state of at least one of a transcoder task, and a junction state of a junction task based on information in a workflow description document (WDD) received by the NBMP workflow manager.

[0013] According to one or more embodiments, controlling the at least one media processing entity to report includes controlling the at least one media processing entity to report at least one of a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a junction state of the junction task while a corresponding one of the diverter task, the transcoder task, and the junction task is executing.

[0014] According to one or more embodiments, the reporting server is configured to enable at least one of the diverter status of the diverter task, the transcoder status of at least one of the transcoder tasks, and the junction status of the junction task to be visualized on the web dashboard.

[0015] According to an embodiment, a system is provided that includes at least one memory configured to store computer program code and at least one processor configured to access and operate as directed by the computer program code, the computer program code including creation code configured to cause a network-based media processing (NBMP) workflow manager implemented by the at least one processor to create an NBMP workflow that includes a splitter task for splitting a compressed video stream into compressed sub-streams, a transcoder task for transcoding each of the compressed sub-streams to become a transcoded sub-stream, and a merger task for merging the transcoded sub-streams into one transcoded sub-stream. The computer program code further includes first control code configured to cause the NBMP workflow manager to control the at least one media processing entity to execute the NBMP workflow, and second control code configured to cause the NBMP workflow manager to control the at least one media processing entity executing the NBMP workflow to report to another entity at least one of a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a junction state of the junction task.

[0016] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control the at least one media processing entity to report a diverter status of the diverter task.

[0017] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control the at least one media processing entity to report a transcoder status of at least one of the transcoder tasks.

[0018] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control the at least one media processing entity to report a junction status of the junction task.

[0019] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control the at least one media processing entity to report a diverter status of the diverter task, a transcoder status of at least one of the transcoder tasks, and a junction status of the junction task.

[0020] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control the at least one media processing entity to report to a reporting server a diverter status of the diverter task, a transcoder status of at least one of the transcoder tasks, and a junction status of the junction task.

[0021] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control the at least one media processing entity to report at least one of a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a junction state of the junction task based on information in a workflow description document (WDD) received by the NBMP workflow manager.

[0022] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control the at least one media processing entity to report at least one of a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a junction state of the junction task based on information in a workflow description document (WDD) received by the NBMP workflow manager.

[0023] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control the at least one media processing entity to report at least one of a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a junction state of the junction task while a corresponding one of the diverter task, the transcoder task, and the junction task is executing.

[0024] According to an embodiment, a non-transitory computer-readable medium is provided having computer code stored thereon, which, when executed by at least one processor, is configured to cause the at least one processor to implement a network-based media processing (NBMP) workflow manager that creates an NBMP workflow including a diverter task diverting a compressed video stream into compressed sub-streams, a transcoder task transcoding each of the compressed sub-streams to become transcoded sub-streams, and a merger task merging the transcoded sub-streams into one transcoded sub-stream. The computer code is further configured to cause the at least one processor to control the at least one media processing entity to execute the NBMP workflow, and to control the at least one media processing entity executing the NBMP workflow to report to another entity at least one of a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a merger state of the merger task.

[0025] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0026] [Figure 1] 1 is a diagram of an environment in which the methods, apparatus and systems described in the present application may be implemented, according to an embodiment. [Diagram 2] 2 is a block diagram of an example of elements of one or more devices of FIG. 1. [Diagram 3] FIG. 1 is a block diagram of an NBMP system according to an embodiment. [Figure 4] FIG. 2 is a diagram of an example architecture for parallel transcoding using splitter and joiner functions according to an embodiment. [Diagram 5]FIG. 13 is a diagram of an example of a shunt function according to an embodiment. [Figure 6] FIG. 13 is a diagram of an example of a junction function according to an embodiment. [Figure 7] FIG. 1 is a block diagram of a system according to an embodiment. [Figure 8] FIG. 2 is a block diagram of computer code according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Figure 1 is a diagram of an environment 100 in which the methods, apparatus and systems described herein may be implemented, according to an embodiment. As shown in Figure 1, environment 100 may include a user device 110, a platform 120 and a network 130. The devices in environment 100 may be connected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0028] User device 110 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to platform 120. For example, user device 110 may include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smartphone, a wireless phone, etc.), a wearable device (e.g., smart glasses or a smart watch), or the like. In some implementations, user device 110 may receive information from platform 120 and / or transmit information to platform 120.

[0029] Platform 120 includes one or more devices as described elsewhere herein. In some implementations, platform 120 may include a cloud server or a group of cloud servers. In some implementations, platform 120 may be designed to be modular such that software components can be swapped in and out depending on what is needed at the time. Thus, platform 120 can be easily and / or quickly reconfigured for different uses.

[0030] In some implementations, as shown, platform 120 may be provided by hosting in a cloud computing environment 122. In particular, while the implementations described herein are described as providing platform 120 by hosting in a cloud computing environment 122, in some implementations platform 120 may not be cloud-based (i.e., may be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0031] Cloud computing environment 122 includes an environment that hosts platform 120. Cloud computing environment 122 may provide computing services, software services, data access services, storage services, and the like, without requiring end-user (e.g., user device 110) awareness of the physical location and configuration of one or more systems and / or one or more devices that host platform 120. As shown, cloud computing environment 122 may include a collection of computing resources 124 (collectively referred to as “computing resources 124” and individually referred to as “computing resource 124”).

[0032] Computational resources 124 may include one or more personal computers, workstation computers, server devices, or other types of computing and / or communication devices. In some implementations, computational resources 124 may provide hosting for platform 120. Cloud resources may include computing instances running on computational resources 124, storage devices provided by computational resources 124, data transfer devices provided by computational resources 124, etc. In some implementations, computational resources 124 may communicate with other computational resources 124 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0033] As further shown in FIG. 1, the computational resources 124 include a group of cloud resources, such as one or more applications (APP) 124-1, one or more virtual machines (VM) 124-2, virtual storage (VS) 124-3, and one or more hypervisors (HYP) 124-4.

[0034] The application 124-1 may include one or more software applications that may be provided to or accessed by the user device 110 and / or the platform 120. The application 124-1 may eliminate the need for the software application to be installed on the user device 110 to run on the user device 110. For example, the application 124-1 may include software related to the platform 120 and / or any other software that may be provided via the cloud computing environment 122. In some implementations, an application 124-1 may send information to one or more other applications 124-1 via the virtual machine 124-2, or an application 124-1 may receive information from one or more other applications 124-1 via the virtual machine 124-2.

[0035] Virtual machine 124-2 includes a software implementation of a machine (e.g., a computer) that executes programs like a physical machine. Virtual machine 124-2 may be either a system virtual machine or a process virtual machine depending on the application and the degree to which virtual machine 124-2 matches any real machine. A system virtual machine can provide a complete system platform that supports the execution of a complete operating system (OS). A process virtual machine can execute only one program and support only one process. In some implementations, virtual machine 124-2 may act on behalf of a user (e.g., user device 110) and virtual machine 124-2 may manage the infrastructure of cloud computing environment 122, such as data management, synchronization, and long-term data transfer.

[0036] Virtual storage 124-3 includes one or more storage systems and / or one or more devices that use virtualization techniques in the storage systems or devices of the computing resources 124. In some implementations, with respect to storage systems, types of virtualization may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage such that the storage system may be accessed whether it is physical or heterogeneous. Separation allows storage system administrators more flexibility in how they manage storage for end users. File virtualization allows the removal of dependencies between data being accessed at a given file level and where the file is physically stored. This may allow for optimization of storage usage, server consolidation, and / or non-disruptive file migration.

[0037] The hypervisor 124-4 may provide hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to simultaneously execute on a host computer, such as the computational resource 124. The hypervisor 124-4 may provide a virtual operating platform for the guest operating systems and may manage the execution of the guest operating systems. Multiple instances of different operating systems may share virtualized hardware resources.

[0038] Network 130 may include one or more wired and / or wireless networks. For example, network 130 may include a cellular network (e.g., a fifth generation (5G) network, a long term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the public switched telephone network (PSTN)), a private network, an ad-hoc network, an intranet, the Internet, a fiber optic based network, etc., and / or a combination of these or other types of networks.

[0039] The number and arrangement of devices and networks shown in Figure 1 are shown as an example. In practice, there may be more devices and / or networks, fewer devices and / or networks, different devices and / or networks, or different arrangements of devices and / or networks compared to those shown in Figure 1. Furthermore, two or more devices shown in Figure 1 may be implemented in one device, or one device shown in Figure 1 may be implemented as multiple distributed devices. Additionally or alternatively, a collection of devices (e.g., one or more devices) of environment 100 may perform one or more functions that are performed by another collection of devices of environment 100.

[0040] Figure 2 is a block diagram of example elements of one or more devices of Figure 1. Device 200 may correspond to user device 110 and / or platform 120. As shown in Figure 2, device 200 may include a bus 210, a processor 220, a memory 230, a storage element 240, an input element 250, an output element 260, and a communication interface 270.

[0041] The bus 210 includes elements that enable communication between elements of the device 200. The processor 220 may be implemented in hardware, firmware, or a combination of hardware and software. The processor 220 may be a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processor. In some implementations, the processor 220 includes one or more processors that can be programmed to perform functions. The memory 230 includes random access memory (RAM), read only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions used by the processor 220.

[0042] The storage element 240 stores information and / or software related to the operation and use of the device 200. For example, the storage element 240 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cartridge, a magnetic tape, and / or another type of non-transitory computer-readable medium along with a corresponding drive.

[0043] Input elements 250 include elements that enable device 200 to accept information, such as via user input (e.g., a touchscreen display, a keyboard, a keypad, a mouse, buttons, switches, and / or a microphone). Additionally or alternatively, input elements 250 may include sensors that sense information (e.g., a Global Positioning System (GPS) element, an accelerometer, a gyroscope, and / or an actuator). Output elements 260 include elements that provide output information from device 200 (e.g., a display, a speaker, and / or one or more light emitting diodes (LEDs)).

[0044] Communications interface 270 includes transceiver-like elements (e.g., a transceiver and / or separate receivers and transmitters) that enable device 200 to communicate with other devices, such as via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communications interface 270 may be used to enable device 200 to receive information from and / or provide information to another device. For example, communications interface 270 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0045] Device 200 may perform one or more of the processes described herein. Device 200 may perform the processes in response to processor 220 executing software instructions stored by a non-transitory computer-readable medium, such as memory 230 and / or storage element 240. A computer-readable medium is defined in this application as a non-transitory memory device. A memory device includes storage space in one physical storage device or across multiple physical storage devices.

[0046] The software instructions may be read from another computer-readable medium or from another device via communications interface 270 and loaded into memory 230 and / or storage element 240. When executed, the software instructions stored in memory 230 and / or storage element 240 may be used to cause processor 220 to perform one or more processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

[0047] The number and arrangement of elements shown in Figure 2 are provided as an example. In practice, device 200 may include more elements, fewer elements, different elements, or a different arrangement of elements compared to those shown in Figure 2. Additionally or alternatively, a collection of elements (e.g., one or more elements) of device 200 may perform one or more functions that are performed by another collection of elements of device 200.

[0048] In one embodiment of the present disclosure, there is provided an NBMP system 300. With reference to Figure 3, the NBMP system 300 includes an NBMP provider 310, an NBMP workflow manager 320, a function repository 330, one or more media processing entities 350, a media provider 360, and a media destination 370.

[0049] NBMP Provider 310 may receive instructions from third party entity 380 and may communicate with NBMP Workflow Manager 320 via NBMP Workflow API 392 and with Feature Repository 330 via Feature Discovery API 391. For example, NBMP Provider 310 may send one or more workflow description documents (WDDs) to NBMP Workflow Manager 320 and may retrieve feature descriptions of features stored in Feature Repository 330, such as media processing features stored in a memory of Feature Repository 330, such as media decoding, feature extraction, camera parameter extraction, projection methods, seam information extraction, compositing, post-processing and / or encoding features. NBMP Provider 310 may include or be implemented by at least one processor and at least one memory storing code configured to cause the processor to perform at least the features of NBMP Provider 310.

[0050] An NBMP provider 310 may request the NBMP workflow manager 320 to create a workflow including tasks 352 to be executed by one or more media processing entities 350 by sending a workflow description document, which may include several descriptors, each of which may have several parameters.

[0051] For example, the NBMP provider 310 may select a function stored in the function repository 330 and send a workflow description document to the NBMP workflow manager 320, where the workflow description document includes various descriptors used to describe details such as input and output data, required functions and requirements of the workflow. The workflow description document may include a collection of task descriptions and a connectivity map of inputs and outputs of the tasks 352 to be performed by one or more of the media processing entities 350. When the NBMP workflow manager 320 receives such information from the NBMP provider 310, the NBMP workflow manager 320 may create a workflow by instantiating tasks based on the function names and connecting the tasks according to the connectivity map.

[0052] Alternatively or additionally, NBMP provider 310 may request NBMP workflow manager 320 to create a workflow using a set of keywords. For example, NBMP provider 310 may send NBMP workflow manager 320 a workflow description document that may include a set of keywords that NBMP workflow manager 320 may use to search for suitable functions stored in function repository 330. When NBMP workflow manager 320 receives such information from NBMP provider 310, NBMP workflow manager 320 may create a workflow by searching for suitable functions using keywords that may be specified in process descriptors in the workflow description document, and may use other descriptors in the workflow description document to provision tasks and connect tasks to create a workflow.

[0053] The NBMP workflow manager 320 may communicate with the feature repository 330 via a feature detection API 393, which may be the same API as the feature detection API 391 or a different API. The NBMP workflow manager 320 may communicate with one or more of the media processing entities 350 via an NBMP task API 394. The NBMP workflow manager 320 may also communicate with one or more of the media processing entities 350 via a media processing entity (MPE) API 396. The NBMP workflow manager 320 may comprise or be implemented by at least one processor and at least one memory storing code configured to cause the processor to perform at least the functions of the NBMP workflow manager 320.

[0054] The NBMP workflow manager 320 may use the NBMP task API 394 to build, configure, manage, and monitor one or more tasks 352 of a workflow that can be executed by one or more media processing entities 350. In one embodiment, the NBMP workflow manager 320 may use the NBMP task API 394 to update and destroy tasks 352. To set up, manage, and monitor the tasks 352 of a workflow, the NBMP workflow manager 320 may send messages, such as requests, to one or more of the media processing entities 350. Each message may have several descriptors, and each of the descriptors may have several parameters. Each of the tasks 352 may include a media processing function 354 and a configuration 353 of the media processing function 354.

[0055] In one embodiment, after receiving a workflow description document from the NBMP provider 310 that does not include a list of tasks (e.g., includes a list of keywords instead of a list of tasks), the NBMP workflow manager 320 may select a task via the function detection API 393 based on the description of the task in the workflow description document to search the function repository 330 to detect a suitable function to be performed as a task 352 of the target workflow. For example, the NBMP workflow manager 320 may select a task based on a keyword provided in the workflow description document. After a suitable function is identified using a set of keywords or task descriptions provided by the NBMP provider 310, the NBMP workflow manager 320 may configure the selected task in the workflow using the NBMP task API 394. For example, the NBMP workflow manager 320 may extract configuration data from the information received from the NBMP provider and configure the task 352 based on the configuration data.

[0056] The one or more media processing entities 350 may be configured to receive media content from a media source 360, process the media content according to a workflow that includes tasks 352 and is created by the NBMP workflow manager 320, and output the processed media content to a media destination 370. Each of the one or more media processing entities 350 may comprise or be implemented by at least one processor and at least one memory that stores code configured to cause the processor to perform at least the functions of the media processing entity 350.

[0057] Media provider 360 may include memory for storing media and may be integrated with or separate from NBMP provider 310. In one embodiment, NBMP workflow manager 320 may notify NBMP provider 310 when a workflow is ready, and media provider 360 may send media content to one or more of media processing entities 350 based on the notification that a workflow is ready.

[0058] The media destination 370 may comprise or be implemented by at least one processor and at least one display configured to display media processed by the one or more media processing entities 350.

[0059] The third party entity 380 may comprise or be implemented by at least one processor and at least one memory storing code configured to cause the processor to perform at least the functions of the third party entity 380.

[0060] As described above, messages from the NBMP provider 310 to the NBMP workflow manager 320 (e.g., a workflow description document to request the creation of a workflow) and messages from the NBMP workflow manager 320 to one or more media processing entities 350 (e.g., messages to cause a workflow to be executed) may include several descriptors, each of which may have several parameters. In some examples, communications between any of the elements of the NBMP system 300 using APIs may include several descriptors, each of which may have several parameters.

[0061] According to an embodiment, a cloud service is provided to operate multiple transcoders, which can increase the transcoding speed, for example by increasing the number of parallel sub-streams.

[0062] According to an embodiment, an architecture 400 shown in FIG. 4 can be implemented, which is a parallel transcoding architecture using splitter and joiner functions.

[0063] 4, a single stream 491 from a media source 410 is split into multiple sub-streams 492 by a splitter 420. Then, each sub-stream 492 is transcoded by a corresponding transcoder (transcoder 430-1, 430-2, ... 430-N), and the transcoded sub-stream 493 is sent from the transcoder to a combiner 440. The combiner 440 combines the transcoded sub-streams 493 to generate an output 494 to a media destination 450. The output 494 may include a single transcoded stream including the transcoded sub-streams 493. According to an embodiment, the media source 410 may correspond to the media source 360 ​​of FIG. 3, and the media destination 450 may correspond to the media destination 370 of FIG. 3. According to an embodiment, the splitter 420, the transcoders (transcoders 430-1, 430-2, ... 430-N) and the joiner 440 may be functions performed by one or more of the media processing entities 350 of FIG. 3 as part of one or more of the tasks 352 (e.g., media processing functions 354 of FIG. 3).

[0064] With reference to Figs. 5 to 6, an embodiment of the diverter 420 and the merger 440 will be described. For example, as shown in Figs. 5 to 6, the diverter 420 may be a diverter function including 1:2 divergence, and the merger 440 may be a merger function including 2:1 merging. However, the diverting ratio and merging ratio of the diverter 420 and the merger 440 are not limited thereto. According to an embodiment, the media processing entity 350 performing the diverter function and the merging function may allow the number of diverting and / or the number of merging of setting parameters defined in the standard. For example, the corresponding setting 353 of the task 352 may include at least one indicator, which may indicate the number of diverting and / or the number of merging performed by the diverter function and / or the merging function included in the task 352.

[0065] With reference to Figures 5-6, both the splitter and merger functions use input and output buffers that are configurable as part of the function template. The inputs and outputs of the splitter and merger functions may be indexed streams provided with timing metadata. The associated timing metadata stream for each segment of the indexed stream may include start time, duration and byte length metadata. Corresponding configuration parameters may be segment duration, segment metadata and repeat-header.

[0066] According to an embodiment, and referring to FIG. 5, the diverter 420 may include an input buffer 422, diverter logic 424, and a number of output buffers (e.g., a first output buffer 426-1 and a second output buffer 426-2). The input buffer 422 may be configured to receive a compressed video stream that is indexed and provided with timing metadata. As an example, the compressed video stream may include compressed segments s4, s3, s2, and s1. The diverter logic 424 may be configured to divert the compressed video stream into compressed sub-streams and provide the compressed sub-streams to the output buffers, respectively. For example, the diverter logic 424 may provide a compressed sub-stream including compressed segments s3 and s1 to the first output buffer 426-1 and a compressed sub-stream including compressed segments s4 and s2 to the second output buffer 426-2. The output buffers may be configured to output the compressed sub-streams received by the output buffers. According to an embodiment, the splitter 420 may output a compressed sub-stream including compressed segments s3 and s1 to a first transcoder (e.g., transcoder 430-1 in FIG. 4) and a compressed sub-stream including compressed segments s4 and s2 to a second transcoder (e.g., transcoder 430-2 in FIG. 4), and the first and second transcoders may perform transcoding on the received compressed sub-streams and output the transcoded sub-streams to the combiner 440.

[0067] 6, the joiner 440 may include multiple input buffers (e.g., a first input buffer 442-1 and a second input buffer 442-2), a joiner logic 444, and an output buffer 446. The input buffers may be configured to receive the transcoded substreams, respectively. For example, the first input buffer 442-1 may receive a transcoded substream including transcoded segments s3* and s1*, and the second input buffer 442-2 may receive a transcoded substream including transcoded segments s4* and s2*. Each of the first input buffer 442-1 and the second input buffer 442-2 may be configured to output the respective received transcoded substream to the joiner logic 444. The combiner logic 444 may be configured to combine the transcoded sub-streams into a single transcoded video stream including transcoded segments (e.g., transcoded segments s4*, s3*, s2*, and s1*) and send the single transcoded video stream to an output buffer 446. The output buffer 446 may be configured to output the single transcoded video stream. According to an embodiment, the combiner 440 may output the single transcoded video stream to a media destination 450. The single transcoded video stream may be indexed and provided with timing metadata.

[0068] According to an embodiment, diverter 420, transcoder, and merger 440 may report the operation of these elements to a reporting server. For example, a media processing entity (or entities) implementing diverter 420, transcoder, and / or merger 440 may send information to another element (e.g., a reporting server) indicating one or more diverter states, one or more transcoder states, and / or one or more merger states of one or more functions performed by the media processing entity.

[0069] With reference to FIG. 7, examples of operations performed by embodiments of the present disclosure are described below. In FIG. 7, solid lines with arrows indicate NBMP API operations, and dotted lines with arrows indicate media processing. FIG. 7 illustrates a system 700 including NBMP management elements. According to an embodiment, the system 700 may include an NBMP client 710, an NBMP service 720, a media source 410, a media destination 450, and a reporting server implementing a web UI backend 740 and a web dashboard 742. According to an embodiment, the reporting server may be implemented by at least one processor and at least one memory storing computer code configured to cause at least the processor to perform the functions of the reporting server. According to an embodiment, the at least one processor and / or memory may be the same as or different from the at least one processor and / or memory used to implement other elements of the embodiments of the present disclosure.

[0070] According to an embodiment, the NBMP client 710 may be implemented by the NBMP provider 310 of Figure 3. The NBMP service 720 may include a function repository 722, a workflow manager 724, and a task manager 726. According to an embodiment, the function repository 722 may be implemented by the function repository 330 of Figure 3, and the workflow manager 724 and the task manager 726 may be implemented by the NBMP workflow manager 320 of Figure 3. According to an embodiment, the workflow 730 may include a splitter 420, a transcoder 430, and a joiner 440, which may be included in tasks 352 implemented by one or more media processing entities 350 of Figure 3.

[0071] According to an embodiment, the system 700 may perform a method including the steps of (1) creating a workflow; (2) obtaining available features; (3) creating splitter, transcoder, and junction tasks; (4) executing the workflow; (5) streaming media for the workflow; and (6) visualizing workflow and task status.

[0072] For example, the NBMP client 710 may create and send a WDD 781 describing the workflow 730 to the workflow manager 724, which may create the workflow 730. According to an embodiment, the workflow 730 may be created as described above with reference to FIG. 3. After receiving the WDD 781, the workflow manager 724 may retrieve (782) a feature template from the feature repository 722 to create the workflow 730. According to an embodiment, the workflow manager 724 (e.g., the NBMP workflow manager 320) may interact with the feature repository 722 (e.g., the feature repository 330) as described above with reference to FIG. 3. Based on the WDD 781 and the feature template, the workflow manager 724 may create and order (783) the tasks of the workflow 730 to be executed by one or more media processing entities (e.g., the media processing entity 350). The tasks may include the media processing functions of the splitter 420, the transcoder 430, and the joiner 440. After the workflow manager 724 creates and sequences the tasks of the workflow 730, the task manager 726 may control (784) one or more media processing entities to perform the tasks of the workflow 730. The tasks may be performed by the media source 410 streaming (785) the media to the media processing entity implementing the diverter 420, processing the media by the one or more media processing entities implementing the diverter 420, the transcoder 430 and the merger 440, and outputting (791) the processed media content to the media destination 450 by the media processing entity implementing the merger 440. According to an embodiment, the media streamed by the media source 410 may be an indexed h265 stream. The functionality of the diverter 420, the transcoder 430 and the merger 440 can be understood from the above description with reference to Figs. 4-6.

[0073] According to an embodiment, the workflow manager 724 may report (786) a workflow status to the reporting server web UI backend 740, the task manager 726 may report (787) a task status to the reporting server web UI backend 740, the diverter 420 may report (788) a diverter status to the reporting server web UI backend 740, the transcoder 430 may report (789) a transcoder status corresponding to the transcoder 430 to the reporting server web UI backend 740, and the merger 440 may report (790) a merger status to the reporting server web UI backend 740. Reporting the status may include sending first information including an indication of the respective status. According to an embodiment, the WDD 781 may include second information indicating where one or more of the workflow manager 724, the task manager 726, the diverter 420, the transcoder 430, and the junction 440 should report their corresponding status, and the second information may further indicate what is to be reported. According to an embodiment, the workflow manager 724 and / or the task manager 726 may report their corresponding status based on the second information, and the workflow manager 724 and / or the task manager 726 may control one or more media processing entities to report the status of the diverter 420, the transcoder 430, and / or the junction 440 implemented by the one or more media processing entities based on the second information.

[0074] According to an embodiment, the reporting server may be used to provide 792 visualization of data in a web dashboard 742 based on the first information received by the reporting server. For example, the visualized data may include workflow states, task states, workflow graphs, and media destination video players. According to an embodiment, diverter states, transcoder states, and merger states may also be visualized. The reporting server may be configured to provide the visualization of data by causing at least one display to display the web dashboard 742.

[0075] According to an embodiment, a system and method for parallel transcoding of media streaming using two or more transcoders can be provided that increases the effective speed of transcoding. The system and method can be used to implement the NBMP splitter and joiner functions of the NBMP standard such that split / join numbers and functions can be configured using timing metadata. The system and method can be used to manage the instantiation, deployment, management and monitoring of workflows using the NBMP standard, where progress is reported in real time to a web-based dashboard by workflow tasks as well as the NBMP workflow manager.

[0076] According to embodiments of the present disclosure, at least one processor may be provided along with a memory storing computer code, which when executed by the at least one processor may be configured to perform any number of aspects of the present disclosure.

[0077] 8, computer code 800 may be implemented in NBMP system 300. For example, the computer code may be stored in a memory of NBMP workflow manager 320 and executed by at least one processor of NBMP workflow manager 320. The computer code may comprise, for example, an acquisition code 810, a creation code 820, a first control code 830, and a second control code 840.

[0078] The acquisition code 810 may be configured to cause the NBMP workflow manager 320 to acquire information from the WDD, according to an embodiment of the present disclosure. For example, the NBMP workflow manager 320 may receive the WDD and may signal the parameters in the WDD to the NBMP workflow manager 320 such that the NBMP workflow manager 320 acquires the corresponding information.

[0079] The creation code 820 may be configured to cause the NBMP workflow manager 320 to create a media processing workflow including tasks 352, according to an embodiment of the present disclosure. For example, the tasks 352 may include the functionality of the splitter 420, the transcoder 430, and the joiner 440. According to an embodiment, the media processing workflow may be created based on information obtained from the WDD.

[0080] The first control code 830 may be configured to cause the NBMP workflow manager 320 to control at least one media processing entity 350 to execute a media processing workflow, according to an embodiment of the present disclosure.

[0081] The second control code 840 may be configured to cause the NBMP workflow manager 320 to control at least one media processing entity 350 implementing the media processing workflow to report at least one of a diverter status, a transcoder status, and a merger status, according to an embodiment of the present disclosure. For example, the media processing entity 350 implementing the diverter 420 may be controlled to report a diverter status, the media processing entity 350 implementing the transcoder 430 may be controlled to report a transcoder status, and the media processing entity 350 implementing the merger 440 may be controlled to report a merger status. According to an embodiment, the NBMP workflow manager 320 may perform the control based on information obtained from the WDD. For example, the NBMP workflow manager 320 may control which information to report and where based on information obtained from the WDD. According to an embodiment, the NBMP workflow manager 320 may control the media processing entity 350 to report the status to a reporting server.

[0082] In accordance with one or more embodiments, embodiments of the present disclosure may be implemented in environments other than the NBMP.

[0083] While the above disclosure has been illustrated and described, it is not intended to be exhaustive or to limit the implementation to the precise forms disclosed. Modifications and variations are possible in light of the above disclosure and may be acquired from practicing the implementations.

[0084] The term elements as used in this application are intended to be broadly interpreted as hardware, firmware, or a combination of hardware and software.

[0085] Even if a combination of features is recited in a claim and / or described in the specification, the combination is not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not explicitly recited in the claim and / or disclosed in the specification. Although each dependent claim listed below may depend directly on only one claim, each dependent claim in combination with any other claim in the set of claims is included in the disclosure of possible implementations.

[0086] No element, act, or instruction used in this application should be construed as essential or required, unless expressly stated to be so. Additionally, as used in this application, the articles "a" and "an" are intended to include one or more items and can also be used to mean "one or more." Additionally, as used in this application, the term "collection" is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and can also be used to mean "one or more." Where only one item is intended, the term "a" or similar description is used. Additionally, as used in this application, the terms "has," "have," "having," and the like are intended to be open-ended terms. Additionally, the phrase "based on" is intended to mean "based at least in part on," unless expressly stated otherwise. [Explanation of symbols]

[0087] 100 Environment 110 User Devices 120 Platform 122 Cloud Computing Environment 124 Computational Resources 124-1 Application 124-2 Virtual Machine 124-3 Virtual Storage 124-4 Hypervisor 130 Network 200 devices 210 Bus 220 Processor 230 Memory 240 Memory Elements 250 input elements 260 Output Elements 270 Communication Interface 300 NBMP System 310 NBMP provider 320 NBMP Workflow Manager 330 Feature Repository 350 Media Processing Entities 352 Tasks 353 Settings 354 Media Processing Functions 360 Media Sources 370 Media Sources 380 Third Party Entities 391 Feature Detection API 392 NBMP Workflow API 393 Feature Detection API 394 NBMP Task API 396 Media Processing Entity API 400 Architecture 410 Media Sources 420 Flow Diverter 422 Input Buffer 424 Shunt Logic 426-1 First output buffer 426-2 Second output buffer 430 Transcoder 430-1 Transcoder 430-2 Transcoder 430-N Transcoder 440 Combiner 442-1 First input buffer 442-2 Second input buffer 444 Confluence Logic 446 Output Buffer 450 media outlets 491 Stream 492 Substream 493 Substream 494 Output 700 System 710 NBMP Client 720 NBMP Service 722 Feature Repository 724 Workflow Manager 726 Task Manager 730 Workflow 740 Web UI Backend 742 Web Dashboard 781 WDD 800 Computer Code 810 Acquisition Code 820 Creation Code 830 First control code 840 Second control code

Claims

1. 1. A method executed by at least one processor implementing a network based media processing (NBMP) workflow manager, comprising: a splitter task for splitting the compressed video stream into compressed sub-streams; a transcoder task for transcoding each of the compressed sub-streams to become a transcoded sub-stream; a merger task for merging the transcoded sub-streams into one transcoded sub-stream; creating an NBMP workflow including: controlling at least one media processing entity to execute said NBMP workflow; controlling the at least one media processing entity executing the NBMP workflow to report to another entity at least one of a diverter state of the diverter task, a transcoder state of at least one of the transcoder tasks, and a junction state of the junction task; Equipped with the compressed sub-stream includes a first segment and a second segment; the first segment and the second segment have metadata of a start time, a duration, and a byte length; The first segment and the second segment are output by a first output buffer and a second output buffer in the diverter task, respectively.

2. The method of claim 1 , wherein the at least one media processing entity is controlled to report the diverter status of the diverter task.

3. The method of claim 1 , wherein the at least one media processing entity is controlled to report the transcoder status of the at least one of the transcoder tasks.

4. The method of claim 1 , wherein the at least one media processing entity is controlled to report the junction state of the junction task.

5. 2. The method of claim 1, wherein the at least one media processing entity is controlled to report the diverter state of the diverter task, the transcoder state of the at least one of the transcoder tasks, and the junction state of the junction task.

6. 2. The method of claim 1 , wherein the step of controlling the at least one media processing entity to report comprises controlling the at least one media processing entity to report the diverter state of the diverter task, the transcoder state of at least one of the transcoder tasks, and the junction state of the junction task to a reporting server.

7. 2. The method of claim 1 , wherein the step of controlling the at least one media processing entity to report comprises controlling the at least one media processing entity to report at least one of the diverter state of the diverter task, the transcoder state of at least one of the transcoder tasks, and the junction state of the junction task based on information in a workflow description document (WDD) received by the NBMP workflow manager.

8. 2. The method of claim 1 , wherein controlling the at least one media processing entity to report comprises controlling the at least one media processing entity to report at least one of the diverter state of the diverter task, the transcoder state of at least one of the transcoder tasks, and the junction state of the junction task while a corresponding one of the diverter task, the transcoder task, and the junction task is executing.

9. 7. The method of claim 6, wherein the reporting server is configured to provide that the at least one of the diverter status of the diverter task, the transcoder status of at least one of the transcoder tasks, and the junction status of the junction task are visualized on a web dashboard.

10. The method of claim 1, wherein the media processing entity is configured to execute the NBMP workflow with configuration parameters including at least one indicator of a split number, which is the number to which the compressed video stream is split in the splitter task, and a merge number, which is the number to which the transcoded sub-streams are merged in the merger task.

11. The method of claim 1, comprising a step of controlling the at least one media processing entity to monitor a state of the media processing (NBMP) workflow manager, the diverter state, the transcoder state, and the merger state.

12. The step of controlling at least one media processing entity to perform a diverter task comprises the steps of: controlling at least one media processing entity to perform the diverter task by executing a diverter function of the NBMP standard; and controlling at least one media processing entity to perform the junction task by executing a junction function of the NBMP standard; controlling the at least one media processing entity to report comprises controlling the at least one media processing entity to report, using the metadata, a progress of at least one of the diverter status, the transcoder status, and the junction status in real time for visualization on a web dashboard. The method of claim 1.

13. at least one memory configured to store computer program code; at least one processor configured to access computer program code configured to cause a network based media processing (NBMP) workflow manager implemented by said at least one processor to perform the method of any one of claims 1 to 9 and to operate as instructed by said computer program code; A system comprising:

14. When executed by at least one processor, A computer program configured to cause said at least one processor to implement a network based media processing (NBMP) workflow manager for performing the method according to any one of claims 1 to 9.

Images