Method, system, and storage medium for parallel transcoding media content

Abstract

Systems and methods are provided for parallel transcoding of media content, in which the media content is split into multiple parallel media substreams and each media substream is transcoded using one transcoder, and then the multiple parallel media substreams are merged into a single transcoded stream. The parallel transcoding can be performed by one or more media processing entities as part of a workflow that includes a splitter task, a transcoder task, and a merger task, and the one or more media processing entities can be controlled to report the status of the tasks.

Description

[0001] Merge by reference

[0002] This application claims priority to U.S. Provisional Application No. 63 / 253,053, filed October 6, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] Embodiments of this disclosure relate to systems and methods for parallel transcoding of media content, wherein the media content is split into parallel media substreams, each media substream is transcoded using a transcoder, and the parallel media substreams are then merged into a single transcoded stream. Background Technology

[0004] Networks and cloud platforms are used to run various applications. The Network-Based Media Processing (NBMP) standard includes specifications for defining, instantiating, and running workflows on cloud platforms. The standard also defines splitter function templates and merger function templates, which use metadata to signal the boundaries of segments. Summary of the Invention

[0005] According to an embodiment, a cloud service is provided that runs multiple transcoders, which allows for increased transcoding speed. For example, the number of parallel substreams can be increased to improve transcoding speed.

[0006] According to an embodiment, a method is provided executed by at least one processor implementing a Network-Based Media Processing (NBMP) workflow manager. The method includes: creating an NBMP workflow comprising: a splitter task that splits a compressed video stream into multiple compressed substreams; multiple transcoder tasks that transcode the multiple compressed substreams into multiple transcoded substreams respectively; and a merger task that merges the multiple transcoded substreams into a single transcoded substream. The method further includes: controlling at least one media processing entity to execute the NBMP workflow; and controlling the at least one media processing entity executing the NBMP workflow to report at least one of the following to another entity: the splitter status of the splitter task, the transcoder status of at least one of the multiple transcoder tasks, and the merger status of the merger task.

[0007] According to one or more embodiments, at least one media processing entity is controlled to report the splitter status of the splitter task.

[0008] According to one or more embodiments, at least one media processing entity is controlled to report the transcoder status of at least one transcoder task among a plurality of transcoder tasks.

[0009] According to one or more embodiments, at least one media processing entity is controlled to report the merger status of the merger task.

[0010] According to one or more embodiments, at least one media processing entity is controlled to report the splitter status of a splitter task, the transcoder status of at least one transcoder task, and the merger status of a merger task.

[0011] According to one or more embodiments, controlling at least one media processing entity to perform reporting includes: controlling at least one media processing entity to report the splitter status of a splitter task, the transcoder status of at least one transcoder task among a plurality of transcoder tasks, and the merger status of a merger task.

[0012] According to one or more embodiments, controlling at least one media processing entity to perform reporting includes: based on information in a workflow description document (WDD) received by the NBMP workflow manager, controlling at least one media processing entity to report at least one of the following: the splitter status of a splitter task, the transcoder status of at least one transcoder task among a plurality of transcoder tasks, and the merger status of a merger task.

[0013] According to one or more embodiments, controlling at least one media processing entity to perform reporting includes: based on information in a workflow description document (WDD) received by the NBMP workflow manager, controlling at least one media processing entity to report at least one of the following: the splitter status of a splitter task, the transcoder status of at least one transcoder task among a plurality of transcoder tasks, and the merger status of a merger task.

[0014] According to one or more embodiments, controlling at least one media processing entity to perform reporting includes: controlling at least one media processing entity to report at least one of the following when one of a splitter task, a plurality of transcoder tasks, and a merger task is executed: the splitter status of the splitter task, the transcoder status of at least one of the plurality of transcoder tasks, and the merger status of the merger task.

[0015] According to one or more embodiments, the reporting server is configured to visualize at least one of the following in a network dashboard: the splitter status of a splitter task, the transcoder status of at least one transcoder task among a plurality of transcoder tasks, and the merger status of a merger task.

[0016] According to an embodiment, a system is provided. The system includes: at least one memory configured to store computer program code; and at least one processor configured to access the computer program code and operate as instructed by the computer program code. The computer program code includes creation code configured to cause a network-based media processing (NBMP) workflow manager implemented by the at least one processor to create an NBMP workflow, the NBMP workflow including: a splitter task that splits a compressed video stream into multiple compressed substreams; multiple transcoder tasks that transcode the multiple compressed substreams into multiple transcoded substreams respectively; and a merger task that merges the multiple transcoded substreams into a single transcoded substream. The computer program code further includes: first control code configured to cause the NBMP workflow manager to control at least one media processing entity to execute an NBMP workflow; and second control code configured to cause the NBMP workflow manager to control at least one media processing entity executing the NBMP workflow to report at least one of the following to another entity: the splitter status of a splitter task, the transcoder status of at least one transcoder task among a plurality of transcoder tasks, and the merger status of a merger task.

[0017] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control at least one media processing entity to report the splitter status of splitter tasks.

[0018] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control at least one media processing entity to report the transcoder status of at least one transcoder task among a plurality of transcoder tasks.

[0019] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control at least one media processing entity to report the merger status of the merger task.

[0020] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control at least one media processing entity to report the splitter status of a splitter task, the transcoder status of at least one transcoder task among a plurality of transcoder tasks, and the merger status of a merger task.

[0021] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control at least one media processing entity to report to a reporting server the splitter status of a splitter task, the transcoder status of at least one transcoder task among a plurality of transcoder tasks, and the merger status of a merger task.

[0022] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control at least one media processing entity to report at least one of the following based on information in the workflow description document (WDD) received by the NBMP workflow manager: the splitter status of a splitter task, the transcoder status of at least one transcoder task among a plurality of transcoder tasks, and the merger status of a merger task.

[0023] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control at least one media processing entity based on information in a workflow description document (WDD) received by the NBMP workflow manager to report at least one of the splitter status of a splitter task, the transcoder status of at least one transcoder task, and the merger status of a merger task.

[0024] According to one or more embodiments, the second control code is configured to cause the NBMP workflow manager to control at least one media processing entity to report at least one of the following when one of the splitter tasks, multiple transcoder tasks, and merger tasks is executed: the splitter status of the splitter task, the transcoder status of at least one of the multiple transcoder tasks, and the merger status of the merger task.

[0025] According to an embodiment, a non-transitory computer-readable medium storing computer code is provided. The computer code is configured, when executed by at least one processor, to cause the at least one processor to implement a Network-Based Media Processing (NBMP) workflow manager that creates an NBMP workflow, the NBMP workflow including: a splitter task that splits a compressed video stream into multiple compressed substreams; multiple transcoder tasks that transcode the multiple compressed substreams into multiple transcoded substreams respectively; and a merger task that merges the multiple transcoded substreams into a single transcoded substream. The computer code is further configured to cause the at least one processor to control at least one media processing entity to execute the NBMP workflow; and to control the at least one media processing entity executing the NBMP workflow to report at least one of the following to another entity: the splitter status of the splitter task, the transcoder status of at least one of the multiple transcoder tasks, and the merger status of the merger task. Attached Figure Description

[0026] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, in which:

[0027] Figure 1 This is a schematic diagram of an environment in which the methods, apparatus and systems described herein can be implemented, according to embodiments.

[0028] Figure 2 yes Figure 1 A block diagram of example components for one or more devices.

[0029] Figure 3 This is a block diagram of an NBMP system according to an embodiment.

[0030] Figure 4 This is a diagram of an example architecture for parallel transcoding using splitter and merger functions, according to an embodiment.

[0031] Figure 5 This is a diagram illustrating the functionality of an example splitter according to an embodiment.

[0032] Figure 6 This is a diagram illustrating the functionality of an example merger according to an embodiment.

[0033] Figure 7 This is a block diagram of the system according to an embodiment.

[0034] Figure 8 This is a block diagram of computer code according to an embodiment. Detailed Implementation

[0035] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0036] Figure 1 This is a schematic diagram of an environment 100 in which the methods, apparatus, and systems described herein can be implemented according to an embodiment. Figure 1 As shown, environment 100 may include user equipment 110, platform 120, and network 130. The devices in environment 100 can be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0037] User equipment 110 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to platform 120. For example, user equipment 110 may include computing devices (e.g., desktop computers, laptop computers, tablet computers, handheld computers, smart speakers, servers, etc.), mobile phones (e.g., smartphones, cordless phones, etc.), wearable devices (e.g., smart glasses or smartwatches), or similar devices. In some implementations, user equipment 110 may receive information from and / or send information to platform 120.

[0038] Platform 120 includes one or more devices as described elsewhere herein. In some implementations, platform 120 may include a cloud server or a group of cloud servers. In some implementations, platform 120 may be designed to be modular, allowing software components to be called in or out as needed. Therefore, platform 120 can be easily and / or quickly reconfigured for different purposes.

[0039] In some implementations, as shown in the figure, platform 120 may reside in cloud computing environment 122. It should be noted that although the implementations described herein depict platform 120 as residing in cloud computing environment 122, in some implementations, platform 120 may not be cloud-based (i.e., it may be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0040] The cloud computing environment 122 includes the environment of the hosting platform 120. The cloud computing environment 122 can provide computing, software, data access, storage, and other services without requiring end users (e.g., user equipment 110) to know the physical location and configuration of the systems and / or devices of the hosting platform 120. As shown in the figure, the cloud computing environment 122 may include a set of computing resources 124 (collectively referred to as "computing resources 124", and individually referred to as "computing resources 124").

[0041] Computing resource 124 includes one or more personal computers, workstations, server devices, or other types of computing and / or communication devices. In some implementations, computing resource 124 may host platform 120. Cloud resources may include computing instances running in computing resource 124, storage devices provided in computing resource 124, data transmission devices provided by computing resource 124, etc. In some implementations, computing resource 124 may communicate with other computing resources 124 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0042] like Figure 1 As further shown, computing resources 124 include a set of cloud resources, such as one or more applications (“APP”) 124-1, one or more virtual machines (“VM”) 124-2, virtualized storage (“VS”) 124-3, one or more hypervisors (“HYP”) 124-4, etc.

[0043] Application 124-1 includes one or more software applications that can be provided to or accessed by user device 110 and / or platform 120. Application 124-1 eliminates the need to install and run software applications on user device 110. For example, application 124-1 may include software associated with platform 120 and / or any other software that can be provided via cloud computing environment 122. In some implementations, an application 124-1 may send information to / receive information from one or more other applications 124-1 via virtual machine 124-2.

[0044] Virtual machine 124-2 includes a software implementation of a machine (e.g., a computer) that runs programs like a physical machine. Depending on the extent to which virtual machine 124-2 corresponds to any physical machine and its purpose, virtual machine 124-2 can be a system virtual machine or a process virtual machine. A system virtual machine can provide a complete system platform supporting the operation of a full operating system (“OS”). A process virtual machine can run a single program and can support a single process. In some implementations, virtual machine 124-2 can run on behalf of a user (e.g., user device 110) and can manage the infrastructure of cloud computing environment 122, such as data management, synchronization, or long-duration data transfer.

[0045] Virtualized storage 124-3 includes one or more storage systems and / or one or more devices that utilize virtualization technology within the storage system or device of computing resource 124. In some implementations, the type of virtualization, within the context of the storage system, may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or partitioning) of logical storage from physical storage, enabling access to the storage system regardless of physical storage or heterogeneous architecture. Partitioning allows storage system administrators flexibility in how they manage storage for end users. File virtualization eliminates the dependency between data accessed at the file level and the location where the files are physically stored. This enables optimization of storage usage, server consolidation, and / or non-disruptive file migration performance.

[0046] Hypervisor 124-4 provides hardware virtualization technology, which allows multiple operating systems (e.g., "guest operating systems") to run simultaneously on the host computer (e.g., computing resource 124). Hypervisor 124-4 can present a virtual operating platform to the guest operating systems and manage their operation. Multiple instances of various operating systems can share virtualized hardware resources.

[0047] Network 130 includes one or more wired and / or wireless networks. For example, network 130 may include cellular networks (such as fifth-generation (5G) networks, long-term evolution (LTE) networks, third-generation (3G) networks, code division multiple access (CDMA) networks, etc.), public land mobile networks (PLMN), local area networks (LAN), wide area networks (WAN), metropolitan area networks (MAN), telephone networks (such as public switched telephone networks (PSTN)), private networks, self-organizing networks, intranets, the Internet, fiber-optic networks, etc., and / or combinations of these networks or other types of networks.

[0048] Figure 1 The number and arrangement of devices and networks shown are provided as examples. In practice, additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or networks may exist. Figure 1 The devices and / or networks shown are arranged differently. Furthermore, Figure 1 The two or more devices shown can be implemented within a single device, or Figure 1 The single device shown can be implemented as multiple distributed devices. Alternatively, a group of devices in environment 100 (e.g., one or more devices) can perform one or more functions described as being performed by another group of devices in environment 100.

[0049] Figure 2 yes Figure 1 A block diagram of example components of one or more devices. Device 200 may correspond to user device 110 and / or platform 120. Figure 2 As shown, device 200 may include bus 210, processor 220, memory 230, storage component 240, input component 250, output component 260 and communication interface 270.

[0050] Bus 210 includes components that allow communication between components of device 200. Processor 220 is implemented in hardware, firmware, or a combination of hardware and software. Processor 220 is a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or another type of processing component. In some implementations, processor 220 includes one or more processors that can be programmed to perform functions. Memory 230 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by processor 220.

[0051] Storage component 240 stores information and / or software related to the operation and use of device 200. For example, storage component 240 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid-state disks), compact discs (CDs), digital versatile discs (DVDs), floppy disks, cassette disks, magnetic tapes, and / or other types of non-transitory computer-readable media, and corresponding drives.

[0052] Input component 250 includes components that allow device 200 to receive information, such as via user input (e.g., a touchscreen display, keyboard, keypad, mouse, buttons, switches, and / or microphone). Alternatively, input component 250 may include sensors for sensing information (e.g., a Global Positioning System (GPS) component, accelerometer, gyroscope, and / or actuator). Output component 260 includes components that provide output information from device 200 (e.g., a display, speaker, and / or one or more light-emitting diodes (LEDs)).

[0053] Communication interface 270 includes transceiver-like components (e.g., a transceiver and / or separate receiver and transmitter) that enable device 200 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 270 allows device 200 to receive information from another device and / or provide information to another device. For example, communication interface 270 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0054] Device 200 can perform one or more processes described herein. Device 200 can perform these processes in response to processor 220 executing software instructions stored in a non-transitory computer-readable medium such as memory 230 and / or storage component 240. Computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0055] Software instructions may be read into memory 230 and / or storage component 240 via communication interface 270 from another computer-readable medium or from another device. When executed, the software instructions stored in memory 230 and / or storage component 240 cause processor 220 to perform one or more processes described herein. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the implementations described herein are not limited to any particular combination of hardware circuitry and software.

[0056] Figure 2The number and arrangement of components shown are provided as an example. In practice, device 200 may include additional components, fewer components, different components, or components with... Figure 2 The components shown are arranged differently. Alternatively, a set of components of device 200 (e.g., one or more components) may perform one or more functions described as being performed by another set of components of device 200.

[0057] In embodiments of this disclosure, an NBMP system 300 is provided. (Refer to...) Figure 3 The NBMP system 300 includes an NBMP source 310, an NBMP workflow manager 320, a function library 330, one or more media processing entities 350, a media source 360, and a media receiver 370.

[0058] NBMP source 310 can receive instructions from third-party entity 380, communicate with NBMP workflow manager 320 via NBMP workflow API 392, and communicate with feature repository 330 via feature discovery API 391. For example, NBMP source 310 can send a workflow description document (WDD) to NBMP workflow manager 320 and can read feature descriptions of features stored in the memory of feature repository 330. These features are media processing functions stored in the memory of feature repository 330, such as media decoding, feature point extraction, camera parameter extraction, projection methods, seam information extraction, mixing, post-processing, and encoding functions. NBMP source 310 may include at least one processor and memory, or NBMP source 310 may be executed by at least one processor and memory, the memory storing code configured to cause at least one processor to execute the functions of NBMP source 310.

[0059] NBMP source 310 can request NBMP workflow manager 320 to create a workflow by sending a workflow descriptor document. This workflow includes tasks 352 that will be performed by one or more media processing entities 350. The workflow descriptor document can include multiple descriptors, and each descriptor can include multiple parameters.

[0060] For example, NBMP source 310 may select functions stored in function store 330 and send a workflow descriptor document to NBMP workflow manager 320. This workflow descriptor document includes descriptors describing details such as input and output data, required functions, and workflow requirements. The workflow descriptor document may further include a set of task descriptions and connection maps of the inputs and outputs of task 352 to be executed by one or more media processing entities 350. When NBMP workflow manager 320 receives such information from NBMP source 310, it can instantiate tasks based on function names and create a workflow by connecting tasks according to the connection maps.

[0061] Alternatively or additionally, NBMP source 310 may request NBMP workflow manager 320 to create a workflow using a set of keywords. For example, NBMP source 310 may send a workflow descriptor document to NBMP workflow manager 320 that may include the set of keywords, which NBMP workflow manager 320 can use to locate functions stored in function store 330. When NBMP workflow manager 320 receives such information from NBMP source 310, NBMP workflow manager 320 can create a workflow by searching for appropriate functions using keywords (which may be specified in the processing descriptor of the workflow descriptor document) and by providing and connecting tasks using other descriptors in the workflow descriptor document.

[0062] The NBMP workflow manager 320 can communicate with the function store 330 via the function discovery API 393, which may be the same as or different from the function discovery API 391, and can communicate with one or more media processing entities 350 via the NBMP task API 394. The NBMP workflow manager 320 can also communicate with one or more media processing entities 350 via the media processing entity (MPE) API 396. The NBMP workflow manager 320 may include, or may be implemented by, at least one processor and memory, the memory storing code configured to cause the at least one processor to perform the functions of the NBMP workflow manager 320.

[0063] The NBMP workflow manager 320 can use the NBMP Task API 394 to set up, configure, manage, and monitor one or more tasks 352 of a workflow that can be executed by one or more media processing entities 350. In an embodiment, the NBMP workflow manager 320 can use the NBMP Task API 394 to update and revoke tasks 352. To configure, manage, and monitor tasks 352 of a workflow, the NBMP workflow manager 320 can send messages such as requests to one or more media processing entities 350, where each message may have multiple descriptors, each descriptor having multiple parameters. Each task 352 may each include a media processing function 354 and a configuration 353 for said media processing function 354.

[0064] In an embodiment, after receiving a workflow descriptor document from NBMP source 310 that does not include a task list (e.g., includes a keyword list instead of a task list), NBMP workflow manager 320 can select tasks based on task descriptions in the workflow descriptor document to search the function store 330 via the function discovery API 393 to find appropriate functions to run as task 352 in the current workflow. For example, NBMP workflow manager 320 can select tasks based on keywords provided in the workflow descriptor document. After identifying appropriate functions using the task description set or keywords provided by NBMP source 310, NBMP workflow manager 320 can configure the selected task in the workflow via NBMP Task API 394. For example, NBMP workflow manager 320 can extract configuration data from information received from the NBMP source and configure task 352 based on the configuration data.

[0065] One or more media processing entities 350 may be configured to: receive media content from a media source 360, process the received media content according to a workflow including task 352 and created by the NBMP workflow manager 320, and output the processed media content to a media receiver 370. Each of the one or more media processing entities 350 may each include at least one processor and a memory storing code, or may be implemented by at least one processor and a memory storing code configured to cause the at least one processor to perform the functions of the media processing entity 350.

[0066] Media source 360 ​​may include a memory for storing media and may be combined with or separated from NBMP source 310. In an embodiment, when preparing a workflow, NBMP workflow manager 320 may notify NBMP source 310 and media source 360, and media source 360 ​​may send media content to one or more media processing entities 350 based on the notification that the workflow is ready.

[0067] The media receiver 370 may include at least one processor and at least one display, or be implemented by at least one processor and at least one display, the at least one display being configured to display media processed by one or more media processing entities 350.

[0068] The third-party entity 380 may include at least one processor and a memory storing code, or may be implemented by at least one processor and a memory storing code configured to cause the at least one processor to perform the functions of the third-party entity 380.

[0069] As described above, messages from NBMP source 310 (e.g., a workflow descriptor document for requesting workflow creation) to NBMP workflow manager 320, and messages from NBMP workflow manager 320 to one or more media processing entities 350 (e.g., for causing the workflow to be executed), can include multiple descriptors, each with multiple parameters. In this case, communication between any components of NBMP system 300 using the API can include multiple descriptors, each with multiple parameters.

[0070] According to an embodiment, a cloud service is provided that runs multiple transcoders, which allows for increased transcoding speed. For example, the number of parallel substreams can be increased to improve transcoding speed.

[0071] According to the embodiments, it is possible to achieve Figure 4 The architecture 400 shown is an architecture for parallel transcoding using splitter and merger functions.

[0072] Reference Figure 4 The splitter 420 splits a single stream 491 from the media source 410 into multiple sub-streams 492. Each sub-stream 492 is then transcoded by a corresponding transcoder (e.g., transcoders 430-1, 430-2, ... 430-N), and the transcoded sub-streams 493 from the transcoders are sent to the merger 440. The merger 440 merges the transcoded sub-streams 493 and generates an output 494, which is then sent to the media receiver 450. The output 494 may include a single transcoded stream comprising the transcoded sub-streams 493. According to an embodiment, the media source 410 may correspond to... Figure 3 The media source 360 ​​and the media receiver 450 can correspond to Figure 3 The media receiver 370 is located in the middle. According to an embodiment, the splitter 420, transcoders (e.g., transcoders 430-1, 430-2, ... 430-N), and merger 440 can be composed of... Figure 3 One or more media processing entities 350 perform functions as part of one or more tasks 352 (e.g., functions performed by the media processing entity 350 as part of one or more tasks 352). Figure 3Media processing function 354 in the middle.

[0073] refer to Figures 5-6 Exemplary embodiments of the splitter 420 and the merger 440 are described. For example, as Figures 5-6 As shown, for example, splitter 420 may be a splitter function with a 1:2 split ratio, and merger 440 may be a merger function with a 2:1 merge ratio. However, the split and merge ratios of splitter 420 and merger 440 are not limited thereto. According to embodiments, media processing entity 350 performing splitter and merger functions may accept multiple splits and / or merges as configuration parameters defined in the standard. For example, at least one indicator may be included in the corresponding configuration 353 of task 352 and may indicate the number of splits and / or merges to be performed by the splitter and / or merger functions included in task 352.

[0074] refer to Figures 5-6 Both the splitter and merger functions use input and output buffers that can be configured as part of the function template. The input and output of the splitter and merger functions can be index streams providing timing metadata. Each segment of the index stream can have metadata such as start time, duration, and byte length in the associated timing metadata stream. Corresponding configuration parameters can be segment duration, segment metadata, and repeat headers.

[0075] According to the embodiments, refer to Figure 5 The splitter 420 may include an input buffer 422, splitter logic 424, and multiple output buffers (e.g., a first output buffer 426-1 and a second output buffer 426-2). The input buffer 422 may be configured to receive a compressed video stream that is indexed and provided with timing metadata. As an example, the compressed video stream may include compressed segments s4, s3, s2, and s1. The splitter logic 424 may be configured to split the compressed video stream into compressed substreams and provide each substream to an output buffer. For example, the splitter logic 424 may provide a compressed substream including compressed segments s3 and s1 to the first output buffer 426-1 and a compressed substream including compressed segments s4 and s2 to the second output buffer 426-2. The output buffers may be configured to output the compressed substreams received by the output buffers. According to an embodiment, the splitter 420 may output the compressed substream including compressed segments s3 and s1 to a first transcoder (e.g., Figure 4 The transcoder 430-1), and outputs the compressed substream including the compressed segments s4 and s2 to the second transcoder (e.g., Figure 4 The transcoder 430-2), the first transcoder and the second transcoder can transcode the received compressed substream and output the transcoded substream to the merger 440.

[0076] Reference Figure 6 The merger 440 may include multiple input buffers (e.g., a first input buffer 442-1 and a second input buffer 442-2), merger logic 444, and an output buffer 446. The input buffers may be configured to receive transcoded substreams, respectively. For example, the first input buffer 442-1 may receive a transcoded substream including transcoded segments s3* and s1*, and the second input buffer 442-2 may receive a transcoded substream including transcoded segments s4* and s2*. Both the first input buffer 442-1 and the second input buffer 442-2 may be configured to output the received transcoded substreams to the merger logic 444. The merger logic 444 may be configured to merge the transcoded substreams into a single transcoded video stream including transcoded segments (e.g., transcoded segments s4*, s3*, s2*, and s1*), and send this single transcoded video stream to the output buffer 446. Output buffer 446 can be configured to output a single transcoded video stream. According to an embodiment, merger 440 can output a single transcoded video stream to media receiver 450. The single transcoded video stream can be indexed and provided with timing metadata.

[0077] According to an embodiment, the splitter 420, transcoder, and merger 440 can report their operations to a reporting server. For example, a media processing entity (or entities) implementing the splitter 420, transcoder, and / or merger 440 can send information to another component (e.g., a reporting server) indicating the splitter status, transcoder status, and / or merger status of the functions performed by the media processing entity.

[0078] Reference Figure 7 The exemplary operations performed by embodiments of this disclosure are described below. Figure 7 In the diagram, solid lines with arrows represent NBMP API operations, while dashed lines with arrows represent media processing. Figure 7 A system 700 including NBMP management components is illustrated. According to an embodiment, system 700 may include an NBMP client 710, an NBMP service 720, a media source 410, a media receiver 450, and a reporting server that implements a web user interface (web UI) backend 740 and a web dashboard 742. According to an embodiment, the reporting server may be implemented by at least one processor and a memory storing computer code configured to cause the at least one processor to perform the functions of the reporting server. According to an embodiment, the at least one processor and / or memory may be the same as or different from at least one processor and / or memory that enables other components implementing embodiments of this disclosure.

[0079] According to an embodiment, the NBMP client 710 can be... Figure 3 The NBMP source 310 is implemented. The NBMP service 720 may include a feature repository 722, a workflow manager 724, and a task manager 726. According to an embodiment, it can be implemented by... Figure 3 Function repository 330 implements function repository 722, and can be provided by Figure 3 The NBMP workflow manager 320 implements workflow manager 724 and task manager 726. According to an embodiment, workflow 730 may include splitter 420, transcoder 430, and merger 440, which may be included in a process... Figure 3 The task 352 is implemented by one or more media processing entities 350.

[0080] According to an embodiment, system 700 may execute a method comprising (1) creating a workflow, (2) obtaining available functions, (3) creating splitter tasks, transcoder tasks and merger tasks, (4) running the workflow, (5) streaming media to the workflow, and (6) visualizing the workflow and task status.

[0081] For example, NBMP client 710 can create a WDD 781 describing workflow 730 and send WDD 781 to workflow manager 724, and workflow manager 724 can create workflow 730. According to an embodiment, it can be referred to as previously... Figure 3 Workflow 730 is created in the manner described. Upon receiving WDD 781, workflow manager 724 can obtain (782) a feature template from feature repository 722 to create workflow 730. According to an embodiment, workflow manager 724 (e.g., NBMP workflow manager 320) can be as previously referenced. Figure 3Interacting with the feature repository 722 (e.g., feature repository 330) as described. Based on WDD 781 and feature templates, workflow manager 724 can create tasks for workflow 730 to be executed by one or more media processing entities (e.g., media processing entity 350) and sort the tasks of workflow 730 (783). Tasks may include media processing functions of splitter 420, transcoder 430, and merger 440. After workflow manager 724 creates tasks for workflow 730 and sorts the tasks of workflow 730, task manager 726 can control (784) one or more media processing entities to run the tasks of workflow 730. Tasks can be run by media source 410 streaming (785) media to the media processing entity implementing splitter 420; processing the media by one or more media processing entities implementing splitter 420, transcoder 430, and merger 440; and outputting the processed media content (791) to media receiver 450 by the media processing entity implementing merger 440. According to an embodiment, the media streamed by media source 410 may be an indexed h265 stream. This can be seen from the above description. Figure 4-6 Use the descriptions to understand the functions of the splitter 420, transcoder 430, and merger 440.

[0082] According to an embodiment, workflow manager 724 may report (786) workflow status to the web UI backend 740 of the reporting server; task manager 726 may report (787) task status to the web UI backend 740 of the reporting server; splitter 420 may report (788) splitter status to the web UI backend 740 of the reporting server; multiple transcoders 430 may report (789) their respective transcoder status to the web UI backend 740 of the reporting server; and merger 440 may report (790) merger status to the web UI backend 740 of the reporting server. Status reporting may include sending first information, which includes indicators indicating the respective statuses. According to an embodiment, WDD 781 may include second information indicating where one or more of workflow manager 724, task manager 726, splitter 420, transcoder 430, and merger 440 should report their respective statuses, and the second information may further indicate what should be reported. According to an embodiment, the workflow manager 724 and / or task manager 726 may report their respective statuses based on the second information, and may control one or more media processing entities to report the statuses of the splitter 420, transcoder 430, and / or merger 440 implemented by one or more media processing entities based on the second information.

[0083] According to an embodiment, the reporting server can visualize data on the network dashboard 742 based on first information received by the reporting server (792). For example, the visualized data may include workflow status, task status, workflow graphs, and media receiver video players. According to an embodiment, splitter status, transcoder status, and merger status can also be visualized. The reporting server can be configured to visualize the data by displaying the network dashboard 742 on at least one display.

[0084] According to embodiments, systems and methods can be provided for parallel transcoding of media streams using two or more transcoders, increasing the effective speed of transcoding. These systems and methods implement the NBMP splitter and merger functionality of the NBMP standard, allowing for configuration with the ability to utilize multiple split / merge operations and timed metadata. The systems and methods can use the NBMP standard to manage the instantiation, deployment, management, and monitoring of workflows, where workflow tasks and the NBMP workflow manager report progress to a web-based dashboard in real time.

[0085] According to embodiments of this disclosure, at least one processor and a memory storing computer code may be provided. The computer code may be configured to execute any number of aspects of this disclosure when executed by the at least one processor.

[0086] For example, refer to Figure 8 The computer code 800 can be implemented in the NBMP system 300. For example, the computer code can be stored in the memory of the NBMP workflow manager 320 and can be executed by at least one processor of the NBMP workflow manager 320. The computer code may include, for example, acquisition code 810, creation code 820, first control code 830, and second control code 840.

[0087] According to embodiments of this disclosure, the acquisition code 810 can be configured to cause the NBMP workflow manager 320 to acquire information from the WDD. For example, the NBMP workflow manager 320 can receive the WDD, and parameters in the WDD can be signaled to the NBMP workflow manager 320, enabling the NBMP workflow manager 320 to acquire the corresponding information.

[0088] According to embodiments of this disclosure, creation code 820 can be configured to cause NBMP workflow manager 320 to create a media processing workflow including task 352. For example, task 352 may include the functionality of splitter 420, transcoder 430, and merger 440. According to embodiments, the media processing workflow can be created based on information obtained from WDD.

[0089] According to embodiments of this disclosure, the first control code 830 may be configured to cause the NBMP workflow manager 320 to control at least one media processing entity 350 to perform a media processing workflow.

[0090] According to embodiments of this disclosure, the second control code 840 can be configured to cause the NBMP workflow manager 320 to control at least one media processing entity 350 executing the media processing workflow to report at least one of a splitter status, a transcoder status, and a merger status. For example, the media processing entity 350 implementing the splitter 420 can be controlled to report the splitter status, the media processing entity 350 implementing the transcoder 430 can be controlled to report the transcoder status, and the media processing entity 350 implementing the merger 440 can be controlled to report the merger status. According to embodiments, the NBMP workflow manager 320 can perform control based on information obtained from the WDD. For example, the NBMP workflow manager 320 can control what information to report and where to report it based on information obtained from the WDD. According to embodiments, the NBMP workflow manager 320 can control the media processing entity 350 to report its status to a reporting server.

[0091] According to one or more embodiments, embodiments of this disclosure may be implemented in environments different from NBMP.

[0092] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit implementations to the precise forms disclosed. Modifications and variations are possible based on the foregoing disclosure, or may be derived from the practice of implementation.

[0093] As used herein, the term “component” is intended to be interpreted broadly as hardware, firmware, or a combination of hardware and software.

[0094] Even if combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically recited in the claims and / or not disclosed in the specification. While each dependent claim listed below may be directly subordinated to only one claim, the disclosure of possible implementations includes combinations of each dependent claim with each other claim in the claim set.

[0095] Unless explicitly stated otherwise, no element, action, or instruction used herein shall be construed as critical or essential. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Additionally, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with “one or more.” The term “one” or similar language is used where only one item is intended. Furthermore, as used herein, the terms “has,” “have,” “having,” etc., are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “at least partially based on,” unless otherwise explicitly stated.

Claims

1. A method for parallel transcoding of media content, said method being executed by at least one processor implementing an NBMP workflow manager, characterized in that, The method includes: Create an NBMP workflow that includes the following tasks: A splitter task that splits a compressed video stream into multiple compressed sub-streams, each compressed sub-stream including a first segment and a second segment, the first segment and the second segment having metadata of start time, duration and byte length; Multiple transcoder tasks, wherein the multiple transcoder tasks transcode the multiple compressed substreams into multiple transcoded substreams respectively, and A merger task that merges the plurality of transcoded substreams into a single transcoded substream; Control at least one media processing entity to execute the NBMP workflow; and Based on information in the workflow description document (WDD) received by the NBMP workflow manager, the at least one media processing entity controlling the execution of the NBMP workflow reports at least one of the following to another entity: the splitter status of the splitter task, the transcoder status of at least one of the plurality of transcoder tasks, and the merger status of the merger task.

2. The method according to claim 1, characterized in that, The control of the at least one media processing entity to perform the report includes: controlling the at least one media processing entity to report to a reporting server the splitter status of the splitter task, the transcoder status of at least one transcoder task among the plurality of transcoder tasks, and the merger status of the merger task.

3. The method according to claim 1, characterized in that, The control of the at least one media processing entity to perform the reporting includes: controlling the at least one media processing entity to report at least one of the following when one of the splitter task, the plurality of transcoder tasks, and the merger task is executed: the splitter status of the splitter task, the transcoder status of the at least one of the plurality of transcoder tasks, and the merger status of the merger task.

4. The method according to claim 2, characterized in that, The reporting server is configured to visualize at least one of the following in the network dashboard: the splitter status of the splitter task, the transcoder status of at least one of the plurality of transcoder tasks, and the merger status of the merger task.

5. A system for parallel transcoding of media content, characterized in that, The system includes: At least one memory configured to store computer program code; and At least one processor is configured to access the computer program code and perform the method of any one of claims 1-4 in accordance with the operations indicated by the computer program code to a network-based media processing NBMP workflow manager implemented by the at least one processor.

6. A non-transitory computer-readable storage medium for storing computer code, characterized in that, The non-transitory computer-readable storage medium is configured such that, when the computer code is executed by at least one processor, the at least one processor implements a network-based media processing NBMP workflow manager, the network-based media processing NBMP workflow manager performing the method of any one of claims 1-4.

Images