Scalable system and method using logical entities for creating programs using multimedia signals - Patents.com

The system addresses the challenge of flexible and scalable live streaming by using logical entities on a computing system to process multimedia signals, enabling automated routing and monitoring across different transport formats, thus enhancing the efficiency of multimedia program creation and distribution.

JP7594695B2Active Publication Date: 2024-12-04LTN GLOBAL COMMUNICATIONS INC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023581061
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-08-16
Publication Date
2024-12-04
Estimated Expiration
2041-08-16

AI Technical Summary

Technical Problem

Existing systems for creating multimedia programs face challenges in supporting flexible and scalable live streaming in the cloud, requiring sophisticated production processes that need to handle various transport formats while ensuring automated routing, monitoring, and alerting without altering the platform structure.

Method used

A scalable system and method using logical entities on a computing system to process multimedia signals, where servers with processors define input and output logical entities based on user-defined predicates and routing rules, supporting flexible routing and monitoring across different transport protocols.

Benefits of technology

Enables flexible and scalable multimedia program creation and distribution by automating routing and monitoring, accommodating various transport formats and ensuring efficient signal processing and distribution without altering the platform's physical structure.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007594695000002
    Figure 0007594695000002
  • Figure 0007594695000003
    Figure 0007594695000003
  • Figure 0007594695000004
    Figure 0007594695000004
Patent Text Reader

Abstract

The objective is to provide a flexible system and method. According to the invention, a program is created in a computing system using signals, such as multimedia signals, received from one or more sources over communication channels by executing application software on one or more servers of the computing system. At least one of the servers has one or more processors for processing defined logical entities. Signals used in creating the program are received from the sources over communication channels at the server. An input logical entity is defined having attributes associated with the sources of the signals such that the input logical entity responds to user-defined predicates containing logical expressions for accepting or rejecting the signals. Also, a stream logical entity is defined that identifies the accepted signals and a routing logical entity that establishes connections between the stream logical entity and a destination. The accepted signals are routed to the destination based on routing rules.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates generally to systems and methods for creating programs that require filtering and routing of multimedia (eg, audiovisual signals) used in the programs. [Background technology]

[0002] In creating a program, streams originating from multiple sources are enriched and ultimately presented to consumers. As used herein, a program can be any one or combination of streamed content available for consumption by any audience, user, viewer, subscriber, etc. Although streaming multimedia content includes both video and audio data, such streaming is often referred to as video streaming or streaming, with the understanding that a stream conveys both video and audio data. This specification refers to multimedia streaming, video streaming, and streaming interchangeably.

[0003] Systems used for live production, post-processing, and delivery of programs by streaming are known. Also known are streaming services used to deliver video / audio data to consumers over the Internet (or other networks). Typically, these services use a media server that divides a program into multiple segments, encodes each segment, and transports the encoded segments to a client device via a wide variety of transport protocols. The client device receives the transported segments, decodes the segments, and presents the decoded segments in an appropriate format to a client, including a viewer of the multimedia stream. Video and audio codec formats represent the technology used to both generate and play digital video and audio. Each format specifies how the video and audio are combined. Video transcoding, sometimes called video encoding, is the conversion from one digital encoding format to another, for example for movie data files. Video transcoding involves the translation of the three elements of digital video simultaneously: file format, video, and audio. Video engines are the underlying code used to support high definition (HD) recording, playback, and editing.

[0004] Video on demand (VOD) is a known distribution service adopted by companies such as Netflix and Apple, which are owners or publishers of streams sent to subscribers. VOD allows such subscribers to access multimedia streams without the constraints of a fixed broadcast schedule. Also known is live streaming of public or private programs, where content originating at a venue, such as a stadium, concert hall, or TV studio, is transmitted to viewers in real time using the Internet. Live streaming deployments using Internet Protocol (IP) are known to route packetized live streams for production and then distribute the produced streams to viewers. Such production may include the insertion of information used to enrich, modify, alter, or limit the stream type, for example, by inserting advertisements, subtitles, etc.

[0005] Live streaming can be accomplished over the Internet using systems and methods disclosed, for example, in U.S. Patent No. 8,599,851 to Amir et al., entitled "System and method that routes flows via multicast flow transport for groups," U.S. Patent No. 8,437,267 to Amir et al., entitled "System and method for recovery of packets in overlay networks," U.S. Patent No. 8,619,775 to Amir et al., entitled "Scalable flow transport and delivery network and associated methods and systems," U.S. Patent No. 9,106,569 to Amir et al., entitled "System and method that routes flows via multicast flow transport for groups," and U.S. Patent No. 8,181,210 to Amir et al., entitled "Method for delivery of deadline-driven content flows over a flow transport system that interfaces with a flow delivery system via a selected gateway," all of which are owned by LTN Global communications, Inc. (Columbia, Maryland), the assignee of the present application, and are incorporated herein by reference.

[0006] When packets are transmitted between two nodes, they encounter many transport impairments such as bandwidth fluctuations and packet loss that degrade the streaming quality. To improve the quality, various transport protocols have been developed. For example, SRT is a transport protocol that delivers high-quality streams with low latency over noisy networks such as the Internet. This protocol optimizes the transport of streams over unpredictable networks by adapting to real-time network conditions, minimizing packet loss and generating a better viewing experience.

[0007] Hypertext Transfer Protocol (HTTP) Live Streaming (HLS) is an HTTP-based adaptive bitrate streaming communication protocol developed by Apple for implementation using a web browser running on a client device. When a client requests a stream, for example by clicking a hyperlink, the browser sends a request message for the requested stream to the HTTP server. The server receives the request and sends a response message with the stream to the browser. However, the HTTP server does not maintain any information about the client, so if the client asks for the same stream again, the server resends the stream. For this reason, HTTP is called a stateless protocol. HTTP can use both non-persistent or volatile connections and persistent connections. A volatile connection is one that the HTTP server closes after sending the requested stream to the client. In other words, the connection is used for one request and one response. In a persistent connection, the HTTP server keeps the connection open after sending the response so that subsequent requests and responses can be exchanged between the same client and server. The server closes the connection only if it is not used for a certain configurable amount of time. HLS works by splitting the entire stream into a sequence of small HTTP-based file downloads, each containing one short chunk of the potentially unlimited stream as a whole.

[0008] The Real-Time Messaging Protocol (RTMP) is a TCP-based protocol that maintains a persistent connection and allows low-latency communication. To deliver the stream smoothly and transmit as much information as possible, RTMP divides the stream into multiple fragments, the size of which is dynamically negotiated between the client and the server. RTMP defines several independent virtual channels that are used to send and receive the packet stream.

[0009] The Real-time Transport Protocol (RTP) is a network protocol for delivering streams over IP networks. RTP is used in communication and entertainment systems, including streaming and video teleconferencing. RTP typically runs over the User Datagram Protocol (UDP). RTP is used in conjunction with the RTP Control Protocol (RTCP). While RTP is sending a stream, RTCP is used to monitor transmission statistics and quality of service (QoS) and to assist in synchronizing multiple streams. The Real Time Streaming Protocol (RTSP) is a network control protocol designed for use in entertainment and communication systems to control media servers. This protocol is used to establish and control media sessions between endpoints. The transmission of streaming video / audio data itself is not a task performed by RTSP. Most RTSP servers use RTP with RTCP for streaming. One use of RTP is Web Real-Time Communication (WebRTC), a free and open source project that provides real-time communication (RTC) to web browsers and mobile applications through an application programming interface (API). It enables streaming within web pages using direct peer-to-peer communication, thus eliminating the need to install plugins or download native applications. Stream tags used in WebRTC are generated out-of-band by a system component and flow downstream in parallel with the stream. Stream tags can be forcefully stopped from propagating by another system component.

[0010] The Moving Picture Experts Group (MPEG) Transport Stream (also known as Transport Stream, MPEG-TS, MTS, or TS) is a standard digital container format for the transmission and storage of audio, video, and Program and System Information Protocol (PSIP) data. TS streams, which contain Program Specific Information (PSI), can be transported over TCP / IP. PSI is a set of tables that carry information needed to demultiplex and present a program, such as the Program Mapping Table (PMT).

[0011] Dynamic Adaptive streaming over HTTP (DASH), also known as MPEG-DASH, is an adaptive bitrate streaming technique that enables high quality streaming over the Internet. MPEG-DASH can be delivered from conventional HTTP web servers. Like HLS, MPEG-DASH works by splitting content into a sequence of small HTTP-based file segments. Each segment contains a short playback time interval of content that may last several hours, such as a movie or a live broadcast of a sporting event. The content is made available at a variety of different bit rates, and alternative segments are encoded at different bit rates to cover aligned short playback time intervals. MPEG-DASH allows devices such as Internet-connected televisions, TV set-top boxes, desktop computers, smartphones, and tablets to consume Internet-delivered multimedia content, thus adapting to a variety of Internet reception conditions.

[0012] FIG. 1 shows an example of content distribution in a system that receives streams, such as restricted TS streams for programs, or stream tags via unrestricted WebRTC streams, or streams from external devices. These streams are ingested by a media server known as an Ingest server. The ingested streams can be sent to an output device, a recorder device, or a multi-view device. Video production requires various systems, platforms, hardware, software, networks, facilities, etc. to develop, test, distribute, monitor, control, or support the streaming. Since video production is performed in an IT infrastructure, to "ingest" video streams, users move them from multiple venues into a production platform that connects with one or more Ingest servers.

[0013] As production processes become more sophisticated, flexible systems and methods are needed to support live streams in the cloud and allow for expansion without changing the platform structure. Such a flexible system must support a variety of transport formats and also support automated routing, monitoring, and alerting. Summary of the Invention

[0014] Briefly, in accordance with the present invention, a program is created in a computing system using signals, such as multimedia signals, received over communication channels from one or more sources by executing application software on one or more servers of the computing system. At least one of the servers has one or more processors for processing defined logical entities. Signals used in creating the program are received from the sources over communication channels at the server. An input logical entity is defined having attributes associated with the sources of the signals such that the input logical entity responds to user-defined predicates containing logical expressions for accepting or rejecting the signals. Also, a stream logical entity is defined that identifies the accepted signals and a routing logical entity that establishes connections between the stream logical entity and a destination. The accepted signals are routed to the destination based on routing rules.

[0015] In accordance with some of the more detailed features of the invention, an output logical entity is defined that associates stream logical entities with destinations, the output logical entity being adapted to transform the stream logical entities according to user-defined program transformation rules that are evaluated to map the received signal to the destination. For example, the signal may be an MPEG signal, and the user-defined program transformation rules are based on Program Specific Information (PSI). In one embodiment, the stream logical entity is associated with a Packetized Elementary Stream (PES), and the PSI includes a Program Mapping Table (PMT) associated with the stream logical entity. In another embodiment, the inter arrival times (IAT) of packets of the MPEG signal are recorded at the server and used by the output logical entity to achieve a constant bit rate.

[0016] According to other more detailed features of the invention, the routing rules are user-defined logical rules associated with the input logical entities such that the user-defined logical rules are evaluated when a signal is presented to the input logical entities. In one embodiment, the user-defined predicates are based on time, geographical region, location, or IP address. In another embodiment, the input logical entities are defined by a user as persistent logical entities such that the input logical entities are configured by the user via a graphical user interface or an application programming interface (API). In yet another embodiment, the stream logical entities are associated with a unique identification (ID) of the owner or publisher of the signal and / or a session tag that creates the local connection.

[0017] In accordance with yet another detailed feature of the invention, features of the signal are recognized based on the confidence level, and the recognized features are used as metadata associated with the signal. [Brief description of the drawings]

[0018] [Figure 1] 1 illustrates an embodiment of content distribution in a system receiving a stream. [Diagram 2] An embodiment of the present invention is shown in a Live Video Cloud (LVC) platform used in streaming program production, post-production, and distribution. [Diagram 3] 1 shows a diagram layering the physical and logical components of the LVC platform. [Figure 4] 1 illustrates a functional block diagram of input attributes when a device provides a video signal to an ingest load balancer coupled to multiple ingest servers. [Diagram 5] 1 illustrates a process for generating input in an LVC platform according to the present invention and illustrates a portal for generating input via a user interface. [Figure 6]1 illustrates a process for creating an entry in the LVC platform in accordance with the present invention, showing the selection of the "General" tab, which allows an operator to enter general information that will allow the entry to be found within the LVC platform. [Figure 7] 1 illustrates a process for generating input in an LVC platform in accordance with the present invention, showing the selection of the "Settings" tab, which includes settings specific to SRT Push. [Figure 8] 1 illustrates a process for generating input in an LVC platform according to the present invention, showing a data center selected from a list for which a URL is provided. [Figure 9] 1 illustrates a process for generating input in an LVC platform in accordance with the present invention, and shows other configurations such as latency, which is set to 50 ms in this example. [Figure 10] 1 illustrates a process for generating input in an LVC platform according to the present invention, and illustrates predicates. [Figure 11] 1 illustrates a process for generating input in an LVC platform in accordance with the present invention, showing the generation of a first predicate that accepts or approves only streams from a given IP address, and a second predicate that requires the stream to be accepted or approved between the hours of 8:00 PM and 10:00 PM. [Figure 12] 1 illustrates the process for generating an input in the LVC platform according to the present invention, showing that when an operator clicks the "Generate" button, the LVC requests the associated API to generate a record of the generated input in the DBMS. [Figure 13] 1 illustrates a process for generating output in an LVC platform according to the present invention and illustrates a portal for generating output via a user interface. [Figure 14] 1 illustrates a process for generating an output in the LVC platform in accordance with the present invention, showing the selection of the "General" tab, which allows an operator to enter general information that will allow the output to be found within the LVC platform. [Figure 15] 1 illustrates a process for generating output in an LVC platform in accordance with the present invention, showing selection of the "Settings" tab where a user can set delay, frame rate resolution, and interlace mode. [Figure 16] 1 illustrates a process for generating output in an LVC platform according to the present invention, showing the selection of the "Program Conversion" tab where a user can set conversion parameters. [Figure 17] 1 illustrates a process for generating output in an LVC platform in accordance with the present invention, showing selection of the "Destination" tab where a user can select destination parameters. [Figure 18] 1 illustrates a process for generating output in an LVC platform in accordance with the present invention, showing selection of the "Destination" tab where a user can select destination parameters. [Figure 19] These are the attributes that define a stream. [Figure 20] 1 shows an example of a routing rule. [Figure 21] Indicates the routing for connecting a stream's source to its target. [Figure 22] 1 shows an example of a flow diagram between entities of the LVC platform. [Figure 23] 1 shows a flowchart illustrating the implementation of one embodiment of the present invention in a method implemented by logic in an LVC platform. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Below is a glossary of terms used to describe the present invention.

[0020] [Table 1]

[0021] FIG. 2 illustrates an embodiment of the present invention in a Live Video Cloud (LVC) platform used in the production, post-processing, and delivery of programs by streaming. LVC uses servers that contain processors that process logical entities representing various components used in program production. These logical entities allow LVC to be scalable, adjusting processing power to suit various production needs based on user-defined parameters, as described further below. The LVC platform is managed by a system administrator, who controls the database and back-end and front-end servers that form a database management system (DBMS) over a cloud computing network, including the Internet. The LVC platform supports a programmable streaming pipeline that allows flexible routing choices based on user-defined parameters without changing the physical structure of the platform. In this way, the described LVC platform provides automated multimedia routing through various streaming formats while scalably supporting monitoring and alerts during the production process. LVC can be used by production personnel, i.e. operators, who are appointed by stream owners or publishers and given the privilege to perform management functions. LVC can also be used by contributors, who are given the privilege to provide signals used in programs, including program owners or publishers. Management functions performed by an Operator include creating logical entities such as inputs, streams, routing, and outputs (described further below), as well as managing the membership of suppliers, either individually or in teams. Users of an LVC can be producers or operators, and suppliers of signals to the LVC. For example, an Operator can be a company that produces on behalf of a program owner or publisher. Production can be generated by an Operator.In the LVC platform, operators can assign inputs and outputs to creations. A creation is a construct that gives structure, filtering by assigning a set of inputs and outputs, and the ability to start / stop the inputs and outputs.

[0022] As used herein, terms such as "component," "system," "platform," "device," "entity," and the like, may refer to computer entities or entities associated with an operable machine having one or more specific functionalities, such as encoder or transcoder functionality, as well as logical entities such as input, output, program conversion, and routing. Entities disclosed herein may be hardware, a combination of hardware and software, software, or running software. Logical entities as used herein represent functionality consumed or produced by logical activities. They are representations of real-world objects within a system and thus have relationships with many other entities within the system. The attributes of entities reflect their purpose and usage. For example, an entity or component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, a thread of execution, a program, and / or a computer. As an example, an application running on a server having a processor and the server or processor may both be components or entities. One or more components or entities may reside within a process and / or thread of execution, and a component may be localized on one computer or server and / or distributed among two or more computers or servers. Additionally, these components or entities may execute from various computer readable media having various data structures stored thereon. The components or entities may communicate via local and / or remote processes, for example, according to signals having one or more data packets (e.g., data from a component interacting with another component in a local system, in a distributed system, and / or across a network such as the Internet).

[0023] FIG. 2 shows that video signals in various formats using various transport protocols, e.g. TS, WebRTC, RTMP, and SRT, are routed as streams over the Internet to an ingest server. As shown, a team of suppliers provides WebRTC and SRT signals, and a single supplier, who is not part of the team, provides an RTMP signal. A publisher or owner of a program can provide a signal using the TS protocol. If the ingested signal is accepted by a logically abstracted input, it is abstracted as a stream and then sent to an abstracted output defined as a logical entity. The output can be mapped to a recorder device or a multi-view device, which shows the video image in so-called tiles. In one routing example using MPEG-DASH over TS, LVC identifies various audiovisual object types using a universal unique identification system known as EIDR. EIDR identifies assets, owners / publishers, titles, edits, and collections, series, periods, episodes, and clips. LVC supports Electronic Programming Guides (EPGs) and Interactive Programming Guides (IPGs). In this example, the TS streams routed to the ingest server contain audio / video data and in-band (IB) metadata associated with movies, news broadcasts, TV and sports programming, etc. There are several industry standards known in the art, including ANSI SCTE351, ANSI SCTE1042, SMPTE2010, and SMPTE2038, that relate to in-band metadata signaling.

[0024] Also routed are out-of-band (OOB) metadata such as PMT and EPG. Metadata can represent any kind of signaling data, such as signaling graphical overlays, tickers, in-program events (such as goals scored in a soccer game), precise timing of events (such as the start of programs, chapters, advertisements, and commercial breaks), asset IDs (such as EIDR, Ad-ID), subtitles, statistical data about a program such as a game, etc. Metadata can also be used to indicate features of a scene that can be recognized or otherwise measured with some degree of confidence, for example using Artificial Intelligence (AI) or Machine Laming (ML). In one embodiment, AI / ML is used to recognize specified features of the audio / video data with a degree of confidence. Once recognized, attributes or parameters associated with the specified features can be used as metadata. For example, AI / ML can be used to analyze the faces of people in a scene and generate metadata associated with the confidence of such analysis, such as smiling or frowning.

[0025] A user of the LVC platform can access the platform through a portal provided by the front-end server at a URL such as www.livevideocloud.com and enter the user's login ID and password. Once logged in, the web portal provides an interface that allows the user / operator to configure various components of the LVC platform in accordance with the present invention. One of the configurable entities of the platform is the input, which can be associated with one or more resulting streams. An input is defined by the user as a persistent logical entity configured and managed via a graphical user interface or an application programming interface (API). In one embodiment, an input exists until it is explicitly deleted via a manual action by an operator. In another embodiment, an input exists as long as there is no incoming signal. In one example, an input can result in many streams. Each stream is associated with a session tag that creates a local connection. Each session tag for a stream is generated by the stream owner / publisher with a unique identification (ID).

[0026] The operator manages the streams using a management interface provided by the front-end server. The DBMS manages the acceptance or approval of the provided signals, and also the enrichment and routing of the accepted or accepted outgoing streams. All incoming signals are aggregated. The ingest server and the operator decide which signals to abstract into logical streams before redistribution and / or recording, and which signals to discard without logical abstraction into streams. The operator can collect the recorded streams into a "media mix" container, typically a ZIP container, for reuse. Reuse may include, for example, downloaded archives, use by third-party editors, viewing on demand after recording, etc. In VOD, the ingest server reads the recorded streams from the media mix container and publishes them to a content delivery network (CDN), which distributes them to downstream viewers.

[0027] In live video streaming, the ingest server allows an operator to manage the accepted streams and select the ones to put "on air" based on configuration information maintained by the DBMS. The back-end server accesses a database to store and retrieve configuration parameters, IDs, attributes, predicates, routing rules, user profile data, etc. In this way, the operator can manage the live session before the accepted streams are sent to the output for distribution. The front-end server displays all the inputs associated with each session tag to the operator, who can view and manage the stream acceptance and routing configuration parameters, as well as the user profiles, e.g., teams, groups, etc.

[0028] The back-end server allows the operator to assign available inputs to start broadcasts and trigger recording of live streams using the interface provided by the front-end server, thus allowing the operator to copy links or embedded codes for sessions for distribution.

[0029] FIG. 3 illustrates a layering diagram of the physical and logical components of the LVC platform by abstracting them as logical layers surrounded by two physical layers: an input physical layer and an output physical layer. As illustrated, the input physical layer represents an input source of a signal, such as a multimedia signal originating from an input hardware device and / or a web browser application running on a computing device. The source is an entity that provides the signal to be provided to a server. The input physical layer also provides the raw provided signal along with IB metadata and / or inserted OOB metadata to a logical layer implemented by a DBMS for processing that implements components such as input, stream, output, routing, and routing rules as logical entities in LVC. The signal is provided to a logical abstraction, referred to herein as input. The admission control component of the policy layer acts as a filter to accept or reject the provided signal based on a user-defined predicate. A predicate is a logical expression that evaluates to TRUE or FALSE. The evaluation dictates the execution logic in processing the code. For example, if the logical expression evaluates to TRUE, the provided signal is accepted. In the same example, if the logical expression evaluates to FALSE, the signal is rejected. In another example, the opposite logic may be applied, as described further below. Signals are logically abstracted as streams based on a user-defined predicate that is used to determine whether the signal is rejected or accepted. Only accepted signals are abstracted as streams, rejected signals are discarded without abstraction. Streams are associated with IB / OOB metadata and further with a Packetized Elementary Stream (PES), a specification of MPEG-2 (ISO / IEC 13818-1) and ITU-T H.222.0. PES defines the transmission of elementary streams (usually the output of an audio or video encoder) in packets within MPEG program streams and MPEG transport streams. Elementary streams are packetized by encapsulating sequential data bytes from the elementary stream in PES packet headers.

[0030] The accepted signal, i.e., the stream representing the accepted stream, is routed based on the routing rules that are part of the policy layer. The policy layer implements the routing rules and other user-defined parameters including program transformations in addition to predicates. The routing rules can take into account IB / OOB metadata during streaming before the logical layer routes the accepted stream to the output. The routing of the accepted stream is generated based on reevaluation by the routing reevaluation loop. The routing is based on the user-defined routing rules before the accepted stream is routed to the output.

[0031] An output can select a subset of PESs in the underlying physical transport based on a user-defined program transformation. Each program transformation contains logical rules associated with the output that are evaluated when generating the stream multiplex. Program transformation uses Program Specific Information (PSI) that contains metadata about the program. The PSI includes, among others, a Program Association Table (PAT) and a Program Mapping Table (PMT). The PMT defines a program by specifying multiple PESs by PID. For example, the PMT references the PES with PID 100 for video and PID 101 for audio. One MPEG signal may contain many PMTs that are listed in the PAT.

[0032] According to the invention, the program transformation selects the accepted stream ahead of time or just in time. For example, the program transformation FIRST_VIDEO for PID100 selects the first video at run time when the program mapping table (PMT) becomes available and delivers the first PES that transmits the video signal on PID100. The logical output is processed by one or more processing units or processors, such as those used by one or more servers, with the logical rules associated with the output being evaluated when generating the multiplex of the streams at the output physical layer. The servers provide the signals including the multiplexes and PESs that are used by the content delivery networks (CDN1-4) or social networks.

[0033] FIG. 4 shows a functional block diagram of attributes of a logical entity defined as an input, including the owner, endpoint, routing rules, predicates, state, and metadata of the signal abstracted by the input. An input has a state such as online, offline, error, unknown, etc. In one example, a source can be a device, an encoder, a web browser providing a signal via public or private WebRTC, or a transcoder. A stream is a representation of an accepted signal that is active and identifiable by a stream. A stream exists while the accepted signal is active. A target / destination is an entity to which one or more streams are routed. A target / destination can be an output, a transcoder, a recorder with an ingress to receive one or more streams. A target / destination can have an ingress for streams used for multiple views. A target / destination can also be an output, a transcoder, or a recorder.

[0034] The output of a source can be an input to a component of the LVC platform. Inputs can be viewed as being implemented in the logical layers shown in Figure 3. LVC supports two types of inputs: "push" inputs and "pull" inputs. Push inputs listen for a connection and push a signal into the LVC platform. Pull inputs require an active connection to a source and pull a signal into the LVC. Examples of inputs are: Pull HLS, MP4 Pull Image (Slate, Overlay) Push SRT Push RTMP ·Push HLS

[0035] In one embodiment, inputs are managed based on type by either a single operator or a team / group of operators, but not both. Certain input types may not be managed. In one example, an input provides one or more URLs to which devices (encoders, software, browsers, etc.) can connect with endpoints. An endpoint is one end of a communication channel, typically represented by a URL of a server or service. An endpoint is disabled when the input is switched off. An endpoint has a Time to Live (TTL) after which it expires. The TTL may be implemented as a counter or timestamp that prevents packets from circulating indefinitely. TTL does not apply to certain inputs such as stable RTMP, push URLs, or other hardware encoders. Endpoints may be dynamic in that they may not be known in advance. For example, an input may dynamically generate endpoints on demand.

[0036] An input may have one or more predicates. A predicate works in the form of f(input, stream) → Boolean (true or false). In one example, if any predicate is false, the input does not accept the signal provided. In this example, the input only accepts the signal provided if all predicates are true. The predicates can be configured by an operator through a user interface provided by a web portal or API. For example, a function can be configured by an operator to be "can accept a signal during a certain time period", "can accept a signal if it starts with a certain geofence or within a certain zone or location", or "can accept a signal from a certain IP address", etc.

[0037] The function Geofence:(lat,lng,d)->(input,stream)->boolean is a function that generates a predicate for a given latitude, longitude, and distance. This function can be used to evaluate the metadata of the fed signals accordingly and to reject all signals that are not near a particular point of interest. If the predicate is false, the signal is dropped from ingest without being abstracted as a stream. The LVC platform can generate predicates for internal use by system administrators. For example, incoming connections for input can be restricted with an internal predicate that allows only a maximum of N connections.

[0038] The LVC platform supports user-defined automated routing of routable entities adhering to various routing protocols. Routing establishes a connection between an accepted stream (or its actual video source) and a target. The target remains valid as long as the stream is valid. More specifically, an input can be routed based on user-defined routing rules. A routing rule is a definition attached to an input that is evaluated when a supplied signal appears at the input. Routing rules are reevaluated periodically (e.g., every second) to determine the state of the stream. The routing rules of an input are reevaluated if the predicate changes.

[0039] With user-defined routing rules, the desired setup can be defined as follows: Switch from on-demand loop to live at 19:30 Drop the stream if it is below 10mbit / s Record incoming streams when not in business hours · Always route stream from input A to output B

[0040] When routing a stream representing an accepted signal, a connection can be established over a communication channel between the source and the target as long as the accepted signal exists. A stream can represent an accepted signal including traditional video, compressed data streams or data files, as well as IB metadata transmitted along with other components of the audiovisual content (video, audio, subtitles, etc.). Inputs can define additional metadata that accompanies the accepted stream when the operator creates or defines it as a logical entity. Since metadata can change over time, streams are reevaluated when their metadata changes. In one embodiment, a change in the state of an input also causes a reevaluation of the routing relationship.

[0041] Generating Input Figures 5 to 12 show a process for generating inputs in the LVC platform according to the present invention. The process begins in Figure 5 where an operator logging into the LVC platform shown in Figure 2 selects the Inputs tab (not shown) via a user interface. Within the Inputs tab, the operator clicks on "New Input" and is presented with a list of available inputs such as Stream Tag, RTMP Push, or SRT Push. In this example, the operator selects SRT Push. The operator is presented with a configuration wizard that includes two tabs: "General" and "Settings". Figure 6 shows the selection of the "General" tab. This allows the operator to enter general information that will allow the input to be found within the LVC platform. Figure 7 shows the selection of the "Settings" tab, which includes settings specific to SRT Push. The LVC platform generates a unique URL for each input for the signal provided. Figure 8 shows the data center selected from the list for which the URL is provided. Figure 9 shows other configurations such as latency, which in this example is set to 50 ms. Figure 10 shows predicates. Each input may include zero or more predicates. Figure 11 shows the generation of a first predicate that only accepts streams from a given IP address and a second predicate that requests that streams be accepted between 8pm and 10pm. When the operator clicks the "Generate" button, as shown in Figure 12, the LVC requests the associated API to generate a record of the generated input in the DBMS.

[0042] Generate output Outputs are the destinations of streams. They are logical entities that may represent one or more physical servers and many potential destinations, such as social websites or content delivery networks (CDNs) as described in connection with FIG. 3. Physical destinations are grouped within one output to allow the same video processing, such as standard conversion, to be applied to the video signal. An output may contain one or more program transformations. Program transformations may be specified separately for each target / destination. Before generating a multiplex for a specific destination, the incoming PESs specified in the PMT are selected and assigned a specific Packet Identifier (PID). Thus, the user can select which destinations will receive which specific set of streams within one transport stream. For example, one destination may receive audio tracks in one language and another destination may receive audio in a different language. Figures 13 to 18 show a process for generating outputs on an LVC platform according to the invention. Figure 13 shows a portal for generating outputs via a user interface, where the user can generate a new output from a list of different outputs, such as transcoding or pass-through outputs. In Figure 13, the user selects the generation of a pass-through output. Figure 14 shows the selection of the "General" tab, which allows the operator to enter general information that will allow the output to be found within the LVC platform. Figure 15 shows the selection of the "Settings" tab, which allows the user to set the delay, frame rate resolution, and interlace mode.

[0043] Converting Programs FIG. 16 shows the selection of the "Program Transformations" tab, which allows the operator / user to set transformation parameters. The operator may specify program transformations for output as a whole or for each destination individually. Program Transformations are individual rules that are applied to PES within one PMT. They select the PES either up front or at run time. Program Transformations have a mode of either "Absolute", "Relative" or "Any" and an action of either "Absolute", "Keep" or "Drop".

[0044] Absolute and discretionary program transformations are a priori and impose no knowledge requirements on the PMT. Absolute rules select a PES by its PID. Discretionary rules select any PES. Relative rules have some kind of nth form, like "first video" or "second audio". These rules select the nth PES as it appears in the PMT. Relative rules therefore require evaluating the presence of a transport stream with a PMT.

[0045] A program transformation can only be resolved and evaluated once the stream is available. Since the existence of all PESs is known only after the PMT is known, it is impossible to know if two program transformations will collide. However, an operator can deliberately define collision rules to normalize different streams. The rules PID100 to PID100 and PID101 to PID100 normalize a stream with a packetized elementary stream defined on either PID100 or PID101 and make this feed available on PID100. Such a rule will only cause a collision if a source publishes both PID100 and PID101. Thus, a program transformation can only be evaluated and realized once the stream is routed to the output and multiplexed to the destination.

[0046] Program transformations are evaluated in order from most specific to least specific. Absolute transformations are evaluated before relative transformations, which are evaluated before arbitrary transformations. Within these categories, transformations are evaluated from more specific and absolute to less specific and dropped based on the action. Program transformations are instantiated once the PMT is known. In this process, all relative transformations are made absolute by selecting the PES with the nth index in the relative transformation. Once all absolute transformations are known, conflict resolution can occur. In this process, mappings that are substantially identical (PID100 to PID100 and FIRST_VIDEO to PID100, where FIRST_VIDEO is PID100) are merged. Here, the conflicting actions select the correct outcome (keep win vs. drop), otherwise the conflicting mappings are reported to the user as an error (e.g., mappings PID100 to PID200 and PID101 to PID200 conflict if both PID100 and PID200 are present in the video signal). In one embodiment, program transformation parameters are user-defined according to the requirements of the program being created using LVC.

[0047] Waiting time A distributed system that abstracts the underlying infrastructure may induce additional latency for users of the system. Good distribution of signals must deliver these signals at a constant bit rate so that the receiving hardware decoder does not fail, e.g., receive too many packets at a time and cause an internal buffer to overflow, thereby dropping packets and ultimately resulting in packet loss and visual artifacts. Distribution of signals over a wide area network typically requires some form of buffering whereby the received packets can be adjusted to a certain discrete time interval. Without such a buffer, the party forwarding the signal can only forward the signal at the pace at which it receives the signal subject to network jitter. The size of the packet buffer, in other words, is a linear increase in latency. Thus, adding additional servers in the signal flow adds to the perceived latency of the signal, which is undesirable. LVC does not require the buffering required to achieve a constant bit rate until the output sends the signal to its final destination. The inter-arrival time (IAT) of the packets is recorded at the ingest server and maintained throughout the system until the output. At this point, the IAT of each individual packet can be reconstructed since it is stored as additional metadata along with the packets of the transport stream.

[0048] Figures 17 and 18 show the selection of the "Destination" tab, which allows the user to select destination parameters. Figure 17 shows the specification of multiple destinations, for example RTMP and SRT destinations. The same stream routed to the output will be multiplexed to these destinations. Figure 18 shows that when the operator clicks the "Generate" button, the Live Video Cloud UI requests the Live Video Cloud API to maintain the output record in the database. The Live Video Cloud API verifies the correctness of the data and maintains the valid output in the database with a record with a unique identifier for the generated output.

[0049] stream Figure 19 shows the attributes that define a stream. A stream can be associated with an accepted video signal referenced by an identified URL and metadata. The metadata can be part of the stream (i.e. IB metadata) or provided from a different source, in which case other components of the system can read the stream metadata without knowledge of the binary format of the used video codec, such as H.264. When an external entity pushes a video signal to the LVC or when a video signal is pulled by the LVC, the supplied signal is accepted based on the admission control predicate of the input. Only accepted signals are abstracted as streams, and streams exist as long as the connection is established, stable, and valid based on the reevaluation of the predicate.

[0050] The stream also includes a session identifier that is used to identify sourced signals of the same publisher or owner during reconnection. As seen above, universally unique identifiers (UUIDs) are used to identify streams sourced by publishers, owners, and other types of public or private providers. Session identifiers can be defined and / or generated by the input or provided during validation by the ingest server. Pull inputs generate session identifiers by themselves.

[0051] For example, the user A opens a landing page for public WebRTC push input, and a session ID is assigned to the WebRTC stream. In this example, the user A is the provider of a WebRTC stream, which can be, for example, a received video signal indicating the provider. A1 However, the operator A1 The LVC Platform tracks the session ID until the user is associated with an input that is configured to route the session to output 1. AThe browser client experiences a brief connection loss and the stream is terminated. The browser client is then able to re-establish the connection, which allows a new stream to be A2 Since the session ID is stable during the reconnection, the routing for output 1 can be evaluated and the stream A2 is routed to Output 1.

[0052] A stream can refer to one input and multiple streams can refer to the same input. Each stream defines at least one stream URL, but there can be more than one URL for a stream. Stream URLs have a MIME type attribute. There can be more than one stream URL for the same MIME type. Below are examples of two use cases: Use Case (i) Stream URLs rtmp: / / ingest.live / _ / X123ZW and https: / / ingest.live / _ / X123ZW / thumbnail.jpg may exist for the same stream, one for the live video signal and the other for the preview thumbnail. Use case (ii) Stream URLs rtmp: / / ingest-af123h.smt-services.com / _ / X123ZW and rtmp: / / ingest-bfdh71.smt-services.com / _ / X123ZW may exist for the same stream, where the streams may be backups to the same stream.

[0053] The identity of the publisher / owner of the supply stream corresponding to the accepted signal may be known. If the publisher is known, a valid and unique publisher / owner ID is referenced in the DBMS. The publisher of some of the streams may be unknown.

[0054] The stream corresponding to the received signal contains additional metadata, such as its audio and video codec, frame rate, and possibly position. This metadata may change during the life of the stream and is stored in the DBMS together with a timestamp for the creation of the stream. The change in metadata may be incremental, i.e. at some time t the metadata key x If the value of changes, e.g. the key y ,key z For different metadata such as, the latest value is considered to be present at time t. For example, it may be desirable to store a tuple (latitude, longitude) at t+0 seconds, t+5 seconds, ..., t+N seconds. Or it may be desirable to store a confidence score from an AI / ML system. However, a change in these values ​​does not imply that other metadata that was previously present, such as the video codec, has been lost.

[0055] Routing Rules FIG. 20 shows an example of a routing rule. A routing rule defines the rules to be applied once a provided signal is accepted and abstracted as a stream. A routing rule can generate multiple routing rules based on the corresponding predicate. Multiple routing rules for the same target can be applied to a single input. A routing rule can be viewed as a function f(stream)→routing, resulting in the routing of a given stream abstracted for an accepted signal. A routing rule causes a desired state of physical routing that must be manifested by executing the policy layer shown in FIG. 3. For routing purposes, each source can be uniquely identifiable. Multiple routing rules can reference the same source. In one embodiment, a routing rule references a target. A routing rule can contain many predicates. As mentioned above, a predicate is a function of the form f(rule, stream)→Boolean (true or false). A routing rule does not generate a route if the specified Boolean condition (true or false) of the predicate is not satisfied.

[0056] routing Figure 21 illustrates routing when connecting a source of a stream to a target. In one example, a target is any location inside or outside the LVC platform, but it is not necessarily an output. For example, a target can be a component that records the received stream or enriches the stream based on metadata, for example routing based on AI / ML decision parameters such as analysis of confidence in defined features of the stream.

[0057] The source of a signal can be an ingest server, while the target can be an output, a tiler engine, a switch, a router, or an internal node, such as a recorder. Routing can be generated automatically when a stream appears. Routing can also be generated by components within the LVC platform. In one embodiment, outputs are immutable, i.e., they are not updated after creation. Routing can be edited or deleted by the user after creation.

[0058] Routing refers to a source with a unique identifier or internal or external URL of the stream, for example, source stream UUID / / 95ac51c7-9af1-497b-b542-07be6458b071, or source URL rtmp: / / ingest.live / _ / ABC123. Routing also refers to a determinable target with a unique identifier or URL, for example, target UUID / / 95ac51c7-9af1-497b-b542-07be6458b071, or target videoengine: / / 23ac51c7-9a11-497b-b542-07be6458b071, which may be, for example, a Tyler target of source stream UUID / / 95ac51c7-9af1-497b-b542-07be6458b071.

[0059] Routing Decomposition According to one embodiment, the routing is resolved by resolving the source and the target. The source reference may be resolvable, for example, to a URL or an ingest server. For example, a stream UUID may ultimately be resolvable to a URL or an ingest server. The URL is already considered as the resolved source. The target reference may also be resolved to a URL. The UUID may ultimately be resolved to: If a video engine is referenced, the URL of the video engine's API endpoint. If a creation program is referenced, the URL of the creation API. If output is referenced, the URL of the output API. A target routing can refer to a creation that can change during the life of the routing. Example: Output foo In so far as it is incidental to its creation, foo can be referred to.

[0060] Routing Types Streams can be routed to targets once they appear. An example of such routing is a stream that is routed to a tiler (selector) application when it first appears. An operator can define automation for routing based on criteria. The DBMS evaluates whether a stream matches the criteria and generates a routing to a target that can be dynamically selected.

[0061] In one example, the stream B is a stream A It is defined as a backup of the stream. A If no longer exists, for example due to a connection loss, the stream B will automatically stream AAs another example, an operator can define rules to route streams and use the highest confidence score for metadata derived from applying AI / ML, such as a keyword like "smile." Applying AI / ML facial recognition, metadata from the stream including confidence scores for facial (e.g., smile) features is used for routing based on whether the confidence exceeds a threshold.

[0062] An operator can also route streams to targets by performing actions through a graphical user interface (or using the LVC API). In one embodiment, an operator uses a presented user interface to select a stream and route it to an output.

[0063] Routing Protocols The target implements a routing protocol. A target may also be a source. For example, a route defined using gRPC is as follows: Service Routing Target { / / Attach the source to this target / / / / The implementation must be idempotent. If the stream is / / If you are already connected to this target, the request is simply ignored. / / The existing token is returned. / / / / The target may disconnect any stream with the same stream_session_id. / / This is only appropriate for targets that accept only one input. / / rpc connect(connection request) returns(connection response); / / Disconnect the source from this target. rpc disconnect(disconnect request) returns(disconnect response); } message disconnect request { String routing_id=1; / / Unique identifier for the route String source_url=2; / / The parsed url of the source String stream_session_id=3; / / Unique session identifier for the stream } message connect response String token = 1; / / An arbitrary token that identifies an established connection repeat string disconnect=2; / / List of connections with newly established connections (each token) } message disconnect request { String token = 1; / / Must match the token given by the connection response } Message disconnect response {}

[0064] Routing and Routing Rules When a stream is routed to an output by an operator, routing rules are entered from input to output against the stream's unique session identifier. There can be at most one routing rule per output target. There can be multiple routing rules for different outputs. However, there can be at most one routing rule for the same source->target relationship. When generating a new routing rule for a given target, existing routing rules, for example from different inputs, are removed.

[0065] Routing rule evaluation and re-evaluation Evaluation of a routing rule may include testing all predicates, building a set of targets for a given routing rule, and mapping the targets to associated routes. When any routing rule, its targets, or inputs are created, updated, or deleted, the routing rule is evaluated regardless of the state of the inputs (e.g., online or offline). Re-evaluation may be performed when: (1) If the referenced target of the creation changes, (2) When a referenced target is updated or deleted, (3) If routing-related preconditions change, (4) If the input preconditions change, (5) If the state of the input changes, (6) If a routing rule is added to the input, and (7) When a routing rule is removed from the input.

[0066] One stream by the operator exp The output is exp If a stream is routed to a , the associated routing rule for the associated input is (stream=s exp , target = output: / / o exp ) The operator creates different P new If you decide to move to P old This action results in a re-evaluation of all routing rules for the input. The DBMS therefore performs a re-evaluation of the current routing and evaluation of the new routing by doing the following: 1. Change the routing rules and old Remove and P new Add 2. Load all current routing for the input and context and run the routing R old See 3. If the input state is online, evaluate all remaining routing relations. If the input is offline, evaluate the resulting routing R old .R new Call a set of equals {} 4. Common set R old ∩R new You can ignore routing in 5. Different Sets R old \R new , R old Inside is R new Delete each route in the set of all routes that are not in 6. Different Sets R new \R old , R new Inside is R old Delete each route in the set of all routes that are not in 7. Routing without context information remains unchanged.

[0067] In effect, the routing rule (stream = s exp , target = output: / / o exp ) is first removed, since the output is related to the production. However, the production is no longer in the list of routing rules. All routings that are no longer valid, e.g. routings resulting from rules for outputs, are disconnected. Finally, new routings, e.g. for a new tiler, can be created.

[0068] The following re-evaluation algorithm can be used to map from a current state or routing to a desired routing state. (1) Build a set of referenced creations from rules that have creations as targets. (2) Fetch all streams for the input being re-evaluated (3) Match each stream against the input predicate and discard non-matches. (4) Remove all rules with creation references that are not in the set from (1). (5) Look at the input and fetch all routes whose context is in the set from (1). (6) Construct a new routing set by evaluating each rule for each stream, assuming that the input is online. If the input is not online, use the empty set. (7) Disconnect all routes that are in the set from (5) but not in (6). (8) Connect all routes that are in the set from (6) but not in (5)

[0069] Routing Management Example In this example, the following criteria are assumed: (1) Input A creates a stream that is accepted A has a rule that routes to (2) Creation A Hey, Tyler A Generate routing for incoming connections to (3) Reassessment is carried out

[0070] In this example, based on criterion (1), A These routes are created as a context. A Receive the UUID of the stream created A If a service is routed to a Tyler, the service receiving the route will be A Generates a new routing for the stream for . The routing is no longer a routing since it is not generated by the input, and has no context information attached to it.

[0071] It then performs a re-evaluation based on criterion (3). Once re-evaluation has been performed, the corresponding routing from create to tiler is removed if it was removed from create. By ignoring the routing to tiler generated by create, a routing to tiler is not removed if the corresponding routing remains active for create. Routing that is not directly generated by the input is ignored during re-evaluation. The service handling the routing for create is assumed to have disconnected the corresponding stream from the tiler if the service determines that the stream has been disconnected. Connecting, on the other hand, is an idempotent operation, i.e., an operation that can be performed an infinite number of times, and since the state is identical, it will not generate the same routing twice.

[0072] A target routing rule can indirectly reference a creation, and this reference can change during the life of the routing rule.

[0073] Example: Output foo created insofar as it is incidental to its creation foo Refer to the routing rules production Create foo is associated with.

[0074] In one embodiment, the LVC platform supports the following non-exhaustive example input types: 1:RTMP push input; 2: Public / Private WebRTC Push; 3:HLS Pull

[0075] An RTMP Push Input predefines an endpoint for a single feeding stream in a 1:1 relationship. An endpoint has a form like rtmp: / / ingest.live / i / XYZ123. An RTMP Push Input can be an external device owned by either a user or a team. An RTMP Push Input does not allow a stream to replace the current video signal if a stream is considered currently active for the same session ID. For example, an RTMP Push Input can support a predicate to accept signals based on IP ranges.

[0076] A public WebRTC push can define both a WebRTC endpoint and a landing page that a supplier can use to start a stream, e.g., in a 1:N relationship where an associated input can result in many streams. A public WebRTC input can be owned by a team of suppliers, rather than a single supplier. In one embodiment, a public WebRTC input supports predicates for accepting signals, e.g., based on the supplier's current location. Private and public WebRTC pushes can potentially be the same input type. A private WebRTC push can define additional predicates, e.g., that a supplier must exist and be a member of a particular team. On the other hand, an HLS pull input is owned by a team, rather than a single user.

[0077] Streams can be recorded. Whether a stream is recorded can change during its life. Streams that are not recorded are lost and cannot be recovered. Stream URLs can be selected based on MIME type. One embodiment of the LVC platform builds a subset of stream URLs for a given MIME type. If the resulting set contains more than one entry, the result is selected randomly using round-robin load balancing.

[0078] Figure 22 shows an example of a flow diagram between entities of the LVC platform. As shown, an external party named a device inside the cloud connects to the ingest server (1), which enables the connection via an input (2). Assuming an input is accepted, for example a WebRTC push input, this creates a new stream and informs the LVC platform of this (3). Once the routing relationships are evaluated (4), three routes are applied to the accepted stream, namely to output, create, or recorder (5).

[0079] FIG. 23 shows a flow chart illustrating an embodiment of the present invention in a method implemented by logic in the LVC platform. As shown, the LVC platform provides a login interface to operators of publishers / owners of programs identified by a universal unique publisher ID. Operators create and manage individuals or groups / teams that supply public or private streams. Operators create inputs that identify streams and associate predicate logic for accepting or rejecting the supplied signal. An admission control process accepts the supplied signal based on the input logic that identifies such signals based on user / operator defined predicates. Decision logic checks the admission control predicates to determine true or false based on the user defined predicates. Based on this decision, the supplied signal is rejected or accepted. One logic may accept a signal if the associated predicate is true and reject a signal if the associated predicate is false. The opposite logic may accept a signal if the associated predicate is false and reject a signal if the associated predicate is true. In one embodiment, the logic selection is made by an operator accessing the LVC platform. Such an operator specifies the routing of the input based on routing rules that are applied to the accepted stream. In one example, an operator specifies predicates for routing from source to destination. All operator specified or otherwise generated input, routing, input and routing predicates, and program transformation parameters are stored in the DBMS shown in Figure 2. To check the predicates, retrieve the predicates and provide them to the admission control and routing logic process for routing the ingress stream based on the routing rules to the output based on the program transformation.

[0080] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuitry, integrated circuits, specially designed application specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs executable and / or interpretable on a programmable system. The programmable system includes at least one programmable processor, which may be special purpose or general purpose, coupled to receive data and instructions from and transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0081] These computer programs (also known as programs, software, software applications, or code) contain machine instructions for a programmable processor and may be implemented in high level procedural and / or object oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine readable medium" and "computer readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic disks, optical disks, memory, programmable logic devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine readable medium that receives machine instructions as a machine readable signal. The term "machine readable signal" refers to any signal used to provide instructions and / or data to a programmable processor.

[0082] To enable interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other types of devices may also be used to enable interaction with a user. For example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user may be received in any form, including acoustic, speech, or tactile input.

[0083] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., a data server), or includes middleware components (e.g., an application server), or includes front-end components (e.g., a client computer having a graphical user interface or a web browser through which a user can interact with embodiments of the systems and techniques described herein), or includes any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0084] The computing system may include a scalable server implemented in a cloud that includes multiple scalable processors that execute logical entities such as the input, stream, routing, and output described above to meet various programming needs. For example, as the number of feed signals increases, more processors can be deployed to execute the described logical entities. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on each computer and having a mutual relationship of client and server.

[0085] The foregoing description does not represent an exhaustive list of all possible embodiments consistent with the present disclosure or all possible variations of the described embodiments. Numerous embodiments have been described. Nevertheless, it will be understood that various modifications are possible without departing from the spirit and scope of the systems, devices, methods, and techniques described herein. For example, various forms of the above flows may be used, and steps may be rearranged, added, or removed. Accordingly, other embodiments are within the scope of the following claims.

Claims

1. 1. A method implemented in a computing system for creating a program using signals received over a communication channel from one or more sources, comprising: a) executing application software on one or more servers of the computing system, at least one of the one or more servers having one or more processors for processing defined logical entities, each logical entity being represented by a corresponding data structure stored in a storage system, the data structures being used to perform functional activities; b) receiving signals used to create the program from a source over a communications channel; c) defining an input logic entity including a first data structure defining attributes used to route said signals, said input logic entity responsive to user defined predicates including logical expressions used to accept or reject said signals; d) defining a stream logical entity including a second data structure used to identify the received signal; e) defining a routing logical entity for establishing a connection to the destination; f) routing the accepted signal to the destination based on the attributes used for routing the accepted signal; The method includes:

2. The method of claim 1 , further comprising the step of defining an output logical entity comprising a fourth data structure that associates the stream logical entity with the destination.

3. 3. The method of claim 2, wherein the output logic entity transforms the stream logic entity according to user-defined program transformation rules that are evaluated to map the received signal to the destination.

4. 4. The method of claim 3, wherein said signal is an MPEG signal and said user-defined program transformation rules are based on Program Specific Information (PSI).

5. The method of claim 4 , wherein the stream logical entity is associated with a Packetized Elementary Stream (PES).

6. The method of claim 5 , wherein the PSI includes a Program Mapping Table (PMT), and the stream logical entities are associated with a PMT.

7. 5. The method of claim 4, wherein the inter-arrival time (IAT) of packets of the MPEG signal is recorded at a server and used by the output logic entity to achieve a constant bit rate.

8. The method of claim 1, wherein f) includes evaluating a user-defined logic rule when the signal is presented to the input logic entity, the user-defined logic rule being associated with the input logic entity.

9. The method of claim 1 , wherein the user-defined predicate is based on a time, a geographic region, a location, or an IP address.

10. The method of claim 1 , wherein the input logical entity is defined by a user as a persistent logical entity.

11. The method of claim 10 , wherein the input logic entity is configured by the user via a graphical user interface or an application programming interface (API).

12. The method of claim 1 , wherein the stream logical entity is associated with a session tag that creates a local connection.

13. The method of claim 1 , wherein the stream logical entity is associated with a unique identification (ID) of an owner or publisher of the signal.

14. The method of claim 1 , further comprising recognizing features of the signal based on a confidence level and using the recognized features as metadata associated with the signal.

Citation Information

Patent Citations

  • video network

    JP2006516372A

  • Methods and apparatus for using tuners efficiently for delivering one or more programs

    US20100020794A1

  • System and Method for Streaming Content from Multiple Servers

    US20190222619A1

  • Video streaming

    US20190289052A1