Method and server for producing live video broadcasts from multiple video sources

Abstract

A method and server for producing a live video broadcast from multiple video sources. The server receives a first raw video signal from a first edge device connected to a first video source. The server processes the first raw video signal to generate a broadcast video signal and an ultra-low latency (ULL) video stream, which are transmitted to the video broadcast equipment and user device, respectively. The server receives a command to switch to a second video source and sends a command to begin transmitting a second raw video signal from a second edge device connected to the second video source. The server receives a second raw video signal from the second edge device and generates a broadcast video signal and an ULL video stream based on the second raw video signal instead of the first raw video signal. The server sends a command to stop transmitting the first raw video signal.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 525,015, filed on July 5, 2023, entitled "METHOD AND SYSTEM FOR PRODUCING LIVE VIDEO BROADCASTING FROM A PLURALITY OF REMOTE VIDEO EQUIPMENT", the entire disclosure of which is incorporated herein by reference.

[0002] This disclosure relates to the field of live video broadcasting. More particularly, this disclosure relates to methods and servers for producing live video broadcasts from a plurality of video sources.

Background Art

[0003] The video broadcasting industry is an industry in which a plurality of technological developments and innovations have occurred in the past few years. One of the most prominent trends is the transition from proprietary technologies developed specifically for the video broadcasting industry to more general - purpose technologies that have been deployed and proven in other industries. In particular, proprietary communication protocols are being replaced by Internet - based protocols. One advantage of using Internet - based protocols is that the cost of deployment is generally lower. Another advantage is the ability to benefit from the development and evolution of new features (e.g., the evolution of existing communication protocols and / or the development of new communication protocols). In particular, innovations in fields not related to the video broadcasting industry are still applicable and useful in the video broadcasting industry. Yet another advantage is the high level of standardization in the development of Internet protocols, which ensures interoperability between devices of different manufacturers.

[0004] One major challenge in implementing a live video broadcasting system is providing the ability to transmit large amounts of data in real time (or at least near real time). For example, in a typical live video broadcasting setup, multiple cameras operate in parallel, providing different viewpoints of the event being filmed. The operator responsible for controlling the live video broadcast in real time needs to have access to and control the video signals generated by the multiple cameras. More specifically, the operator needs to have information (e.g., images) representing the video signal generated by each camera, so that they can select in real time which camera should be used for the live broadcast. Furthermore, the video signals generated from the selected cameras need to be transmitted to the video broadcasting equipment in an efficient manner (to minimize latency, avoid data loss, etc.). This ensures that the video signals generated by a given camera are always available and ready to be effectively transmitted to the live video broadcasting equipment when selected by the operator.

[0005] Therefore, there is a need for new methods and servers for producing live video broadcasts from multiple video sources. [Overview of the Initiative]

[0006] According to a first aspect, the disclosure relates to a method for producing a live video broadcast from multiple video sources. The method includes the step of receiving a first raw video signal from a first edge device connected to a first video source in a server, wherein the first raw video signal is generated by the first edge device based on the first video signal received from the first video source. The method includes the step of processing the first raw video signal in the server to generate a broadcast video signal based on the first raw video signal. The method includes the step of transmitting the broadcast video signal to video broadcast equipment in the server. The method includes the step of processing the broadcast video signal in the server to generate an ultra-low latency (ULL) video stream based on the broadcast video signal. The method includes the step of transmitting the ULL video stream to a user device in the server. The method includes the step of receiving a command from the user device in the server to switch to a second video source. The method includes the step of the server sending a command to a second edge device connected to a second video source to initiate the transmission of a second raw video signal generated by the second edge device based on a second video signal received from the second video source. The method includes the server receiving the second raw video signal from the second edge device. The method includes the server processing the second raw video signal to generate a broadcast video signal based on the second raw video signal instead of the first raw video signal. The method includes the server sending a command to a first edge device connected to a first video source to stop the transmission of the first raw video signal.

[0007] According to a second aspect, the disclosure relates to a non-temporary computer-readable medium containing instructions that can be executed by a server processing unit. The execution of instructions by the server processing unit provides for producing a live video broadcast from multiple video sources by performing the aforementioned method.

[0008] According to a third aspect, the disclosure relates to a server for producing live video broadcasts from multiple video sources. The server comprises at least one network interface card (NIC) for receiving and transmitting data, and a processing unit. The processing unit receives a first raw video signal from a first edge device connected to a first video source, the first raw video signal being generated by the first edge device based on the first video signal received from the first video source. The processing unit processes the first raw video signal to generate a broadcast video signal based on the first raw video signal. The processing unit transmits the broadcast video signal to a video broadcasting device. The processing unit processes the broadcast video signal to generate an ultra-low latency (ULL) video stream based on the broadcast video signal. The processing unit transmits the ULL video stream to a user device. The processing unit receives a command from the user device to switch to a second video source. The processing unit sends a command to a second edge device connected to the second video source to start transmitting a second raw video signal generated by the second edge device based on the second video signal received from the second video source. The processing unit receives the second raw video signal from the second edge device. The processing unit processes the second raw video signal to generate a broadcast video signal based on the second raw video signal instead of the first raw video signal. The processing unit sends a command to the first edge device connected to the first video source to stop transmitting the first raw video signal.

[0009] In certain embodiments, the server implements zero-copy networking functionality for transmitting a first raw video signal from a first edge device to the server and a second raw video signal from a second edge device to the server. In certain embodiments, the server includes at least one network interface card (NIC) that supports the zero-copy networking functionality. In another particular embodiment, the zero-copy networking is Remote Direct Memory Access (RDMA).

[0010] In another specific embodiment, the server performs a virtual switch function.

[0011] In yet another specific embodiment, the server is a cloud-based server located in a cloud infrastructure. In a particular embodiment, the first and second edge devices are cloud-based edge devices that are also located in a cloud infrastructure.

[0012] In yet another specific embodiment, at least one of the following steps is performed by the server's processing unit: applying one or more effects to a first raw video signal or a second raw video signal to generate a broadcast video signal; and applying a video transition between the first raw video signal and the second raw video signal to generate a broadcast video signal.

[0013] In another specific embodiment, the server's processing unit receives multiple ULL video streams from multiple edge devices, each ULL video stream being generated by one of the multiple edge devices based on a video signal received from a video source connected to the edge device, and the multiple edge devices comprise a first edge device and a second edge device. The server's processing unit combines the multiple ULL video streams into a combined ultra-low latency video stream and transmits the combined ultra-low latency video stream to the user device.

[0014] Embodiments of this disclosure are described for illustrative purposes only with reference to the accompanying drawings. [Brief explanation of the drawing]

[0015] [Figure 1] This refers to a live video broadcasting system comprising a server, user devices, and multiple video equipment.

[0016] [Figure 2] Figure 1 shows the components of the server.

[0017] [Figure 3] Figure 1 illustrates the interaction between the server, the user device, and multiple video devices. [Figure 4] Figure 1 illustrates the interaction between the server, the user device, and multiple video devices. [Figure 5] Figure 1 illustrates the interaction between the server, the user device, and multiple video devices.

[0018] [Figure 6A] This represents the video stream displayed on the screen of the device being used. [Figure 6B] This represents the video stream displayed on the screen of the device being used.

[0019] [Figure 7] This represents an alternative implementation of the live video broadcasting system shown in Figure 1, where the server performs the virtual switch function.

[0020] [Figure 8A] This illustrates an alternative implementation of the live video broadcasting system shown in Figure 1, where the server and optionally edge devices are localized within the cloud infrastructure. [Figure 8B]Represents an alternative implementation of the live video broadcast system of FIG. 1, where the server shown in FIG. 1 and optionally the edge device are localized in the cloud infrastructure.

[0021] [Figure 9A] Represents a method for producing a live video broadcast from a plurality of video sources. [Figure 9B] Represents a method for producing a live video broadcast from a plurality of video sources.

[0022] <000**********99>Represents the components of the user device shown in FIG. 1.

[0023] [Figure 11] Represents a method for performing zero-delay transitions between broadcast sources.

[0024] [Figure 12] Represents an alternative implementation of the signaling shown in FIG. 3. **DETAILED DESCRIPTION OF THE INVENTION**

[0025] The foregoing features and other features will become more apparent when reading the following non-limiting description of its exemplary embodiments given for illustrative purposes only with reference to the accompanying drawings. Like numbers represent like features on various drawings.

[0026] Various aspects of the present disclosure generally address one or more of the problems related to the transport and processing of video signals generated by a plurality of video sources (e.g., cameras) in an Internet protocol (IP) - based live video broadcast system. More specifically, the present disclosure addresses the efficient transport of video signals between devices of a live video broadcast system and the selection / control of video signals (issued from one of the video sources) transmitted to a video broadcast device (for being broadcast).

[0027] Here, we refer to Figures 1 and 2 simultaneously. A live video broadcasting system is shown in Figure 1. The live video broadcasting system comprises multiple interacting components that support live video broadcasting time. The components shown in Figure 1 include a server 100, a user device 200, a switch 60, multiple edge devices 300, multiple video sources 10, and broadcasting equipment 20. A detailed description of the components of the server 100 is provided in Figure 2.

[0028] Server 100 is a central entity responsible for supporting the operation of the live video broadcasting system. The server is implemented by a single computing device (e.g., a single computer) or a group of computing devices (e.g., a group of computers working together to perform the functions of Server 100). Server 100 communicates with user devices 200 via the Internet Protocol (IP) networking infrastructure 50. Although a single user device 200 is shown in Figure 1, Server 100 can communicate with multiple user devices 200. Server 100 also communicates with edge devices 300 and broadcasting equipment 20 via the IP networking infrastructure 50.

[0029] Switch 60 is part of an IP networking infrastructure 50 that interconnects Server 100 with user devices 200, edge devices 300, and broadcast equipment 20. In an exemplary configuration, at least some of Server 100's communications with other equipment pass through Switch 60. As is well known in the art of IP networking, the IP networking infrastructure 50 may include two or more networking devices (e.g., additional switches and / or routers) that provide interconnection with other equipment (e.g., 200, 300, and 30) of Server 100. When the IP networking infrastructure 50 is referred to in the following parts of this specification, it is implied that the IP networking infrastructure 50 comprises networking devices such as Switch 60.

[0030] Examples of video sources 10 include, but are not limited to, cameras. For the sake of simplification, Figure 1 shows each video source 10 connected to a single edge device 300, but a given video source 10 may be connected to two or more edge devices 300 for redundancy. For example, if a given video source 10 is connected to two different edge devices 300, and one of the edge devices 300 fails, the other edge device is still available to transmit the video flow generated based on the video signal of the given video source 10 via the networking infrastructure 50. Figure 1 shows an exemplary configuration having three video sources 10, referred to as video source (1), video source (2), and video source (3), with each of the three corresponding edge devices 300 referred to as edge device (1), edge device (2), and edge device (3). Furthermore, two or more video sources 10 may be connected to the same edge device 300. In this case, the processing (described later in this specification) applied by the edge device 300 to the video signal generated by a single video source 10 can be applied to each of the video sources 10 connected to the same edge device 300. Similarly, although a single broadcasting device 20 is shown in Figure 1 for simplification purposes, multiple broadcasting devices 20 may be controlled by the server 100.

[0031] The live video broadcasting system shown in Figure 1 is capable of supporting various types of video sources 10 that generate various types of video signals. Examples of video signals generated by video sources 10 include Serial Digital Interface (SDI) video signals, High-Definition Multimedia Interface (HDMI) video signals, and video signals compliant with the Society of Motion Picture & Television Engineers (SMPTE) 2110 standard. Each video source 10 is physically connected to the corresponding edge device 300 via a cable compatible with the type of video signal generated by the video source 10. Alternatively, a wireless connection is used between the video source 10 and the edge device 300. Details regarding the components of video source 10 are well known in the art and are outside the scope of this disclosure.

[0032] The live video broadcasting system shown in Figure 1 can support various types of broadcasting equipment 20 capable of broadcasting various types of video signals. The types of video signals supported by the broadcasting equipment 20 are the same as the types of video signals supported by the video source 10. The broadcasting equipment 20 broadcasts video signals over large-scale infrastructure such as cable television networks, satellite television networks, and internet-based video streaming infrastructure. Optionally, an edge device 300 similar to the one used for the video source 10 is used to convert the video flow received from the server 100 into a video signal that the broadcasting equipment 20 can process. Details regarding the components of the broadcasting equipment 20 are well known in the art and are outside the scope of this disclosure.

[0033] Figure 2 shows the details of the components of server 100. Server 100 comprises a processing unit 110, memory 120, and at least one network interface card (NIC) 130. Optionally, server 100 may include additional components such as a user interface 140 and a screen 150.

[0034] The processing unit 110 comprises one or more processors (not shown in Figure 2 for illustrative purposes) capable of executing computer program instructions. Each processor may further comprise one or more cores. The processing unit may also include one or more dedicated processing components (e.g., network processors, application-specific integrated circuits (ASICs)) to perform specialized functions (e.g., video processing functions, networking functions).

[0035] Memory 120 stores instructions for computer programs executed by the processing unit 110, data generated by the execution of computer programs by the processing unit 110, and data received via the NIC 130. Although only a single memory 120 is shown in Figure 2, the server 100 may have multiple types of memory, including volatile memory (e.g., Random Access Memory (RAM)) and non-volatile memory (e.g., hard drives, Erasable Programmable Read-Only Memory (EPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM)).

[0036] Each NIC 130 enables the server 100 to exchange data with other devices (a first NIC and a second optional NIC are shown in Figure 2 for illustrative purposes only) via the IP networking infrastructure 50. Examples of wired NICs 130 include standard (electrical) Ethernet ports, fiber optic ports, and ports adapted to receive small form factor pluggable (SFP) units. NICs 130 may also be wireless (e.g., Wi-Fi interface). NICs 130 comprise a combination of hardware and software executed by the hardware for performing the communication functions of NIC 130. Alternatively, the combination of hardware and software for performing the communication functions of NIC 130 is at least partially included in the processing unit 110.

[0037] Regarding the user device 200, various types of user devices 200, such as computers, smartphones, and tablets, are considered. Details of the user device 200 are not shown in Figure 1 for the sake of simplification (they will be described in detail later in relation to Figure 10). The components of the user device 200 are similar to those of the server 100 shown in Figure 2. The user device 200 generally comprises a processing unit, memory, one or more NICs, a user interface, and a screen. The processing unit comprises at least one processor capable of executing instructions for a computer program. At least one of the NICs is adapted for receiving and transmitting data over the IP networking infrastructure 50.

[0038] With respect to the edge device 300, multiple embodiments are supported by this disclosure. Details of the edge device 300 are not shown in Figure 1 for the sake of simplification. In a first exemplary embodiment, the edge device 300 is a computing device having components similar to those of the server 100 shown in Figure 2. The edge device 300 typically comprises a processing unit, memory, and one or more NICs. The processing unit comprises at least one processor capable of executing instructions for a computer program. The processing unit may also include one or more dedicated processing components (e.g., a network processor, an application-specific integrated circuit (ASIC)) for performing specialized functions (e.g., video processing functions, networking functions). At least one of the NICs is adapted to receive and transmit data over the IP networking infrastructure 50 to interact with the server 100 and the user device 200. The exchange of data (video signals) between the edge device 300 and the video source 10 may be performed over one of the NICs or over a dedicated video interface.

[0039] In a second exemplary implementation, the edge device 300 is a standardized hot-pluggable transceiving unit, such as a small form factor pluggable (SFP) unit. The transceiver unit comprises a processing unit and a front connector adapted to receive video signals generated and transmitted by a video source 10 to which the transceiver unit is connected. The transceiver unit also comprises a front connector that implements a NIC adapted for exchanging data with the server 300. The transceiver unit further comprises a chassis adapted for insertion into the chassis of a host unit. This allows multiple transceivers, each implementing an edge device 300, to be inserted into a single chassis.

[0040] Now, refer to Figures 2 and 3 simultaneously. Figure 3 corresponds to the live video broadcasting system shown in Figure 1.

[0041] Each video source 10 transmits its video signal to the corresponding edge device 300. For illustrative purposes, consider that the video signal generated by video source (1) is currently being broadcast, and the video signals generated by video sources (2) and (3) are not being broadcast.

[0042] Based on the received video signal, each edge device 300 generates an ultra-low latency (ULL) video stream and a raw video signal. The edge device performs one or more video processing functions (e.g., decoding, encoding, decompression, compression, subsampling, etc.) to generate the ULL video stream and the raw video signal.

[0043] A ULL video stream is a low-quality video stream that requires significantly less bandwidth than a raw video signal for transmission over an IP networking infrastructure 50 (for example, 100 times, or possibly 1000 times, in some cases). For example, generating a ULL video stream includes (for illustrative purposes only) decoding the video signal transmitted by the video source 10, optionally subsampling the video signal, and compressing the video signal. The implementation of generating a ULL video stream based on a video signal received from the video source 10 is outside the scope of this disclosure and may vary, for example, based on the characteristics of the video signal received from the video source 10.

[0044] All ULL video streams generated by the edge device 300 are transmitted to the user device 200. On the user device 200, the ULL video streams are displayed on the user device 200's screen as shown later in this specification. Alternatively, only some of the edge devices 300 generate and transmit ULL video streams (for example, based on a command requesting only some of the edge devices 300 to transmit ULL video streams sent by the user device 200).

[0045] As mentioned above, the raw video signal is a high-quality video signal that requires significantly more bandwidth than the ULL video stream for transmission over the IP networking infrastructure 50. For example, generating the raw video signal includes (for illustrative purposes only) decoding the video signal transmitted by the video source 10. Performing the generation of the raw video signal based on the video signal received from the video source 10 is outside the scope of this disclosure and may vary, for example, based on the characteristics of the video signal received from the video source 10.

[0046] Only one raw video signal generated by one of the edge devices 300 (for example, edge device (1) in Figure 3) is transmitted to the server 100 3. In the server 100, the raw video signal is processed as shown later herein.

[0047] In the first exemplary implementation, all edge devices 300 each generate a raw video signal based on the video signal received from the video source 10, but only one edge device 300 sends it to the server 100. The other edge devices 300 are in standby mode and ready to send the raw video signal when they receive a command from the server 100 to start sending.

[0048] In the second exemplary implementation, only the edge device 300 that is sending the raw video signal to the server 100 generates the raw video signal based on the video signal received from the video source 10. The other edge devices 300 are in standby mode and are ready to generate and transmit the raw video signal when they receive a command from the server 100 to start transmission.

[0049] Based on the raw video signal received from the edge device 300 (for example, edge device (1) in Figure 3), the server 100 generates a broadcast video signal. The server 100 then generates a ULL video stream based on the broadcast video signal. The server 100 performs one or more video processing operations (e.g., decoding, encoding, decompression, compression, subsampling, etc.) to generate the broadcast video signal.

[0050] A broadcast video signal is a high-quality video stream that requires significantly more bandwidth than a ULL video stream for transmission over the IP networking infrastructure 50. For example, generating a broadcast video signal may (for illustrative purposes only) involve decoding a raw video signal transmitted by an edge device 300 and optionally applying one or more effects to the raw video signal. The one or more effects to be applied may be pre-configured in server 100, transmitted by user device 200, etc. Performing the generation of a broadcast video signal based on a raw video signal received from edge device 300 is outside the scope of this disclosure and may vary, for example, based on the characteristics of the raw video signal received from edge device 300, based on the characteristics and capabilities of broadcast equipment 20, based on the pre-configuration in server 100, based on the information transmitted by user device 200, etc.

[0051] The broadcast video signal is transmitted to the broadcasting equipment 20. In the broadcasting equipment 20, the broadcast video signal is processed (e.g., decoded) for broadcast on a video broadcasting infrastructure (not shown in Figure 3). The processing performed by the broadcasting equipment 20 is well known in the art and is outside the scope of this disclosure.

[0052] Broadcast video signals transmitted to broadcast equipment 20 are typically interoperable video streams that can be processed by different types of broadcast equipment 20 from different manufacturers. For this reason, one or more standardized video protocols, such as the Society of Motion Picture and Television Engineers (SMPTE) 2110, are used to transmit broadcast video signals.

[0053] The ULL video stream generated by server 100 is similar to the ULL video stream generated by edge device 300. The implementation of generating a ULL video stream based on a broadcast video signal received from edge device 300 is outside the scope of this disclosure and may differ, for example, based on the characteristics of the raw video signal received from edge device 300.

[0054] The ULL video stream generated by server 100 is transmitted to user device 200. On user device 200, the ULL video stream is displayed on the user device 200's screen. This ULL video stream received from server 100 represents the broadcasted video signal. The ULL video stream received from edge device 300 represents the video signal received from video source 10.

[0055] Here, we refer to Figures 3 and 6A simultaneously. Figure 6A represents the screen 250 of the user device 200. As previously stated, the ULL video streams transmitted by the edge device 300 and corresponding to the video sources 10 (video sources (1), (2), and (3) respectively) are displayed on the screen 250. Although only three ULL video streams are shown, additional ULL video streams may be displayed if additional video sources 10 are used. A visual indication 255 (displayed on the screen 250) is used to identify the ULL video stream corresponding to the video source 10 currently being used as a source for broadcast (video source (1) in Figure 6A). The implementation of the visual indication 255 may vary and is outside the scope of this disclosure.

[0056] In an exemplary implementation (not shown in Figure 6A), the ULL video stream received from the edge device 300 is scaled (resized) by the user device 200 and displayed on screen 250 as a single scaled ULL video stream.

[0057] The ULL video stream transmitted by server 100 (corresponding to the current broadcast video signal using video source (1) as the source in Figure 6A) is also displayed on screen 250.

[0058] In the alternative implementation, the server 100 does not generate a ULL video stream, and the ULL video stream is not subsequently sent to the user device 200 (and is not displayed on the screen 250 of the user device 200).

[0059] Here, we refer to Figures 2, 3, and 4 simultaneously. Figure 4 corresponds to the live video broadcasting system shown in Figure 3. The ULL stream transmitted 2 by the edge device 300 to the user device 200 (in Figure 3) is not shown in Figure 4 for simplification purposes only (transmission 2 is not affected by the signaling shown in Figure 4 and is still occurring).

[0060] User device 200 sends a command to server 100 to switch the source for broadcasting.6 For illustrative purposes only, Figure 4 shows server 100 switching the source from video source (1) to video source (2). The command includes identification of the new video source 10 to be used as the source for broadcasting. The identification is used by server 100 to determine the corresponding edge device 300 that needs to be triggered (for example, the identifier of the new video source 10 sent by user device 100 is mapped to the IP address of the corresponding edge device 300). The command is generated through user interaction of user device 200 via the user interface of user device 200. Performing user interaction to select a new video source is outside the scope of this disclosure.

[0061] Optionally, the user of user device 200 has the ability to select one or more effects to be applied to the raw video signal from a newly selected video source. Alternatively, one or more effects are pre-selected and not determined in real time by the user when a new video source is selected. Six commands sent from user device 200 to server 100 include the one or more effects to be applied. Alternatively, the one or more effects to be applied are transmitted via another command (not shown in the diagram) sent from user device 200 to server 100.

[0062] Optionally, the user of user device 200 has the ability to select a video transition to apply to the raw video signal from a newly selected video source. Alternatively, the video transition is pre-selected and not determined in real time by the user when a new video source is selected. Six commands sent from user device 200 to server 100 include the video transition to be applied. Alternatively, the video transition to be applied is transmitted via another command (not shown in the diagram) sent from user device 200 to server 100.

[0063] Server 100 sends a command to edge device 300 (edge ​​device (2) in Figure 4) which has been selected to be a new source for broadcasting.

[0064] Upon receiving the command 7, the selected edge device 300 (for example, edge device (2)) begins transmitting its raw video signal 8 to the server 100.

[0065] When server 100 begins receiving a new raw video signal transmitted by edge device (2), server 100 begins using this new raw video signal (instead of the one transmitted by edge device (1)) to generate and transmit a broadcast video signal to broadcast equipment 20. The generation and transmission of a ULL video stream (based on the broadcast video signal) to user device 200 is also affected by the use of the new raw video signal.

[0066] Optionally, one or more of the aforementioned effects (if any) are applied to a new, unprocessed video signal to generate a broadcast video signal.

[0067] Optionally, the aforementioned video transitions (if any) are applied to the new raw video signal to generate the broadcast video signal. More specifically, the video transitions are integrated into the broadcast video signal before the use of the new raw video signal to generate the broadcast video signal begins (the video transitions are inserted between the previously used raw video signal and the new raw video signal).

[0068] The server 100 then sends a command 9 to the edge device 300 (for example, edge device (1) in Figure 4) that was previously used as a source for broadcasting, to stop transmitting the corresponding raw video signal 3.

[0069] Upon receiving command 9, the edge device 300 (for example, edge device (1) in Figure 4) stops transmitting the raw video signal 3 to the server 100 if the operator is switching sources.

[0070] In an alternative implementation, upon receiving command 9, edge device 300 (for example, edge device (1) in Figure 4) continues sending the raw video signal to server 100 if the operator is swapping between program mode and preview mode. If the source in preview mode is changed, edge device 300 stops sending the raw video signal to server 100.

[0071] Here, we refer to Figures 2, 4, and 5 simultaneously. Figure 5 shows the video flow after the transition from edge device (1) to edge device (2) as a source for broadcast (shown in Figure 4) is complete.

[0072] Upon receiving command 6, the generation of the broadcast video signal based on the raw video signal transmitted 8 by the edge device (2) and the transition to transmission 4 must be as fast as possible (e.g., less than 20 milliseconds).

[0073] In existing (conventional) live video broadcasting systems, transitions are structurally fast because each edge device 300 constantly transmits raw video signals (even if not being broadcast) to the server 100. However, the cost in terms of bandwidth consumption is very high due to the bandwidth required to transmit each raw video signal. This may limit the number of video sources 10 that can be supported by the live video broadcasting system.

[0074] In contrast, using the live video broadcasting system described above, transmitting only the raw video signal being broadcast to server 100 is highly efficient in terms of bandwidth consumption, and this makes it possible to support a large number of video sources 10.

[0075] To limit the duration of the aforementioned transitions, a suitable protocol must be used to perform steps 7 and 8. For example, the NIC 130 of server 100 and the NIC of edge device 300 (not shown in the diagram for simplification purposes) used to perform steps 7 and 8 implement zero-copy networking functionality. For example, the zero-copy networking functionality is Remote Direct Memory Access (RDMA). However, other implementations of zero-copy networking functionality are also considered relevant to this disclosure. Zero-copy networking and RDMA are well known in the art. They provide the ability to directly read raw video signals stored in the memory of edge device 300 and directly copy the raw video signals to the memory 120 of server 100. More specifically, the NIC 130 of server 100 and the NIC of edge device 300 that support zero-copy networking functionality (e.g., RDMA) provide the ability to perform data transfer (e.g., raw video signals) by directly accessing the respective memories of server 100 and edge gateway 300 without using a processor, cache memory, operating system, kernel network stack, etc. Therefore, the network performance of data transfer is improved (e.g., better throughput, lower latency). This disclosure is not limited to zero-copy networking features such as RDMA. Other transport protocols that support low-latency IP streams can also be used to limit the duration of the aforementioned transitions.

[0076] With regard to ULL video streams, Web Real-Time Communication (WebRTC) is an example of a protocol suite that can be used to implement the transmission of ULL video streams. It is well known in the art that WebRTC is adapted to support effective real-time peer-to-peer communication for the exchange of voice, video, and other types of data. More generally, the following protocols, namely Internet Protocol (IP), Transmission Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP), and alternatively or complementaryly IP, User Datagram Protocol (UDP), and Real-time Transport Protocol (RTP), are generally relied upon for the transmission of ULL video streams. Those skilled in the art will readily understand that other Internet communication protocols may be used.

[0077] Here, we refer to Figures 5 and 6B simultaneously. Figure 6B is similar to Figure 6A and represents the screen 250 of user device 200. Figure 6B corresponds to the configuration shown in Figure 5 after the transition from video source (1) to video source (2) as the source for broadcast. The ULL video stream transmitted by server 100 and displayed on screen 250 corresponds to the current broadcast video signal, which here uses video source (2) as the source for broadcast. The visual indication 255 here identifies the ULL video stream corresponding to video source (2) as the source for broadcast.

[0078] Here, we refer to Figures 1, 2, and 7 simultaneously. Figure 7 corresponds to the live video broadcasting system shown in Figure 1, where the functions of switch 60 in Figure 1 are integrated into server 100. More specifically, virtual switch 160 is executed by processing unit 110 of server 100. Virtual switch 116 is implemented by a computer program having instructions executed by processing unit 110.

[0079] Server 100 uses a virtual switch 116 to implement networking functions provided by the IP networking infrastructure 50 (including switch 60) in Figure 1, and is directly connected to at least some of the other devices (edge ​​device 300, user device 200, and broadcast equipment 20). However, Server 100 may still need to use the IP networking infrastructure 50 to communicate with some of the other devices. In an exemplary implementation, Server 100 is directly connected to Edge Device 300 via a virtual switch 160, but still uses the IP networking infrastructure 50 to communicate with User Device 200 and Broadcast Equipment 20.

[0080] The virtual switch 160 supports and optionally optimizes the reception and / or transmission of at least one of the following via the NIC 130 of the server 100: the raw video signal transmitted by the edge device 300, the ULL video stream transmitted to the user device 200, and the broadcast video stream transmitted to the broadcast equipment 20.

[0081] Furthermore, multiple virtual switches 160 may be run simultaneously by the processing unit 110 of the server 100. In an exemplary implementation, each time a new user device 200 interconnects with the server 100, a new instance of the virtual switch 160 dedicated to this new user device 200 is launched. In another exemplary implementation, a pool of virtual switches 160 is available, providing redundancy and load balancing. Optionally, each virtual switch 160 is dedicated to a given set of edge devices 300.

[0082] With respect to NIC130, in the first embodiment, each NIC130 is dedicated to a given virtual switch 160. In another exemplary embodiment, a pool of NIC130s is available and can be allocated on demand to a given virtual switch 160, thereby providing redundancy and load balancing. Optionally, each NIC130 is dedicated to a given set of edge devices 300.

[0083] Refer to Figures 1, 8A, and 8B. Figures 8A and 8B correspond to the live video broadcasting system shown in Figure 1, where the server 100 and optionally the edge device 300 are localized in the cloud infrastructure 70.

[0084] As is well known in this technology, the cloud infrastructure 70 typically comprises numerous general-purpose computing devices (not shown in Figures 8A and 8B for simplification purposes) managed by a cloud management platform. The cloud management platform typically comprises dedicated hardware and software components. The cloud infrastructure 70 provides a variety of cloud services, including third-party software, security, redundancy, load balancing, and easy-to-scalable deployment capabilities such as effective communication (with external devices via internal and external communication networks to the cloud infrastructure 70).

[0085] General-purpose computing devices belonging to the cloud infrastructure 70 can provide the aforementioned cloud services. Furthermore, by deploying and running dedicated third-party software, the general-purpose computing devices can be customized to provide corresponding third-party services / functions. For example, in the context of a current live video broadcasting system, one or more general-purpose computing devices in the cloud infrastructure 70 are configured to perform the aforementioned functions of the cloud-based server 100. Similarly, one or more general-purpose computing devices in the cloud infrastructure 70 are configured to perform the aforementioned functions of each cloud-based edge device 300 (Figure 8A). Alternatively, only the server 100 is deployed in the cloud infrastructure 70, but the edge devices 300 are not (Figure 8B).

[0086] User devices 200 and / or other components of the live video broadcasting system, such as broadcasting equipment 20, may be integrated into the cloud infrastructure 70. Furthermore, the server 100 having the virtual switch function 160 shown in Figure 7 can also be implemented by the cloud-based server 100 shown in Figures 8A and 8B.

[0087] In the context of cloud infrastructure, protocols specifically designed to operate within the cloud infrastructure may be used to reduce latency when switching from one video source to another as a source for broadcast. For example, libfabric (also known as the Open Fabric Interface (OFI)) can be used in this context.

[0088] Here, we refer to Figures 1, 2, 9A, and 9B simultaneously. Figures 9A and 9B represent a method 400 for producing a live video broadcast from multiple video sources. Method 400 is performed by a processing unit 110 of server 100.

[0089] A dedicated computer program has instructions for performing at least some of the steps of method 400. The instructions are contained in a non-temporary computer-readable medium (e.g., the memory 120 of server 100). When executed by the processing unit 110 of server 100, the instructions provide for producing a live video broadcast from multiple video sources. The instructions can be delivered to server 100 via an electronically readable medium such as a storage medium (e.g., a CD-ROM, a USB key, etc.) or via a communication link (e.g., via a communication network through the NIC 130 of server 100).

[0090] Method 400 includes step 405 of receiving a first raw video signal from a first edge device 300 (e.g., edge device (1)) connected to a first video source 10 (e.g., video source (1)), the first raw video signal being generated by the first edge device 300 based on the first video signal received from the first video source 10. Step 405 is performed by a processing unit 110. This step has been described in detail above in relation to Figure 3.

[0091] Method 400 includes step 410 of processing a first raw video signal to generate a broadcast video signal based on the first raw video signal. Step 410 is performed by a processing unit 110. This step was described in detail above in relation to Figure 3.

[0092] Method 400 includes step 415 of transmitting a broadcast video signal to the video broadcasting equipment 20 (via one of the NICs 130). Step 415 is performed by the processing unit 110. This step was described in detail above in relation to Figure 3.

[0093] Method 400 includes step 420 of processing a broadcast video signal to generate an ultra-low latency (ULL) video stream based on the broadcast video signal. Step 420 is performed by a processing unit 110. This step was described in detail above in relation to Figure 3.

[0094] Method 400 includes step 425 of sending the ULL video stream to the user device 200 (via one of the NICs 130). Step 425 is performed by the processing unit 110. This step was described in detail above in relation to Figure 3.

[0095] Method 400 includes step 430 of receiving a command from user device 200 (via one of the NICs 130) to switch to a second video source. Step 430 is performed by processing unit 110. This step was described in detail above in relation to Figure 4.

[0096] Method 400 includes step 435, which involves sending a command (via one of the NICs 130) to a second edge device 300 (e.g., edge device (2)) connected to a second video source 10 (e.g., video source (2)) to initiate the transmission of a second raw video signal generated by the second edge device 300 based on a second video signal received from the second video source 10. Step 435 is performed by the processing unit 110. This step was described in detail above in relation to Figure 4.

[0097] Method 400 includes step 440 of receiving a second raw video signal from a second edge device 300 (e.g., edge device (2)). Step 440 is performed by a processing unit 110. This step was described in detail above in relation to Figure 4.

[0098] Method 400 includes step 445 of processing a second raw video signal to generate a broadcast video signal based on the second raw video signal instead of the first raw video signal. Step 445 is performed by a processing unit 110. This step was described in detail above in relation to Figure 4.

[0099] After step 445, the broadcast video signal continues to be transmitted according to step 415, the ULL video stream continues to be generated according to step 420, and the ULL video stream continues to be transmitted according to step 425.

[0100] Method 400 includes step 450 of sending a command to a first edge device 300 (e.g., edge device (2)) connected to a first video source (e.g., video source (1)) to stop transmitting the first raw video signal. Step 455 is performed by the processing unit 110. This step was described in detail above in relation to Figure 4.

[0101] It should be noted that some of the steps in Method 400 may be performed in a different order. The order of the steps shown in Figures 9A and 9B does not necessarily represent the time frame in which the steps are performed. For example, step 420 may be performed before step 415.

[0102] Now, refer to Figure 10. Figure 10 is a detailed representation of the components of the user device 200. The user device 200 comprises a processing unit 210, memory 220, at least one network interface card (NIC) 230, a user interface 240, and a screen 250. Some of the components have been described above in relation to Figures 1 and 2. All of the components of the user device 200 are well known in the art.

[0103] Here, we refer to Figures 3, 4, 5, 6A, 6B, 10, and 11 simultaneously. Figure 11 illustrates method 500 for performing zero-delay transitions between broadcast sources. Method 500 is performed by the processing unit 210 of the user device 200.

[0104] A dedicated computer program has instructions for performing at least some of the steps of method 500. The instructions are contained in a non-temporary computer-readable medium (e.g., the memory 220 of the user device 200). When the instructions are executed by the processing unit 210 of the user device 200, they provide the execution of zero-delay transitions between broadcast sources. The instructions can be delivered to the server 100 via an electronically readable medium such as a storage medium (e.g., a CD-ROM, a USB key, etc.) or via a communication link (e.g., via a communication network through the NIC 230 of the user device 200).

[0105] Method 500 includes step 505 of receiving multiple ULL video streams from multiple edge devices 300 (via one of the NICs 230), each ULL video stream being generated by one of the multiple edge devices 300 based on a video signal received from a video source connected to the edge device 300. Step 505 is performed by the processing unit 210. This step is described in detail above in relation to Figure 3, where the multiple ULL video streams are identified by reference number 2.

[0106] Method 500 includes step 510 of displaying multiple ULL video streams on screen 250. Step 510 is performed by processing unit 210. This step was described in detail above in relation to Figure 6A.

[0107] Method 500 includes step 515 of displaying a visual indication 255 on screen 250, the visual indication 255 identifying a ULL video stream corresponding to a video source 10 (e.g., video source (1) in Figure 6A) currently being used as a source for broadcast. Step 515 is performed by a processing unit 210, which has been described in detail above in relation to Figure 6A.

[0108] Method 500 includes step 520 of receiving a ULL broadcast video stream from server 300 (via one of the NICs 230), where the ULL broadcast video stream represents the video signal currently being broadcast by server 300. Step 520 is performed by processing unit 210. This step is described in detail above in relation to Figure 3, where the ULL broadcast video is identified by reference number 5.

[0109] Method 500 includes step 525 of displaying ULL broadcast video on screen 250. Step 525 is performed by processing unit 210. This step was described in detail above in relation to Figure 6A.

[0110] Method 500 includes step 530 of receiving user input (e.g., via user interface 240) to select video source 10 to be used as a new source for broadcast. Step 530 is performed by processing unit 210.

[0111] As stated above, the performance of user actions is outside the scope of this disclosure. Various implementations can be used, as is well known in the art. For example, referring to Figure 6A, the visual indication 255 identifies video source (1) as the current source for broadcast. If the user wishes to select video source (2) as a new source for broadcast, the user can simply click on the ULL video stream from the displayed video source (2).

[0112] Method 500 includes step 535 of displaying a visual indication 255 on screen 250, the visual indication 255 identifying a ULL video stream corresponding to the video source 10 selected as a new source for broadcast (in step 530) (for example, video source (2) instead of video source (1) in Figure 6B). Step 535 is performed by processing unit 210, which has been described in detail above in relation to Figure 6B.

[0113] Method 500 includes step 540, which sends a command to server 100 (via one of the NICs 230) to switch the source for broadcast to the video source 10 (e.g., video source 2) selected in step 530. Step 540 is performed by processing unit 210. This step is described in detail above in relation to Figure 4, and the command is identified by reference number 6.

[0114] As mentioned above, the command includes identifying the selected video source to be used as a new source for broadcasting (or identifying the raw video signal generated by an edge device connected to the selected video source).

[0115] Now, refer to Figure 12. Figure 12 is based on Figure 3 and represents an alternative implementation.

[0116] All ULL video streams generated by the edge device 300 are sent to the server 100 (instead of the user device 200 shown in Figure 3). The server 100 combines all ULL video streams received from the edge device 300 into a single combined ULL video stream. For example, individual ULL video streams are combined into a scaled ULL video stream that is sent to the user device 100. The combined ULL video stream is sent to the user device 200 and displayed on the user device 200's screen.

[0117] While this disclosure has been described above in this specification by its non-limiting and exemplary embodiments, these embodiments may be modified at will within the scope of the appended claims without departing from the spirit and essence of this disclosure.

Claims

1. A method for producing a live video broadcast from multiple video sources, A step of receiving a first raw video signal from a first edge device connected to a first video source in a server, wherein the first raw video signal is generated by the first edge device based on a first video signal received from the first video source. The steps include: processing the first raw video signal by the server to generate a broadcast video signal based on the first raw video signal; The steps include: transmitting the broadcast video signal to the video broadcasting equipment via the server; The steps include: processing the broadcast video signal by the server to generate an ultra-low latency (ULL) video stream based on the broadcast video signal; The steps include: transmitting the ULL video stream to the user device via the server; The steps include: the server receiving a command from the user device to switch to a second video source; The steps include: sending a command by the server to a second edge device connected to the second video source to initiate the transmission of a second raw video signal generated by the second edge device based on a second video signal received from the second video source; The steps include: receiving the second raw video signal from the second edge device by the server; The steps include: processing the second raw video signal by the server to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal; The steps include: sending a command via the server to the first edge device connected to the first video source to stop the transmission of the first raw video signal; Methods that include...

2. The method according to claim 1, wherein the server performs zero-copy networking functions for transmitting the first raw video signal from the first edge device to the server and for transmitting the second raw video signal from the second edge device to the server.

3. The method according to claim 2, wherein the server comprises at least one network interface card (NIC) that supports the zero-copy networking function.

4. The method according to claim 2, wherein the zero-copy networking is remote direct memory access (RDMA).

5. The method according to claim 1, wherein the server performs a virtual switch function.

6. The method according to claim 1, wherein the server is a cloud-based server located in a cloud infrastructure.

7. The method according to claim 6, wherein the first edge device and the second edge device are cloud-based edge devices also deployed in the cloud infrastructure.

8. The method according to claim 1, further comprising at least one of the steps of: applying one or more effects to the first raw video signal or the second raw video signal to generate the broadcast video signal; and applying a video transition between the first raw video signal and the second raw video signal to generate the broadcast video signal.

9. The steps include: receiving a plurality of ULL video streams from a plurality of edge devices by the server, wherein each ULL video stream is generated by one of the plurality of edge devices based on a video signal received from a video source connected to the edge device, and the plurality of edge devices comprises the first edge device and the second edge device; The steps include: combining the multiple ULL video streams into a combined ULL video stream using the server; The steps include: transmitting the synthesized ULL video stream from the server to the user device; The method according to claim 1, further comprising:

10. A non-temporary computer-readable medium containing instructions that can be executed by a server's processing unit, wherein the execution of the instructions by the server's processing unit is The server receives a first raw video signal from a first edge device connected to a first video source, wherein the first raw video signal is generated by the first edge device based on a first video signal received from the first video source. The first raw video signal is processed by the server to generate a broadcast video signal based on the first raw video signal, The broadcast video signal is transmitted to the video broadcasting equipment by the server, The broadcast video signal is processed by the server to generate an ultra-low latency (ULL) video stream based on the broadcast video signal, The server transmits the UL video stream to the user device, The server receives a command from the user device to switch to a second video source, The server sends a command to a second edge device connected to the second video source to initiate the transmission of a second raw video signal generated by the second edge device based on a second video signal received from the second video source. The server receives the second raw video signal from the second edge device, The server processes the second raw video signal to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal, The server sends a command to the first edge device connected to the first video source to stop the transmission of the first raw video signal. A non-temporary, computer-readable medium that provides the ability to produce live video broadcasts from multiple video sources.

11. A server for producing live video broadcasts from multiple video sources, A network interface card (NIC) for receiving and transmitting data, A processing unit, Receiving a first raw video signal from a first edge device connected to a first video source, wherein the first raw video signal is generated by the first edge device based on a first video signal received from the first video source. The process involves processing the first raw video signal to generate a broadcast video signal based on the first raw video signal, Transmitting the aforementioned broadcast video signal to video broadcasting equipment, The first raw video signal is processed to generate an ultra-low latency (ULL) video stream based on the broadcast video signal, Transmitting the ULL video stream to the user device, Receiving a command from the user device to switch to a second video source, Sending a command to a second edge device connected to the second video source to initiate the transmission of a second raw video signal generated by the second edge device based on a second video signal received from the second video source, Receiving the second raw video signal from the second edge device, The process involves processing the second raw video signal to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal, Sending a command to the first edge device connected to the first video source to stop the transmission of the first raw video signal and Processing unit and A server equipped with the following features.

12. The server according to claim 11, wherein the server performs zero-copy networking functions for transmitting the first raw video signal from the first edge device to the server and for transmitting the second raw video signal from the second edge device to the server.

13. The server according to claim 12, wherein one or more of the at least one NICs support the zero-copy networking function.

14. The server according to claim 12, wherein the zero-copy networking is Remote Direct Memory Access (RDMA).

15. The server according to claim 11, wherein the processing unit performs a virtual switch function.

16. The server according to claim 11, wherein the server is a cloud-based server located on a cloud infrastructure.

17. The server according to claim 11, wherein the processing unit further performs at least one of the following: applying one or more effects to the first raw video signal or the second raw video signal to generate the broadcast video signal; and applying a video transition between the first raw video signal and the second raw video signal to generate the broadcast video signal.

18. The aforementioned processing unit Receiving multiple ULL video streams from multiple edge devices, wherein each ULL video stream is generated by one of the multiple edge devices based on a video signal received from a video source connected to the edge device, and the multiple edge devices comprise the first edge device and the second edge device. The process involves combining the aforementioned multiple ULL video streams into a combined ULL video stream, The synthesized ULL video stream is transmitted to the user device. The server according to claim 11, further performing the following:

Images