Method and server for producing live video broadcasting from a plurality of video sources

Abstract

Method and server for producing live video broadcasting from several video sources. The server receives a first raw video signal from a first edge device connected to a first video source. The server processes the first raw video signal to generate a broadcast video signal and an ultra low latency (ULL) video stream, which are respectively transmitted to a video broadcasting equipment and a user device. The server receives a command for switching to a second video source and transmits a command to start transmitting a second raw video signal from a second edge device connected to the second video source. The server receives the second raw video signal from the second edge device and generates the broadcast video signal and the ULL video stream based on the second instead of the first raw video signal. The server transmits a command to stop transmitting the first raw video signal.

Description

METHOD AND SERVER FOR PRODUCING LIVE VIDEO BROADCASTING FROM A PLURALITY OF VIDEO SOURCESCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 525,015, filed on July 5, 2023, titled “METHOD AND SYSTEM FOR PRODUCING LIVE VIDEO BROADCASTING FROM A PLURALITY OF REMOTE VIDEO EQUIPMENT,” the entire disclosure of which is incorporated by reference herein.TECHNICAL FIELD

[0002] The present disclosure relates to the field of live video broadcasting. More specifically, the present disclosure relates to a method and server for producing live video broadcasting from a plurality of video sources.BACKGROUND

[0003] The video broadcasting industry is an industry in which multiple technological developments and innovations have been occurring in the past years. One of the most significant trends is the transition from proprietary technologies which have been developed specifically for the video broadcasting industry, to more generic technologies which have been deployed and validated in other industries. In particular, proprietary communication protocols are replaced with Internet based protocols. One advantage of using Internet based protocols is that the cost of deployment is generally lower. Another advantage is the capability to benefit from evolutions and development of new functionalities (e.g. evolution of existing communication protocols and I or development of new communication protocols). In particular, innovations in a domain not related to the video broadcasting industry may still be applicable and useful to the video broadcasting industry. Still another advantage is the high level of standardization in the development of Internet protocols, which ensures interoperability between equipment of different manufacturers.

[0004] One major challenge for implementing a live video broadcasting system is to provide the capability to transmit a large amount of data in real-time (or at least quasi real-time). For instance, in a typical live video broadcasting setup, a plurality of cameras is operating in parallel, providing different views of an event being filmed. An operator in charge of controlling in real-time the live video broadcast needs to have access to and control the video signals generated by the multiple cameras. More specifically, the operator needs to have information (e.g. images) representative of the video signals generated by each camera, so as to select in real-time which camera should be used for the live broadcast. Furthermore, the video signal generated by the selected camera needs to be transmitted to a video broadcasting equipment in an efficient manner (to minimize latency, avoid data loss, etc.). Thus, the video signal generated by a given camera shall always be available and ready to be effectively transmitted to the live video broadcasting equipment, upon selection by the operator.

[0005] There is therefore a need for a new method and server for producing live video broadcasting from a plurality of video sources.SUMMARY

[0006] According to a first aspect, the present disclosure relates to a method for producing live video broadcasting from a plurality of video sources. The method comprises receiving at a server a first raw video signal from a first edge device connected to a first video source, the first raw video signal being generated by the first edge device based on a first video signal received from the first video source. The method comprises processing by the server the first raw video signal to generate a broadcast video signal based on the first raw video signal. The method comprises transmitting by the server the broadcast video signal to a video broadcasting equipment. The method comprises processing by the server the broadcast video signal to generate an ultra low latency (ULL) video stream based on the broadcast video signal. The method comprises transmitting by the server the ULL video stream to a user device. The method comprises receiving by the server from the user device a command for switching to a second video source. The method comprises transmitting by the server a command to asecond edge device connected to the second video source to start transmitting a second raw video signal generated by the second edge device based on a second video signal received from the second video source. The method comprises receiving by the server the second raw video signal from the second edge device. The method comprises processing by the server the second raw video signal to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal. The method comprises transmitting by the server a command to the first edge device connected to the first video source to stop transmitting the first raw video signal.

[0007] According to a second aspect, the present disclosure relates to a non-transitory computer readable medium comprising instructions executable by a processing unit of a server. The execution of the instructions by the processing unit of the server provides for producing live video broadcasting from a plurality of video sources by implementing the aforementioned method.

[0008] According to a third aspect, the present disclosure relates to a server for live video broadcasting production from a plurality of video sources. The server comprises at least one Network Interface Card (NIC) for receiving and transmitting data, and a processing unit. The processing unit receives a first raw video signal from a first edge device connected to a first video source, the first raw video signal being generated by the first edge device based on a first video signal received from the first video source. The processing unit processes the first raw video signal to generate a broadcast video signal based on the first raw video signal. The processing unit transmits the broadcast video signal to a video broadcasting equipment. The processing unit processes the broadcast video signal to generate an ultra low latency (ULL) video stream based on the broadcast video signal. The processing unit transmits the ULL video stream to a user device. The processing unit receives from the user device a command for switching to a second video source. The processing unit transmits a command to a second edge device connected to the second video source to start transmitting a second raw video signal generated by the second edge device based on a second video signal received from the second video source. The processing unit receives the second raw video signal from the second edge device. The processing unitprocesses the second raw video signal to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal. The processing unit transmits a command to the first edge device connected to the first video source to stop transmitting the first raw video signal.

[0009] In a particular aspect, the server implements a zero-copy networking functionality for transmitting the first raw video signal from the first edge device to the server and for transmitting the second raw video signal from the second edge device to the server. In a particular embodiment, the server comprises at least one Network Interface Card (NIC) supporting the zero-copy networking functionality. In another particular embodiment, the zero-copy networking is Remote Direct Memory Access (RDMA).

[0010] In another particular aspect, the server executes a virtual switch functionality.

[0011] In still another particular aspect, the server is a cloud-based server located in a cloud infrastructure. In a particular embodiment, the first and second edge devices are cloud-based edge devices also located in the cloud infrastructure.

[0012] In yet another particular aspect, at least one of the following is performed by the processing unit of the server: applying one or more effects to the first raw video signal or to the second raw video signal to generate the broadcast video signal; and applying a video transition between the first raw video signal and the second raw video signal to generate the broadcast video signal.

[0013] In another particular aspect, the processing unit of the server receives a plurality of ULL video streams from a plurality of edge devices, each ULL video stream being generated by one of the plurality of edge devices based on a video signal received from a video source connected to the edge device, the plurality of edge devices comprising the first edge device and the second edge device. The processing unit of the server combines the plurality of ULL video streams into a combined ULL video stream, and transmits the combined ULL video stream to the user device.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Embodiments of the disclosure will be described by way of example only with reference to the accompanying drawings, in which:

[0015] Figure 1 represents a live video broadcasting system comprising a server, a user device and a plurality of video equipment;

[0016] Figure 2 represents components of the server illustrated in Figure 1 ;

[0017] Figures 3, 4 and 5 represent interactions between the server, the user device and the plurality of video equipment of Figure 1 ;

[0018] Figures 6A and 6B represent video streams displayed on a screen of the used device;

[0019] Figure 7 represents an alternative implementation of the live video broadcasting system of Figure 1 with the server implementing a virtual switch functionality;

[0020] Figures 8A and 8B represent an alternative implementation of the live video broadcasting system of Figure 1 with the serverand optionally the edge devices illustrated in Figure 1 localized in a cloud infrastructure;

[0021] Figures 9A and 9B represent a method for producing live video broadcasting from a plurality of video sources;

[0022] Figure 10 represents components of the user device illustrated in Figure 1 ;

[0023] Figure 11 represents a method for performing a zero-delay transition between broadcasting sources; and

[0024] Figure 12 represents an alternative implementation of the signaling illustrated in Figure 3.DETAILED DESCRIPTION

[0025] The foregoing and other features will become more apparent upon reading of the following non-restrictive description of illustrativeembodiments thereof, given by way of example only with reference to the accompanying drawings. Like numerals represent like features on the various drawings.

[0026] Various aspects of the present disclosure generally address one or more of the problems related to the transport and processing of video signals generated by a plurality of video sources (e.g. cameras) in an Internet protocol (IP) based live video broadcasting system. More specifically, the present disclosure addresses efficient transport of the video signals between equipment of the live video broadcasting system and selection I control of the video signal (originating from one of the video sources) to be transmitted to a video broadcasting equipment (for being broadcasted).

[0027] Reference is now made concurrently to Figures 1 and 2. A live video broadcasting system is represented in Figure 1. The live video broadcasting system comprises a plurality of components interacting with each other, to support live video broadcasting sessions. The components illustrated in Figure 1 include a server 100, a user device 200, a switch 60, a plurality of edge devices 300, a plurality of video sources 10 and a broadcasting equipment 20. A detailed description of the components of the server 100 is provided in Figure 2.

[0028] The server 100 is a central entity in charge of supporting operations of the live video broadcasting system. The server is implemented by a single computing device (e.g. a single computer) or a group of computing devices (e.g. a group of computers cooperating to implement the functionalities of the server 100). The server 100 communicates with the user device 200 through an Internet Protocol (IP) networking infrastructure 50. Although a single user device 200 is represented in Figure 1 , the server 100 is capable of communicating with a plurality of user devices 200. The server 100 also communicates with the edge devices 300 and the broadcasting equipment 20 through the IP networking infrastructure 50.

[0029] The switch 60 is part of the IP networking infrastructure 50 interconnecting the server 100 with the user device 200, the edge devices 300 and the broadcasting equipment 20. In an exemplary configuration, at least someof the communications of the server 100 with the other equipment go through the switch 60. As is well known in the art of IP networking, the IP networking infrastructure 50 may include more than one networking equipment (e.g. additional switch(es) and I or router(s)) to provide interconnectivity of the server 100 with the other equipment (e.g. 200, 300 and 30). In the rest of the description, when referring to the IP networking infrastructure 50, it will be implied that it comprises networking equipment such as the switch 60.

[0030] Examples of video sources 10 include, but are not limited to, cameras. Although Figure 1 represents each video source 10 being connected to a single edge device 300 for simplification purposes, a given video source 10 may be connected to more than one edge device 300 for redundancy purposes. For example, if the given video source 10 is connected to two different edge devices 300, and one of the edge devices 300 fails, the other edge device is still available for transmitting video flow(s) generated based on the video signal of the given video source 10 through the networking infrastructure 50. Figure 1 illustrates an exemplary configuration with three video sources 10 referred to as video source (1 ), video source (2) and video source (3) and three respective corresponding edge devices 300 referred to as edge device (1 ), edge device (2) and edge device (3). Furthermore, more than one video source 10 may be connected to the same edge device 300. In this case, the processing applied by the edge device 300 to the video signal generated by a single video source 10 (which will be described later in the description) can be applied to each of the video sources 10 connected to the same edge device 300. Similarly, although a single broadcasting equipment 20 is represented in Figure 1 for simplification purposes, a plurality of broadcasting equipment 20 may be controlled by the server 100.

[0031] The live video broadcasting system illustrated in Figure 1 is capable of supporting various types of video sources 10 generating various types of video signals. Examples of video signals generated by the video sources 10 comprise Serial Digital Interface (SDI) video signals, High-Definition Multimedia Interface (HDMI) video signals, video signals compliant with the Society of Motion Picture & Television Engineers (SMPTE) 2110 standard, etc. Each video source 10 is physically connected to the corresponding edge device(s) 300 through acable compatible with the type of video signal generated by the video source 10. Alternatively, a wireless connection is used between the video source 10 and the edge device 300. Details related to the components of the video sources 10 are well known in the art and are out of the scope of the present disclosure.

[0032] The live video broadcasting system illustrated in Figure 1 is capable of supporting various types of broadcasting equipment 20 capable of broadcasting various types of video signals. The types of video signals supported by the broadcasting equipment 20 are similar to the type of video signals supported by the video sources 10. The broadcasting equipment 20 broadcasts video signals on a large-scale infrastructure such as a cable television network, a satellite television network, an Internet based video streaming infrastructure, etc. Optionally, an edge device 300 similar to the one used for the video sources 10 is used for converting video flows received from the server 100 into video signals that the broadcasting equipment 20 is capable of processing. Details related to the components of the broadcasting equipment 20 are well known in the art and are out of the scope of the present disclosure.

[0033] Details of the components of the server 100 are illustrated in Figure 2. The server 100 comprises a processing unit 110, memory 120, at least one Network Interface Card (NIC) 130. Optionally, the server 100 comprises additional components, such as a user interface 140, a screen 150, etc.

[0034] The processing unit 110 comprises one or more processor (not represented in Figure 2 for illustration purposes) capable of executing instructions of a computer program. Each processor may further comprise one or several cores. The processing unit may also include one or more dedicated processing components (e.g. a network processor, an Application Specific Integrated Circuits (ASIC), etc.) for performing specialized functions (e.g. video processing functions, networking functions, etc.).

[0035] The memory 120 stores instructions of computer program(s) executed by the processing unit 110, data generated by the execution of the computer program(s) by the processing unit 1 10, data received via the NIC(s) 130, etc. Only a single memory 120 is represented in Figure 2, but the server 100may comprise several types of memories, including volatile memory (such as Random Access Memory (RAM)) and non-volatile memory (such as a hard drive, Erasable Programmable Read-Only Memory (EPROM), Electrically-Erasable Programmable Read-Only Memory (EEPROM), etc.).

[0036] Each NIC 130 allows the server 100 to exchange data with other devices (a first NIC and a second optional NIC are represented in Figure 2 for illustration purposes only) over the IP networking infrastructure 50. Examples of NIC 130 of the wireline type include standard (electrical) Ethernet ports, fiberoptic ports, ports adapted for receiving Small Form-factor Pluggable (SFP) units, etc. The NIC 130 may also be of the wireless type (e.g. a Wi-Fi interface). The NIC 130 comprises a combination of hardware and software executed by the hardware, for implementing the communication functionalities of the NIC 130. Alternatively, the combination of hardware and software for implementing the communication functionalities of the NIC130 is at least partially included in the processing unit 110.

[0037] With respect to the user device 200, various types of user devices 200 are considered, such as a computer, a smartphone, a tablet, etc. Details of the user device 200 are not illustrated in Figure 1 for simplification purposes (and will be detailed later in relation to Figure 10). Components of the user device 200 are similar to the components of the server 100 illustrated in Figure 2. The user device 200 generally comprises a processing unit, memory, one or more NICs, a user interface, and a screen. The processing unit comprises at least one processor capable of executing instructions of computer programs. At least one of the NIC(s) is adapted for receiving and transmitting data over the IP networking infrastructure 50.

[0038] With respect to the edge device 300, several implementations are supported by the present disclosure. Details of the edge device 300 are not illustrated in Figure 1 for simplification purposes. In a first exemplary implementation, the edge device 300 is a computing device with components similar to the components of the server 100 illustrated in Figure 2. The edge device 300 generally comprises a processing unit, memory, and one or moreNICs. The processing unit comprises at least one processor capable of executing instructions of computer programs. The processing unit may also include one or more dedicated processing components (e.g. a network processor, an Application Specific Integrated Circuits (ASIC), etc.) for performing specialized functions (e.g. video processing functions, networking functions, etc.). At least one of the NIC(s) is adapted for receiving and transmitting data over the IP networking infrastructure 50, to interact with the server 100 and the user device 200. The exchange of data (video signal) between the edge device 300 and the video source 10 can be performed via one of the NIC(s) or via a dedicated video interface.

[0039] In a second exemplary implementation, the edge device 300 is a standardized hot-pluggable transceiving unit, such as a Small Form-factor Pluggable (SFP) unit. The transceiving unit comprises a processing unit, a front connector adapted for receiving the video signal(s) generated and transmitted by the video source(s) 10 to which the transceiving unit is connected. The transceiving unit also comprises a front connector implementing a NIC adapted for exchanging data with the server 300. The transceiving unit further comprises a housing adapted to being inserted into a chassis of a hosting unit. Thus, a plurality of transceiving units respectively implementing edge device(s) 300 can be inserted in a single chassis.

[0040] Reference is now made concurrently to Figures 2 and 3. Figure 3 corresponds to the live video broadcasting system illustrated in Figure 1.

[0041] Each video source 10 transmits 1 its video signal to the corresponding edge device 300. For illustrations purposes, we consider that the video signal generated by video source (1 ) is currently broadcasted, the video signals generated by video sources (2) and (3) not being broadcasted.

[0042] Based on the received video signal, each edge device 300 generates an Ultra Low Latency (ULL) video stream and a raw video signal. The edge device implements one or more video processing capabilities (e.g.decoding, encoding, decompression, compression, sub-sampling, etc.) for generating the ULL video stream and the raw video signal.

[0043] The ULL video stream is a low-quality video stream requiring significantly less bandwidth than the raw video signal for transmission over the IP networking infrastructure 50 (e.g. by a factor of magnitude of 100 or even possibly 1000). For example, the generation of the ULL video stream includes (for illustration purposes only) decoding the video signal transmitted by the video source 10, optionally sub-sampling the video signal, and compressing the video signal. The implementation of the generation of the ULL video stream based on the video signal received from the video source 10 is out of the scope of the present disclosure, and may vary for example based on characteristics of the video signal received from the video source 10, etc.

[0044] All the ULL video streams generated by the edge devices 300 are transmitted 2 to the user device 200. At the user device 200, the ULL video streams are displayed on the screen of the user device 200, as will be illustrated later in the description. Alternatively, only some of the edge devices 300 generate and transmit the ULL video streams (e.g. based on a command requesting transmission of the ULL video streams sent by the user device 200 to only some of the edge devices 300).

[0045] As mentioned previously, the raw video signal is a high-quality video signal requiring significantly more bandwidth than the ULL video streams for transmission over the IP networking infrastructure 50. For example, the generation of the raw video signal includes (for illustration purposes only) decoding the video signal transmitted by the video source 10. The implementation of the generation of the raw video signal based on the video signal received from the video source 10 is out of the scope of the present disclosure, and may vary for example based on characteristics of the video signal received from the video source 10, etc.

[0046] Only one raw video signal generated by one of the edge devices 300 (e.g. edge device (1 ) in Figure 3) is transmitted 3 to the server 100. At theserver 100, the raw video signal is processed, as will be illustrated later in the description.

[0047] In a first exemplary implementation, all the edge devices 300 respectively generate the raw video signal based on the video signal received from the video sources 10, but only one edge device 300 is transmitting to the server 100. The other edge devices 300 are in standby mode, ready to transmit the raw video signal upon reception of a command from the server 100 to start the transmission.

[0048] In a second exemplary implementation, only the edge device 300 transmitting the raw video signal to the server 100 generates the raw video signal based on the video signal received from the video source 10. The other edge devices 300 are on standby mode, ready to generate and transmit the raw video signal upon reception of a command from the server 100 to start the transmission.

[0049] Based on the raw video signal received from the edge device 300 (e.g. edge device (1 ) in Figure 3), the server 100 generates a broadcast video signal. Then, the server 100 generates an ULL video stream based on the broadcast video signal. The server 100 implements one or more video processing capabilities (e.g. decoding, encoding, decompression, compression, subsampling, etc.) for generating the broadcast video signal.

[0050] The broadcast video signal is a high-quality video stream requiring significantly more bandwidth than the ULL video stream for transmission over the IP networking infrastructure 50. For example, the generation of the broadcast video signal includes (for illustration purposes only) decoding the raw video signal transmitted by the edge device 300, optionally applying one or more effects to the raw video signal, etc. The one or more effects to apply can be preconfigured at the server 100, transmitted by the user device 200, etc. The implementation of the generation of the broadcast video signal based on the raw video signal received from the edge device 300 is out of the scope of the present disclosure, and may vary for example based on characteristics of the raw video signal received from the edge device 300, based on characteristics andcapabilities of the broadcasting equipment 20, based on a pre-configuration at the server 100, based on information transmitted by the user device 200, etc.

[0051] The broadcast video signal is transmitted 4 to the broadcasting equipment 20. At the broadcasting equipment 20, the broadcast video signal is processed (e.g. decoded) for being broadcasted on a video broadcasting infrastructure (not represented in Figure 3). The processing performed by the broadcasting equipment 20 is well known in the art and out of the scope of the present disclosure.

[0052] The broadcast video signal transmitted to the broadcasting equipment 20 is usually an interoperable video stream, which can be processed by different types of broadcasting equipment 20 from different manufacturers. For this purpose, one or more standardized video protocols are used for transmitting the broadcast video signal, such as Society of Motion Picture and Television Engineers (SMPTE) 2110.

[0053] The ULL video stream generated by the server 100 is similar to the ULL video streams generated by the edge devices 300. The implementation of the generation of the ULL video stream based on the broadcast video signal received from the edge device 300 is out of the scope of the present disclosure, and may vary for example based on characteristics of the raw video signal received from the edge device 300, etc.

[0054] The ULL video stream generated by the server 100 is transmitted 5 to the user device 200. At the user device 200, the ULL video stream is displayed on the screen of the user device 200. This ULL video stream received from the server 100 is representative of the video signal being broadcasted. The ULL video streams received from the edge devices 300 are representative of the video signals received from the video sources 10.

[0055] Reference is now made concurrently to Figures 3 and 6A. Figure 6A represents the screen 250 of the user device 200. As mentioned previously, the ULL video streams transmitted by the edge devices 300 and corresponding to the video sources 10 (respectively video sources (1 ), (2) and (3)) are displayed on the screen 250. Only three ULL video streams arerepresented, but additional ULL video streams may be displayed if additional video sources 10 were used. A visual indication 255 is used (displayed on the screen 250) for identifying the ULL video stream corresponding to the video source 10 (video source (1 ) in Figure 6A) currently used as the source for the broadcasting. The implementation of the visual indication 255 may vary and is out of the scope of the present disclosure.

[0056] In an exemplary implementation (not represented in Figure 6A), the ULL video streams received from the edge devices 300 are scaled (resized) by the user device 200 and displayed on the screen 250 as a single scaled ULL video stream.

[0057] The ULL video stream transmitted by the server 100 (corresponding to the currently broadcast video signal, which uses video source (1 ) as a source in Figure 6A) is also displayed on the screen 250.

[0058] In an alternative implementation, the server 100 does not generate the ULL video stream, which is consequently not transmitted 5 to the user device 200 (and not displayed on the screen 250 of the user device 200).

[0059] Reference is now made concurrently to Figures 2, 3 and 4 Figure 4 corresponds to the live video broadcasting system illustrated in Figure 3. The ULL streams transmitted 2 (in Figure 3) by the edge devices 300 to the user device 200 are not represented in Figure 4 for simplification purposes only (the transmission 2 is not affected by the signaling represented in Figure 4 and is still occurring).

[0060] The user device 200 sends 6 a command to the server 100 to switch the source for the broadcasting. For illustration purposes only, Figure 4 illustrates the server 100 switching the source from video source (1 ) to video source (2). The command comprises an identification of the new video source 10 to be used as the source for the broadcasting. The identification is used by the server 100 for determining the corresponding edge device 300 which needs to be triggered (e.g. the identifier of the new video source 10 sent by the user device 100 is mapped to an IP address of the corresponding edge device 300). The command is generated through an interaction of the user of the user device 200through the user interface of the user device 200. The implementation of the user interaction for selecting the new video source is out of the scope of the present disclosure.

[0061] Optionally, the user of the user device 200 has the capability to select one or more effects to be applied to the raw video signal from the selected new video source. Alternatively, the one or more effects are pre-selected and are not determined in real-time by the user upon selection of the new video source. The command sent 6 from the user device 200 to the server 100 comprises the one or more effects to be applied. Alternatively, the one or more effects to be applied are transmitted via another command (not represented in the Figures) sent from the user device 200 to the server 100.

[0062] Optionally, the user of the user device 200 has the capability to select a video transition to be applied to the raw video signal from the selected new video source. Alternatively, the video transition is pre-selected and is not determined in real-time by the user upon selection of the new video source. The command sent 6 from the user device 200 to the server 100 comprises the video transition to be applied. Alternatively, the video transition to be applied is transmitted via another command (not represented in the Figures) sent from the user device 200 to the server 100.

[0063] The server 100 sends 7 a command to the edge device 300 (e.g. edge device (2) in Figure 4) selected for being the new source for the broadcasting.

[0064] Upon reception of the command 7, the selected edge device 300 (e.g. edge device (2)) starts transmitting 8 its raw video signal to the server 100.

[0065] When the server 100 starts receiving the new raw video signal transmitted 8 by edge device (2), the server 100 starts using this new raw video signal (instead of the one transmitted 3 by edge device (1 )) for generating and transmitting 4 the broadcast video signal to the broadcasting equipment 20. The generation (based on the broadcast video signal) and transmission 5 of the ULL video stream to the user device 200 is also affected by the usage of the new raw video signal.

[0066] Optionally, the previously mentioned one or more effects (if any) are applied to the new raw video signal for generating the broadcast video signal.

[0067] Optionally, the previously mentioned video transition (if any) is applied to the new raw video signal for generating the broadcast video signal. More specifically, the video transition is integrated to the broadcast video signal before starting to use the new raw video signal for generating the broadcast video signal (the video transition is inserted between the previously used raw video signal and the new raw video signal).

[0068] The server 100 then sends 9 a command to the edge device 300 (e.g. edge device (1) in Figure 4) that was previously used as the source for the broadcasting, to stop transmitting 3 the corresponding raw video signal.

[0069] Upon reception of the command 9, the edge device 300 (e.g. edge device (1) in Figure 4) stops transmitting 3 its raw video signal to the server 100 if the operator is switching sources.

[0070] In an alternative implementation, upon reception of the command 9, the edge device 300 (e.g. edge device (1 ) in Figure 4) continues transmitting 3 its raw video signal to the server 100 if the operator is swapping between program mode and preview mode. If the source in preview mode changes, then the edge device 300 will stop transmitting 3 its raw video signal to server 100.

[0071] Reference is now made concurrently to Figures 2, 4 and 5. Figure 5 illustrates the video flows after the transition from edge device (1 ) to edge device (2) as the source for broadcasting (as illustrated in Figure 4) has been completed.

[0072] Upon reception of the command 6, the transition to generating and transmitting 4 the broadcast video signal based on the raw video signal transmitted 8 by edge device (2) needs to be as fast as possible (e.g. less than 20 milliseconds).

[0073] In existing (prior art) live video broadcasting systems, each edge device 300 always transmits the raw video signal (even when it is not broadcasted) to the server 100, so that the transition is fast by construction.However, the cost in terms of bandwidth consumption is very high, due to the bandwidth required for transmitting each raw video signal. This may limit the number of video sources 10 which can be supported by the live video broadcasting system.

[0074] By contrast, with the presently described live video broadcasting system, the transmission of only the raw video signal being broadcasted to the server 100 is very effective in terms of bandwidth consumption, allowing to support a large number of video sources 10.

[0075] In order to limit the duration of the aforementioned transition, adapted protocol(s) need to be used for implementing steps 7 and 8. For example, the NIC 130 of the server 100 and the NIC (not represented in the Figures for simplification purposes) of the edge device 300 used for performing steps 7 and 8 implement a zero-copy networking functionality. For example, the zero-copy networking functionality is Remote Direct Memory Access (RDMA). However, other implementations of zero-copy networking functionality are also considered relevant for the present disclosure. Zero-copy networking and RDMA are well known in the art. They provide the capability of directly reading the raw video signal stored in the memory of the edge device 300 and copying the raw video signal directly in the memory 120 of the server 100. More specifically, the NIC 130 of the server 100 and the NIC of the edge device 300 supporting the zero-copy networking functionality (e.g. RDMA) provide the capability to perform data transfers (e.g. of the raw video signal) by directly accessing the respective memories of the server 100 and the edge gateway 300, without using the processors, cache memory, operating systems, kernel networking stacks, etc. The network performance of the data transfer is therefore improved (e.g. better throughput, lower latency, etc.). The present disclosure is not limited to a zerocopy networking functionality, such as RDMA. Other transport protocols supporting low latency IP streams can also be used to limit the duration of the aforementioned transition.

[0076] With respect to the ULL video streams, Web Real-Time Communication (WebRTC) is an example of a protocol suite which can be usedfor implementing the transmission of the ULL video streams. WebRTC is well known in the art for being adapted to support effective real-time peer-to-peer communications for the exchange of voice, video and other types of data. More generally, the following protocols are generally relied upon for the transmission of the ULL video streams: the Internet Protocol (IP), the Transmission Control Protocol (TCP), and the Hypertext Transfer Protocol (HTTP); alternatively or complementarily the IP, the User Datagram Protocol (UDP), and the Real-time Transport Protocol (RTP). A person skilled in the art would readily understand that other Internet communication protocols may be used.

[0077] Reference is now made concurrently to Figures 5 and 6B.Figure 6B is similar to Figure 6A and represents the screen 250 of the user device 200. Figure 6B corresponds to the configuration illustrated in Figure 5, after the transition from video source (1 ) to video source (2) as the source for broadcasting. The ULL video stream transmitted by the server 100 and displayed on the screen 250 corresponds to the currently broadcast video signal, which now uses video source (2) as the source for broadcasting. The visual indication 255 now identifies the ULL video stream corresponding to video source (2) as the source for the broadcasting.

[0078] Reference is now made concurrently to Figures 1, 2 and 7.Figure 7 corresponds to the live video broadcasting system illustrated in Figure 1 , with the functionalities of the switch 60 of Figure 1 integrated to the server 100. More specifically, a virtual switch 160 is executed by the processing unit 1 10 of the server 100. The virtual switch 116 is implemented by computer program(s) having instructions executed by the processing unit 110.

[0079] The server 100 is directly connected to at least some of the other equipment (edge devices 300, user device 200 and broadcasting equipment 20), using the virtual switch 160 to implement networking functionalities provided by the IP networking infrastructure 50 (including the switch 60) in Figure 1 . However, the server 100 may still need to use the IP networking infrastructure 50 to communicate with some of the other equipment. In an exemplary implementation, the server 100 is directly connected to the edge devices 300 through the virtualswitch 160, but still uses the IP networking infrastructure 50 for the communications with the user device 200 and the broadcasting equipment 20.

[0080] The virtual switch 160 supports and optionally optimizes the reception and I or transmission, via the NIC(s) 130 of the server 100, of at least one of: the raw video signals transmitted by the edge devices 300, the ULL video stream transmitted to the user device 200 and the broadcast video steam transmitted to the broadcasting equipment 20.

[0081] Furthermore, a plurality of virtual switches 160 may be executed concurrently by the processing unit 110 of the server 100. In an exemplary implementation, each time a new user device 200 interconnects with the server 100, a new instance of virtual switch 160 dedicated to this new user device 200 is launched. In another exemplary implementation, a pool of virtual switches 160 is available, providing redundancy and load-balancing. Optionally, each virtual switch 160 is dedicated to a given set of edge devices 300.

[0082] With respect to the NICs 130, in a first implementation, each NIC 130 is dedicated to a given virtual switch 160. In another exemplary implementation, a pool of NICs 130 is available and assignable on demand to a given virtual switch 160, providing redundancy and load-balancing. Optionally, each NIC 130 is dedicated to a given set of edge devices 300.

[0083] Reference is now made to Figure 1, 8A and 8B. Figures 8A and 8B correspond to the live video broadcasting system illustrated in Figure 1 , with the server 100 and optionally the edge devices 300 localized in a cloud infrastructure 70.

[0084] As is well known in the art, the cloud infrastructure 70 generally comprises a large number of generic computing devices managed by a cloud management platform (not represented in Figures 8A and 8B for simplification purposes). The cloud management platform usually comprises dedicated hardware and software components. The cloud infrastructure 70 provides various types of cloud services, such as easy and scalable deployment of third-party software, security, redundancy, load balancing, effective communicationcapabilities (internally to the cloud infrastructure 70, as well as with external devices via external communication networks), etc.

[0085] A generic computing device belonging to the cloud infrastructure 70 is capable of providing the aforementioned cloud services. Furthermore, by deploying and executing dedicated third-party software, the generic computing device is customized to provide corresponding third-party services I functionalities. For example, in the context of the present live video broadcasting system, one or more generic computing devices of the cloud infrastructure 70 are configured to execute the previously described functionalities of the cloud-based server 100. Similarly, one or more generic computing devices of the cloud infrastructure 70 are configured to execute the previously described functionalities of each cloud-based edge device 300 (Figure 8A). Alternatively, only the server 100 is located in the cloud infrastructure 70, but not the edge devices 300 (Figure 8B).

[0086] Other components of the live video broadcasting system may be integrated to the cloud infrastructure 70, such as the user device 200 and I or the broadcasting equipment 20. Furthermore, the server 100 with a virtual switch functionality 160 illustrated in Figure 7 can also be implemented by the cloudbased server 100 illustrated in Figures 8A and 8B.

[0087] In the context of the cloud infrastructure 70, protocol(s) specifically designed for operating in a cloud infrastructure may be used for reducing the delay when switching from one video source to another video source as the source for the broadcasting. For example, libfabric (also referred to as Open Fabric Interfaces (OFI)) can be used in this context.

[0088] Reference is now made concurrently to Figures 1 , 2, 9A and 9B. Figures 9A and 9B represent a method 400 for producing live video broadcasting from a plurality of video sources. The method 400 is executed by the processing unit 110 of the server 100.

[0089] A dedicated computer program has instructions for implementing at least some of the steps of the method 400. The instructions are comprised in a non-transitory computer-readable medium (e.g. the memory 120 of the server100). The instructions, when executed by the processing unit 110 of the server 100, provide for producing live video broadcasting from a plurality of video sources. The instructions are deliverable to the server 100 via an electronically- readable media such as a storage media (e.g. CD-ROM, USB key, etc.), or via communication links (e.g. via a communication network through the NIC 130 of server 100).

[0090] The method 400 comprises step 405 of receiving a first raw video signal from a first edge device 300 (e.g. edge device (1 )) connected to a first video source 10 (e.g. video source (1 )), the first raw video signal being generated by the first edge device 300 based on a first video signal received from the first video source 10. Step 405 is executed by the processing unit 1 10. This step has been previously detailed in relation to Figure 3.

[0091] The method 400 comprises step 410 of processing the first raw video signal to generate a broadcast video signal based on the first raw video signal. Step 410 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 3.

[0092] The method 400 comprises step 415 of transmitting (via one of the NIC(s) 130) the broadcast video signal to the video broadcasting equipment 20. Step 415 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 3.

[0093] The method 400 comprises step 420 of processing the broadcast video signal to generate an ultra low latency (ULL) video stream based on the broadcast video signal. Step 420 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 3.

[0094] The method 400 comprises step 425 of transmitting (via one of the NIC(s) 130) the ULL video stream to the user device 200. Step 425 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 3.

[0095] The method 400 comprises step 430 of receiving (via one of the NIC(s) 130) from the user device 200 a command for switching to a second videosource. Step 430 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 4.

[0096] The method 400 comprises step 435 of transmitting (via one of the NIC(s) 130) a command to a second edge device 300 (e.g. edge device (2)) connected to the second video source 10 (e.g. video source (2)) to start transmitting a second raw video signal generated by the second edge device 300 based on a second video signal received from the second video source 10. Step 435 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 4.

[0097] The method 400 comprises step 440 of receiving the second raw video signal from the second edge device 300 (e.g. edge device (2)). Step 440 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 4.

[0098] The method 400 comprises step 445 of processing the second raw video signal to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal. Step 445 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 4.

[0099] Following step 445, the broadcast video signal continues to be transmitted according to step 415, the ULL video stream continues to be generated according to step 420 and the ULL video stream continues to be transmitted according to step 425.

[0100] The method 400 comprises step 450 of transmitting a command to the first edge device 300 (e.g. edge device (2)) connected to the first video source 10 (e.g. video source (1 )) to stop transmitting the first raw video signal. Step 455 is executed by the processing unit 110. This step has been previously detailed in relation to Figure 4.

[0101] It should be noted that the order of some of the steps of the method 400 may be varied. The order of the steps illustrated in Figures 9A and 9B is not necessarily representative of the timeframe at which the steps are performed. For example, step 420 may occur before step 415.

[0102] Reference is now made to Figure 10. Figure 10 is a detailed representation of the components of the user device 200. The user device 200 comprises the processing unit 210, the memory 220, at least one Network Interface Card (NIC) 230, the user interface 240 and the screen 250. Some of the components have been previously described in relation to Figures 1 and 2. All the components of the user device 200 are well known in the art.

[0103] Reference is now made concurrently to Figures 3, 4, 5, 6A, 6B, 10 and 11. Figure 11 represents a method 500 for performing a zero-delay transition between broadcasting sources. The method 500 is executed by the processing unit 210 of the user device 200.

[0104] A dedicated computer program has instructions for implementing at least some of the steps of the method 500. The instructions are comprised in a non-transitory computer-readable medium (e.g. the memory 220 of the user device 200). The instructions, when executed by the processing unit 210 of the user device 200, provide for performing a zero-delay transition between broadcasting sources. The instructions are deliverable to the server 100 via an electronically-readable media such as a storage media (e.g. CD-ROM, USB key, etc.), or via communication links (e.g. via a communication network through the NIC 230 of user device 200).

[0105] The method 500 comprises step 505 of receiving (via one of the NIC(s) 230) a plurality of ULL video streams from a plurality of edge devices 300, each ULL video stream being generated by one of the edge devices 300 based on a video signal received from a video source connected to the edge device 300. Step 505 is executed by the processing unit 210. This step has been previously detailed in relation to Figure 3, the plurality of ULL video streams being identified with reference number 2.

[0106] The method 500 comprises step 510 of displaying the plurality of ULL video streams on the screen 250. Step 510 is executed by the processing unit 210. This step has been previously detailed in relation to Figure 6A.

[0107] The method 500 comprises step 515 of displaying the visual indication 255 on the screen 250, the visual indication 255 identifying the ULLvideo stream corresponding to the video source 10 (e.g. video source (1 ) in Figure 6A) currently used as the source for the broadcasting. Step 515 is executed by the processing unit 210. This step has been previously detailed in relation to Figure 6A.

[0108] The method 500 comprises step 520 of receiving (via one of the NIC(s) 230) an ULL broadcast video stream from the server 300, the ULL broadcast video stream being representative of the video signal currently broadcasted by the server 300. Step 520 is executed by the processing unit 210. This step has been previously detailed in relation to Figure 3, the ULL broadcast video stream being identified with reference number 5.

[0109] The method 500 comprises step 525 of displaying the ULL broadcast video stream on the screen 250. Step 525 is executed by the processing unit 210. This step has been previously detailed in relation to Figure 6A.

[0110] The method 500 comprises step 530 of receiving a user interaction (e.g. through the user interface 240) for selecting a video source 10 to be used as the new source for the broadcasting. Step 530 is executed by the processing unit 210.

[0111] As mentioned previously, the implementation of the user interaction is out of the scope of the present disclosure. Various implementations may be used, as is well known in the art. For example, referring to Figure 6A, the visual indication 255 identifies video source (1) as the current source for the broadcasting. If the user wants to select video source (2) as the new source for the broadcasting, the user may simply click on the displayed ULL video stream from video source (2).

[0112] The method 500 comprises step 535 of displaying the visual indication 255 on the screen 250 identifying the ULL video stream corresponding to the video source 10 (e.g. video source (2) instead of video source (1 ) in Figure 6B) selected (at step 530) as the new source for the broadcasting. Step 535 is executed by the processing unit 210. This step has been previously detailed in relation to Figure 6B.

[0113] The method 500 comprises step 540 of sending (via one of the NIC(s) 230) a command to the server 100 to switch the source for the broadcasting to the video source 10 (e.g. video source 2) selected at step 530. Step 540 is executed by the processing unit 210. This step has been previously detailed in relation to Figure 4 the command being identified with reference number s.

[0114] As mentioned previously, the command comprises an identification of the selected video source (or an identification of the raw video signal generated by the edge device connected to selected video source, etc.) to be used as the new source for the broadcasting.

[0115] Reference is now made to Figure 12. Figure 12 is based on Figure 3 and represents an alternative implementation.

[0116] All the ULL video streams generated by the edge devices 300 are transmitted 2 to the server 100 (instead of the user device 200 as illustrated in Figure 3). The server 100 combines all the ULL video streams received 2 from the edge devices 300 into a single combined ULL video stream. For example, the individual ULL video streams are combined into a scaled ULL video stream transmitted 2’ to the user device 100. The combined ULL video stream is transmitted 2’ to the user device 200, where it is displayed on the screen of the user device 200.

[0117] Although the present disclosure has been described hereinabove by way of non-restrictive, illustrative embodiments thereof, these embodiments may be modified at will within the scope of the appended claims without departing from the spirit and nature of the present disclosure.

Claims

WHAT IS CLAIMED IS:

1. A method for producing live video broadcasting from a plurality of video sources, the method comprising: receiving at a server a first raw video signal from a first edge device connected to a first video source, the first raw video signal being generated by the first edge device based on a first video signal received from the first video source; processing by the server the first raw video signal to generate a broadcast video signal based on the first raw video signal; transmitting by the server the broadcast video signal to a video broadcasting equipment; processing by the server the broadcast video signal to generate an ultra low latency (ULL) video stream based on the broadcast video signal; transmitting by the server the ULL video stream to a user device; receiving by the server from the user device a command for switching to a second video source; transmitting by the server a command to a second edge device connected to the second video source to start transmitting a second raw video signal generated by the second edge device based on a second video signal received from the second video source; receiving by the server the second raw video signal from the second edge device; processing by the server the second raw video signal to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal; andtransmitting by the server a command to the first edge device connected to the first video source to stop transmitting the first raw video signal.

2. The method of claim 1 , wherein the server implements a zero-copy networking functionality for transmitting the first raw video signal from the first edge device to the server and for transmitting the second raw video signal from the second edge device to the server.

3. The method of claim 2, wherein the server comprises at least one Network Interface Card (NIC) supporting the zero-copy networking functionality.

4. The method of claim 2, wherein the zero-copy networking is Remote Direct Memory Access (RDMA).

5. The method of claim 1 , wherein the server executes a virtual switch functionality.

6. The method of claim 1 , wherein the server is a cloud-based server located in a cloud infrastructure.

7. The method of claim 6, wherein the first and second edge devices are cloud-based edge devices also located in the cloud infrastructure.

8. The method of claim 1 , further comprising at least one of the following: applying one or more effects to the first raw video signal or to the second raw video signal to generate the broadcast video signal; and applying a video transition between the first raw video signal and the second raw video signal to generate the broadcast video signal.

9. The method of claim 1 , further comprising: receiving by the server a plurality of ULL video streams from a plurality of edge devices, each ULL video stream being generated by one of the plurality of edge devices based on a video signal received from a video source connected to the edge device, the plurality of edge devices comprising the first edge device and the second edge device;combining by the server the plurality of ULL video streams into a combined ULL video stream; and transmitting by the server the combined ULL video stream to the user device.

10. A non-transitory computer readable medium comprising instructions executable by a processing unit of a server, the execution of the instructions by the processing unit of the server providing for producing live video broadcasting from a plurality of video sources by: receiving by the server a first raw video signal from a first edge device connected to a first video source, the first raw video signal being generated by the first edge device based on a first video signal received from the first video source; processing by the server the first raw video signal to generate a broadcast video signal based on the first raw video signal; transmitting by the server the broadcast video signal to a video broadcasting equipment; processing by the server the broadcast video signal to generate an ultra low latency (ULL) video stream based on the broadcast video signal; transmitting by the server the ULL video stream to a user device; receiving by the server from the user device a command for switching to a second video source; transmitting by the server a command to a second edge device connected to the second video source to start transmitting a second raw video signal generated by the second edge device based on a second video signal received from the second video source; receiving by the server the second raw video signal from the second edge device;processing by the server the second raw video signal to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal; and transmitting by the server a command to the first edge device connected to the first video source to stop transmitting the first raw video signal.

11. A server for live video broadcasting production from a plurality of video sources, the server comprising: at least one Network Interface Card (NIC) for receiving and transmitting data; and a processing unit for: receiving a first raw video signal from a first edge device connected to a first video source, the first raw video signal being generated by the first edge device based on a first video signal received from the first video source; processing the first raw video signal to generate a broadcast video signal based on the first raw video signal; transmitting the broadcast video signal to a video broadcasting equipment; processing the first raw video signal to generate an ultra low latency (ULL) video stream based on the broadcast video signal; transmitting the ULL video stream to a user device; receiving from the user device a command for switching to a second video source; transmitting a command to a second edge device connected to the second video source to start transmitting a second raw video signal generated by the second edgedevice based on a second video signal received from the second video source; receiving the second raw video signal from the second edge device; processing the second raw video signal to generate the broadcast video signal based on the second raw video signal instead of the first raw video signal; and transmitting a command to the first edge device connected to the first video source to stop transmitting the first raw video signal.

12. The server of claim 11 , wherein the server implements a zero-copy networking functionality for transmitting the first raw video signal from the first edge device to the server and for transmitting the second raw video signal from the second edge device to the server.

13. The server of claim 12, wherein one or more of the at least one NIC supports the zero-copy networking functionality.

14. The server of claim 12, wherein the zero-copy networking is Remote Direct Memory Access (RDMA).

15. The server of claim 11 , wherein the processing unit executes a virtual switch functionality.

16. The server of claim 11 , wherein the server is a cloud-based server located in a cloud infrastructure.

17. The server of claim 11 , wherein the processing unit further performs at least one of the following: applying one or more effects to the first raw video signal or to the second raw video signal to generate the broadcast video signal; and applying a video transition between the first raw video signal and the second raw video signal to generate the broadcast video signal.

8. The server of claim 11 , wherein the processing unit further: receives a plurality of ULL video streams from a plurality of edge devices, each ULL video stream being generated by one of the plurality of edge devices based on a video signal received from a video source connected to the edge device, the plurality of edge devices comprising the first edge device and the second edge device; combines the plurality of ULL video streams into a combined ULL video stream; and transmits the combined ULL video stream to the user device.

Images