Video broadcasting system equipped with a switching device for switching between multiple extended performance video sequences captured from real events

Abstract

The present invention comprises a server-side video distribution device, in particular an OTT platform, configured to receive and process from a production video device a number of staged video sequences captured at a real event and augmented with virtual content, and a user-side video output device, in particular a playback device for streaming, connectable or connected to the video distribution device and configured to receive at least one of the number of staged video sequences and to output the received staged video sequence or one of the received number of staged video sequences, in particular to display the same on a display. In this process, the video output device is a video broadcasting system including a switching device configured to automatically switch to another received staged video sequence during the output of one staged video sequence and output the other staged video sequence.

Description

[Technical Field]

[0001] The present invention is a video distribution system comprising a server-side video distribution device, in particular an OTT platform, configured to capture real events and receive multiple production image sequences augmented with virtual content from the production image device and to process them, and at least one user-side video output device, in particular a streaming playback device, connectable to or connected to the video distribution device and configured to receive at least one of the multiple production image sequences and to output, in particular for displaying on a display, the received production image sequence or one of the received production image sequences. [Background technology]

[0002] It is known from the prior art, for example from DE 10 2016 119 637 A1, which is also from the applicant, to generate a change in at least one subsection of a staged image in a television broadcasting system by augmentation, so that an extended staged image is generated and the extended staged image can be output.

[0003] The terms "augment" or "augmentation" as used herein refer to a type of enhancement or change in the reproduction of reality, such as overlaying an advertising message actually written in English on a stadium board with the same or another advertisement written in another language, such as the language of the target country where the television broadcast is available. Furthermore, completely different messages or information can be overlaid. However, overlay is not the only possibility for modifying or enhancing the staged image. Rather, within the scope of the present invention, augmentation can also be understood to mean that the staged image itself is locally modified. The technical term "augmentation" as used herein is derived from the English word "augment," which means to improve, enhance, or enrich.

[0004] In the system known from DE 10 2016 119 637 A1, a video sequence (video feed) that is produced or captured live, for example at a sporting event, is used as input. From this input, the system generates a number of augmented, staged video sequences (so-called localized feeds) as output. These localized feeds are similar to the world feed, i.e. the feed that is normally transmitted to television viewers at home. The difference between the two types of feed is that in the localized feed, existing advertisements (e.g. on boards) are replaced by virtual advertisements or new virtual advertisements are introduced where no physical advertisements were previously present (e.g. on a football pitch).

[0005] Known systems are therefore designed to augment staged video sequences (localized feeds) with virtual content, in other words, to provide exactly one augmented staged video sequence for one region or one country. Summary of the Invention

[0006] The underlying objective of the present invention is to define a video broadcasting system that allows for enhanced and efficient use of extended staged video sequences.

[0007] This object is achieved by a video broadcasting system having the features of independent claim 1. Advantageous embodiments with suitable further developments are set out in the dependent claims.

[0008] Therefore, a video broadcasting system is proposed, comprising: a server-side video distribution device, in particular an OTT platform, configured to receive and process a plurality of staged video sequences captured at a real event and augmented with virtual content from a staged video device; and a user-side video output device, in particular a playback device for streaming, connectable to or connected to the video distribution device, configured to receive at least one of the plurality of staged video sequences and to output the received staged video sequence or one of the received plurality of staged video sequences, in particular to display the same on a display. In this process, it is envisaged that the video output device includes a switching device configured to automatically switch to another received staged video sequence during the output of one staged video sequence and output the other staged video sequence.

[0009] This allows for a changeover from one staged video sequence to another in a manner that is transparent to the end user and has essentially no time delay, although of course the changeover or switch can also be reversed to a previously output staged video sequence.

[0010] The video broadcasting system may include a matching device configured to transmit switching data to a switching device, and the switching device may be configured to output a rendered video sequence assigned to the switching data.

[0011] The matching device can be configured to receive a plurality of receiving group data sets, the number of receiving group data sets being equal to the number of staged video sequences augmented with virtual content. In this manner, a receiving group data set can be provided for each staged video sequence augmented with virtual content, thereby allowing technical and / or user-specific receiving groups to be assigned to particular staged video sequences.

[0012] In this context, the matching device may be configured to generate the switching data in response to the receiving group data set.

[0013] In a video broadcasting system, the matching device can be configured to assign one of the receiving group data sets to a user of a video output device. In this way, technical and / or user-specific information about the user, such as the user's location, browser usage, terminal device used, browsing behavior on the Internet, user age, user gender, etc., can be used as a technical data basis for assigning an appropriate receiving group data set to a particular user.

[0014] In this process, the matching device may be configured to determine the allocation of a particular receiving group data set to a particular user based on a comparison of the user data and the receiving group data set.

[0015] In this context, the matching device may be configured to assign receiving group data sets that correspond to:

number

[0016] In this way, the minimization function specified determines the number of users in a given group I who view a staged video sequence j during a certain time segment t. j,t The optimal augmentation a for is selected. Alternatively, the matching device may select an optimal user group l for each time segment t of a rendition video sequence j having a given augmentation a. j,t It can also be configured to determine Also, given a collection of augmentations A and all user groups I, we can find the optimal augmentation a and the optimal user group I. j,t A matching device that simultaneously determines the above is also conceivable. In a video broadcasting system, the matching device may be part of a server-side video distribution device or part of a user-side video output device. That is, the matching device may be provided in a centralized or distributed manner. In either case, however, it is guaranteed that the switching data determined or calculated in a centralized or distributed manner is transmitted to the switching device of a specific video output device of a specific user.

[0017] In a video broadcasting system, the matching device may be configured to receive a received group data set from a presentation video device.

[0018] In other words, the matching device, which may be implemented on the server side or the user side, in combination with the switching device, can reliably receive a particular rendered video sequence and display it on the video output device.

[0019] In a video broadcasting system, each staged video sequence can include multiple sections with virtually augmented image content, and the sections of each staged video sequence start and end at the same time, thereby enabling the switching time from one staged video sequence to another to be synchronized, making the switching between the two staged video sequences unnoticeable to a particular user.

[0020] In a video broadcasting system, a staging video device can be configured to receive a video sequence from a captured real event, generate from this video sequence a plurality of staged video sequences augmented with virtual content, and transmit the staged video sequences to a video distribution device, in the process the staging video device can be part of a television broadcasting system, for example, as known from DE 10 2016 119 637 A1.

[0021] The video broadcasting system described above can therefore also be combined with or connected to a television broadcasting system known from DE 10 2016 119 637 A1.

[0022] Therefore, an auxiliary television broadcasting system, a plurality of television cameras configured to capture camera images and output them as camera signals; at least one production image device or production unit configured to generate a series of produced images from the camera images of the plurality of television cameras and output the produced images as production signals; a plurality of analysis modules, one analysis module assigned to each television camera, the analysis modules configured to receive camera footage in a synchronous buffered manner, the analysis modules configured to analyze each camera footage, calculate camera footage metadata and a camera footage hash value associated with each camera footage, and asynchronously output the camera footage metadata and the camera footage hash value for each camera footage; a synchronization module connected to the production video device or the production unit and the analysis module, configured to receive the production video from the production video device or the production unit in a synchronous buffering manner, calculate an associated production video hash value for each production video, and receive camera video metadata and the associated camera video hash value for each camera video from the analysis module in an asynchronous buffering manner, the synchronization module further configured to compare the production video hash value with the camera video hash value, and based on the comparison, assign the associated camera video metadata of the camera video to the current production video as production video metadata, and output the production video metadata in a synchronous buffering manner; a multiplication module coupled to the synchronization module and the staged video device or the staged unit, configured to receive the staged video metadata from the synchronization module in a synchronous buffering manner and to receive the staged video from the staged video device or the staged unit in a synchronous buffering manner, the multiplication module being further configured to compensate for a time offset between the staged video and associated staged video metadata, generate augmented staged video modifications of at least one subsection of the staged video based on the staged video metadata, such that an augmented staged video is generated, and output the augmented staged video; Auxiliary television broadcasting systems have also been proposed.

[0023] Such a television broadcasting system comprises a server-side video distribution device, in particular an OTT platform, configured to receive and process a plurality of staged video sequences captured at real events and augmented with virtual content from a staged video device or a staged unit, and at least one user-side video output device, in particular a playback device for streaming, connectable to or connected to the video distribution device and configured to receive at least one of the plurality of staged video sequences and to output, in particular, display the received staged video sequence or one of the received staged video sequences on a display. In this process, it is envisaged that the video output device includes a switching device configured to automatically switch to another received staged video sequence during the output of one staged video sequence and output the other staged video sequence.

[0024] Further advantages and details of the invention will become apparent from the following description of embodiments with reference to the drawings. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 is a simplified schematic diagram illustrating an example of a video broadcasting device. [Figure 2] FIG. 1 is a diagram illustrating a simplified schematic diagram of functions of a video broadcasting device. [Figure 3] FIG. 10 is a simplified schematic diagram illustrating another example of a video broadcasting device. [Figure 4] FIG. 2 is a simplified schematic diagram of a further example of a video broadcasting device. [Figure 5] FIG. 4 is a simplified schematic diagram showing the connection between a known television broadcasting system and the video broadcasting devices of FIGS. 1 to 3. DETAILED DESCRIPTION OF THE INVENTION

[0026] 1 is a simplified schematic diagram of a video broadcasting device 50. The video broadcasting device 50 includes, inter alia, a video distribution device 52 configured as a so-called over-the-top (OTT) platform. The video distribution device is configured to receive from a production video device 54 and process a plurality of staged video sequences LF1, LF2, LF3, LFN that have been captured at a real event and augmented with virtual content.

[0027] In particular, the staged video device 54 is capable of augmenting each staged video sequence LF1, LF2, LF3, LFN with different virtual content. In this way, each staged video sequence LF1, LF2, LF3, LFN can be augmented with virtual content that is appropriate for, for example, a particular group of end users.

[0028] The video broadcasting device 50 further includes at least one user-side video output device 56, particularly a playback device for streaming, which is connectable to or connected to the video distribution device 52. The video output device 56 can be, for example, a so-called standard streaming video playback device based on technologies such as MPEG DASH (Dynamic Adaptive Streaming over HTTP) and HLS (HTTP Live Streaming).

[0029] For clarity, only a single video output device 56 is depicted in Figure 1. However, it will be appreciated that video distribution device 52 may be connected to multiple video output devices 56. This is particularly illustrated by content delivery network (CDN) component 58, which may be part of video distribution device 52.

[0030] The video output device 56 is further configured to receive a plurality of rendered video sequences LF1, LF2, LF3, LFN, and to output, in particular display on a display 60, one of the received rendered video sequences LF1. The video output device 56 includes a switching device 62, which is configured to automatically switch to other received performance video sequences LF2, LF3, and LFN while one performance video sequence LF1 is being output, and output the other performance video sequences.

[0031] The video broadcasting system 50 includes a matching device 64, which is configured to transmit switching data SD to the switching device 62. At this time, the switching device 62 is configured to output the performance video sequences LF1, LF2, LF3, and LFN assigned to the switching data SD.

[0032] The matching device 64 is configured to receive a plurality of receiving group data sets EGD1, EGD2, EGD3, EGDN, where the number of receiving group data sets EGD1, EGD2, EGD3, EGDN is the same as the number of the plurality of staged video sequences LF1, LF2, LF3, LFN augmented with virtual content.

[0033] Each receiving group data set EGD1, EGD2, EGD3, EGDN contains information about the virtual content to which the assigned presentation video sequence LF1, LF2, LF3, LFN is extended. Furthermore, each receiving group data set EGD1, EGD2, EGD3, EGDN contains information about the user target group to which the corresponding virtual content should be presented.

[0034] In the video broadcasting system 50, the matching device 64 is configured to generate the switching data SD as a function of the receiving group data sets EGD1, EGD2, EGD3, EGDN.

[0035] Additionally, the matching device 64 is configured to assign one of the receiving group data sets to a user of the video output device 56. The assignment of a particular receiving group data set EGD1, EGD2, EGD3, EGDN to a particular user may be determined based on a comparison of the user data with the receiving group data sets EGD1, EGD2, EGD3, EGDN.

[0036] The video broadcasting system 50 described above with reference to FIG. 1 and its functions will now be described in detail with reference to FIG.

[0037] 2 shows three staged video sequences LF1, LF2, and LF3. The staged video sequences LF1, LF2, and LF3 each include multiple sections A0 to A8 having virtually extended video content, and the sections A0 to A8 in each staged video sequence LF1, LF2, and LF3 start and end at the same time, as indicated by vertical dashed lines.

[0038] Furthermore, two video output devices 56 are depicted in a simplified manner in FIG. 2 and can be understood as representing users P1 and P2, respectively.

[0039] 2, by using the architecture of the video broadcasting device 50 described above, each of the rendered video sequences LF1, LF2, and LF3 can be divided chronologically into sections A0 to A8 or periods. During these periods, each viewer or user P1 or P2 can view augmentation by virtual content made up of sections A0 to A8 of the different rendered video sequences LF1, LF2, and LF3.

[0040] 2, user P1 watches staged video sequence LF2 in section A0, then watches staged video sequence LF1 in sections A1 and A2, watches staged video sequence LF2 (again) in section A3, watches staged video sequence LF3 in sections A4 to A7, and watches staged video sequence LF1 in section A8. The sequence of stagesd video sequences LF1, LF2, and LF3 from sections A0 to A8 is indicated by a solid black arrow for user P1.

[0041] 2, user P2 views the staged video sequence LF3 in sections A0 to A2, then views the staged video sequence LF2 in sections A3 and A4, views the staged video sequence LF1 in sections A5 and A6, and views the staged video sequence LF2 in sections A7 and A8. The sequence of sections A0 to A8 from the staged video sequences LF1, LF2, and LF3 is shown by a dashed black arrow pointing toward user P2.

[0042] The system configuration of the video broadcasting device 50 described above allows for the provision of rendered video sequences LF1, LF, LF3 to be used to present users P1, P2 with the rendered video sequences LF1, LF2, LF3 that best fit their profile, particularly their user data, during each section A0-A8. That is, each user P1, P2 will see a personalized sequence (feed) made up of the most suitable sections from the rendered video sequences LF1, LF2, LF3, where the suitable sections A0-A8 are selected from the various rendered video sequences LF1, LF2, LF3 based on the above-mentioned switching data SD, particularly the relationship between the user data and the receiving group data set EGD.

[0043] Thus, the video broadcasting system 50 presented herein can be used to address a specific selection or subset of a user target group in each section A0 through A8, which can also be referred to as a time interval or period, through virtual augmentation. This selection can vary from section to section. As can be seen from FIG. 2, both users P1 and P2 view the same staged video sequence LF2 during section A3, and therefore also view the same virtual augmentation from staged video sequence LF2 included in this section A3. In all other sections A0 through A2 and A4 through A8, users P1 and P2 view different staged video sequences.

[0044] User P1 may, for example, use a first type of device as video output device 56. User P2 may, for example, use a second type of device as video output device 56. Information about the type of device may, for example, be part of the user data.

[0045] For example, the staged video sequence may include virtual augmentations in sections A8, A2, and A1, each of which relates to a first type of device used by user P1. Thus, the receiving group data set EGD1 includes information such that sections A8, A2, and A1 are targeted to users with the first type of device. Thus, by comparing the receiving group data set EGD1 with the user data of user P1, the control data SD for user P1 can be determined such that sections A8, A2, and A1 of the staged video sequence LF1 are displayed to user P1.

[0046] Because user P2 uses a second type of device, sections A8, A2, and A1 from the staged video sequence are not of interest to user P2. However, for example, staged video sequence LF1 may include virtual augmentations in sections A6 and A5 that relate to the second type of device used by user P2. Thus, receiving group data set EGD2 includes information such that sections A6 and A5 are targeted to users with the second type of device. By comparing receiving group data set EGD2 with the above-mentioned user data for user P2, control data SD for user P2 can be determined so that sections A6 and A5 of staged video sequence LF1 are displayed to user P2.

[0047] For example, in sections A7, A4, A3, and A0, the staged video sequence LF1 includes virtual augmentations that can target additional users using other types of devices.

[0048] FIG. 3 is an illustration similar to FIG. 1, in which the matching device 64 is part of the server-side video distribution device 52. In other words, the video distribution device 52 is part of the infrastructure of the OTT platform. In such a technical implementation, the video distribution device 52 requires slightly more computing power to enable the execution of the matching device 64. A separate matching instance needs to be executed for each user P1, P2, and the additional computing power required increases with the number of users P1, P2. However, the matching device 64 can be implemented so that a matching instance for one user uses only a small amount of computing power.

[0049] 4 is an explanatory diagram similar to FIGS. 1 and 3, in which the matching device 64 is part of the user-side video output device 56, 56a. For example, a standard streaming video player 56 (MPEG DASH, HLS-based, or another technology) may be extended with the matching device 64. Then, this extended player 56a or this extended video output device 56a runs on each terminal device of each user P1, P2.

[0050] Both the design according to Figure 3 and the design according to Figure 4 are technically feasible and achievable. In the implementation according to Figure 3, the video output device 56 (player) can remain standard-compliant, i.e., it does not include the (proprietary) extensions 56a due to the proprietary technology [omitted!] of the matching device 64. This allows the corresponding application (video output device 56 or player) on the terminal device to remain unchanged for all users.

[0051] 5 shows, as an example, a television broadcasting system 10 known from the aforementioned DE. This television broadcasting system 10 is composed of multiple cameras C1 and C2 and a production unit 12, and cameras C1 and C2 transmit their camera images as camera image signals to production unit 12. Typically, such a television broadcasting system 10 includes two or more cameras C1 and C2, which are indicated by three black dots to the right of camera C2.

[0052] The well-known, conventional processing path begins with cameras C1 and C2. A camera signal S1 is captured by multiple cameras C1, C2 (to Cn) and transmitted to the directing unit 12. Within the directing unit 12, a directed video sequence or directed video signal is compiled (or edited) from the individual camera feeds C1 to Cn. The edited directed video signal S3 or the continuous directed video is technically referred to as the world feed and is then broadcast. It should be noted that in this context, post-production scenarios such as repetitions, slow motion, interviews, etc. also offer the possibility of capturing or superimposing augmentations, in particular the same augmentations, on previously broadcast live footage. For this purpose, the directing unit can be configured by a control module to enable or disable augmentations for such post-production, during or after the live broadcast sequence.

[0053] The television broadcasting system 10 can be expanded as follows: Camera signal S1 is passed to analysis modules A1 and A2 (up to An, indicated by three black dots below A2). Analysis modules A1 and A2 pass signal S2 to a selection or synchronization module 14. One analysis module A1 and A2 is assigned to each camera C1 and C2. For each camera image, camera image metadata M1 and M2 (up to Mn) are generated by analysis modules A1 and A2, and associated hash values ​​H1 and H2 (up to Hn) are calculated. Each pair of metadata and hash value, i.e., M1, H1 or M2, H2, for example, is transmitted by analysis modules A1 and A2 to synchronization module 14 as signal S2. Synchronization module 14 also receives a staged video signal S3a or staged video from the staging unit. Synchronization module 14 passes modified metadata S4 for the camera currently displayed in the staged video to multiplication module 16, which generates multiple expanded staged video signals S5 to be broadcast. The analysis modules A1, A2, the synchronization module 14 and the multiplication module 16 are controlled by a control module 18. The control module 18 is controlled via the directing unit 12.

[0054] For completeness, the diagram of Figure 5 shows an optional LED system 20 that transfers video data to the control module 18 via a video transmission protocol, such as HDMI or DVI, but is not required for the basic functionality of the television broadcast system 10.

[0055] In such a television broadcasting system 10, the video broadcasting system 50 described above with reference to Figures 1 to 4 can start or be connected with the multiplication module 16 generating a plurality of extended effect video signals S5. Accordingly, the plurality of effect video signals S5 can be referred to as effect video sequences LF1, LF2, and LF3. That is, the effect video device 54 in Figures 1, 3, and 4 is a simplified representation of the components of the television broadcasting system 10 described above, enclosed within a dashed rectangle.

Claims

1. a server-side video distribution device (52), in particular an OTT platform, configured to receive and process a plurality of staged video sequences (LF1, LF2, LF3, LFN) captured at a real event and augmented with virtual content from a staged video device (54); at least one user-side video output device (56), in particular a playback device for streaming, connectable to or connected to the server-side video distribution device (52), configured to receive the plurality of performance video sequences (LF1, LF2, LF3, LFN), and to output one of the received plurality of performance video sequences (LF1, LF2, LF3, LFN), in particular to display the same on a display (60); The user-side video output device (56) is a video broadcasting system (50) including a switching device (62) configured to automatically switch to another received performance video sequence (LF1, LF2, LF3, LFN) while outputting one performance video sequence (LF1, LF2, LF3, LFN) among the plurality of performance video sequences (LF1, LF2, LF3, LFN) and output the other performance video sequence (LF1, LF2, LF3, LFN).

2. a matching device (64) configured to transmit switching data (SD) to said switching device (62); The video broadcasting system (50) of claim 1, characterized in that the switching device (62) is configured to output one of the plurality of performance video sequences (LF1, LF2, LF3, LFN) assigned to the switching data (SD).

3. The matching device (64) is configured to receive a plurality of receiving group data sets (EGD1, EGD2, EGD3, EGDN); The video broadcasting system (50) of claim 2, characterized in that the number of receiving group data sets (EGD1, EGD2, EGD3, EGDN) is the same as the number of multiple staged video sequences (LF1, LF2, LF3, LFN) extended with virtual content.

4. 4. The video broadcasting system (50) according to claim 3, wherein the matching device (64) is configured to generate the switching data (SD) in response to the receiving group data sets (EGD1, EGD2, EGD3, EGDN).

5. 5. A video broadcasting system (50) as claimed in claim 3 or 4, characterized in that the matching device (64) is configured to assign one of the receiving group data sets (EGD1, EGD2, EGD3, EGDN) to a user (P1, P2) of the user-side video output device (56).

6. 6. The video broadcasting system (50) of claim 5, wherein the matching device (64) is configured to determine allocation of particular receiving group data sets (EGD1, EGD2, EGD3, EGDN) to particular users (P1, P2) based on a comparison between user data and the receiving group data sets (EGD1, EGD2, EGD3, EGDN).

7. 7. A video broadcasting system (50) according to claim 6, characterized in that the matching device (64) is adapted to assign the following corresponding receiving group data sets (EGD1, EGD2, EGD3, EGDN). where Dt is the distance measurement over a period of time t. Pi is the user data (profile) of user i. Pa is the received group data set from the content provider. i is an individual user out of all current users I. j is a single staged video sequence. t is the time segment of one virtual augmentation. a is an augmentation of augmentation collection A.

8. A video broadcasting system (50) as described in claim 3 or 4, characterized in that the matching device (64) is configured to receive the receiving group data set (EGD1, EGD2, EGD3, EGDN) from the performance video device (54).

9. A video broadcasting system (50) as described in claim 2, characterized in that the matching device (64) is part of the server-side video distribution device (52) or part of the user-side video output device (56, 56a).

10. The plurality of staged video sequences (LF1, LF2, LF3, LFN) each include a plurality of sections (A0 to A8) having virtually expanded video content, The video broadcasting system (50) of claim 1, wherein the sections (A0 to A8) in each of the plurality of performance video sequences (LF1, LF2, LF3, LFN) start and end at the same time.

11. The video broadcasting system (50) of claim 1, wherein the staged video device (54) is configured to receive a video sequence from a captured real event, generate a plurality of staged video sequences (LF1, LF2, LF3, LFN) augmented with virtual content from the video sequence, and transmit the generated video sequences to the server-side video distribution device (52).

Images