Video broadcasting system equipped with a switching device for switching between a plurality of extended performance video sequences captured from a real event

Abstract

The present invention comprises a server-side video distribution device, in particular an OTT platform, configured to receive and process from a production video device a number of staged video sequences captured at a real event and augmented with virtual content, and a user-side video output device, in particular a playback device for streaming, connectable or connected to the video distribution device and configured to receive at least one of the number of staged video sequences and to output the received staged video sequence or one of the received number of staged video sequences, in particular to display the same on a display. In this process, the video output device is a video broadcasting system including a switching device configured to automatically switch to another received staged video sequence during the output of one staged video sequence and output the other staged video sequence.

Description

[Technical field]

[0001] The present invention is a video distribution system comprising a server-side video distribution device, in particular an OTT platform, configured to capture a real event and receive multiple production image sequences augmented with virtual content from the production image device and process them, and at least one user-side video output device, in particular a streaming playback device, connectable or connected to the video distribution device and configured to receive at least one of the multiple production image sequences and output the received production image sequence or one of the received production image sequences, in particular for display on a display. [Background technology]

[0002] It is known from the prior art, for example from DE 10 2016 119 637 A1 originating from the applicant, to generate a change of at least one subsection of a staged image in a television broadcasting system by augmentation, so that an augmented staged image is generated and that the augmented staged image can be output.

[0003] The term "augment" or "augmentation" as used herein means a kind of enhancement or change in the reproduction of reality, for example, superimposing an advertising message actually written in English on a stadium board with the same or another advertisement written in another language, such as the language of the target country where the television broadcast can be viewed. Furthermore, completely different messages or information can be superimposed. However, superimposition is not the only possibility for modifying or enhancing the staged image. Rather, within the scope of the present invention, augmentation can also be understood to mean that the staged image itself is locally modified. The technical term "augmentation" as used herein is directed to the English word "augment," which means to improve, enhance, or enrich.

[0004] In the system known from DE 10 2016 119 637 A1, a video sequence (video feed) that is produced or captured live, for example at a sporting event, is used as input. From this input, the system generates a number of augmented staged video sequences (so-called localized feeds) as output. These localized feeds are similar to the world feed, i.e. the feed that is normally transmitted to a television viewer at home. The difference between the two types of feed is that in the localized feed, existing advertisements (e.g. on a board) are replaced by virtual advertisements or new virtual advertisements are introduced in places where no physical advertisements were present so far (e.g. on the football pitch).

[0005] Known systems are therefore designed to augment staged video sequences (localized feeds) with virtual content, in other words to provide exactly one augmented staged video sequence for one region or one country. Summary of the Invention

[0006] The underlying objective of the present invention is to define a video broadcasting system that allows for enhanced and efficient use of extended staged video sequences.

[0007] This object is achieved by a video broadcasting system having the features of the independent claim 1. Advantageous embodiments with preferred further developments are given in the dependent claims.

[0008] Therefore, a video broadcasting system is proposed, comprising a server-side video distribution device, in particular an OTT platform, configured to receive and process a plurality of staged video sequences captured at real events and augmented with virtual content from a staged video device, and a user-side video output device, in particular a playback device for streaming, connectable or connected to the video distribution device and configured to receive at least one of the plurality of staged video sequences and to output the received staged video sequence or one of the received plurality of staged video sequences, in particular to display the same on a display. In this process, it is envisaged that the video output device includes a switching device configured to automatically switch to another received staged video sequence during the output of one staged video sequence and output the other staged video sequence.

[0009] This allows for a changeover from one staged video sequence to another in a manner that is transparent to the end user and has essentially no time delay, although of course the changeover or switch can also be back to a previously output staged video sequence.

[0010] The video broadcasting system may include a matching device configured to transmit switching data to a switching device, and the switching device may be configured to output a rendered video sequence assigned to the switching data.

[0011] The matching device can be configured to receive a plurality of receiving group data sets, the number of receiving group data sets being equal to the number of the plurality of staged video sequences augmented with the virtual content. In this manner, a receiving group data set can be provided for each staged video sequence augmented with the virtual content, thereby allowing technical and / or user specific receiving groups to be assigned to specific staged video sequences.

[0012] In this context, the matching device may be configured to generate the switching data depending on the receiving group data set.

[0013] In the video broadcasting system, the matching device can be configured to assign one of the receiving group data sets to a user of the video output device. In this way, technical and / or user-specific information about the user, such as the user's location, browser usage, terminal device used, browsing behavior on the Internet, the user's age, the user's gender, etc., can be used as a technical data basis for assigning an appropriate receiving group data set to a particular user.

[0014] In this process, the matching device may be configured to determine an allocation of a particular receiving group data set to a particular user based on a comparison of the user data and the receiving group data set.

[0015] In this context, the matching device may be configured to assign a receiving group data set that corresponds to:

number

[0016] In this way, the minimization function defined determines the number of users of a given group I who view a staged video sequence j during a certain time segment t. j,t The optimal augmentation a for is selected. Alternatively, the matching device may select an optimal user group l for each time segment t of a rendition video sequence j having an augmentation a given j,t It may also be configured to determine Also, from a given augmentation collection A and all user groups I, we can find the optimal augmentation a and the optimal user group I j,t A matching device that simultaneously determines In a video broadcasting system, the matching device may be a part of the server-side video distribution device or a part of the user-side video output device. That is, the matching device may be provided in a centralized or distributed manner. However, in either case, it is guaranteed that the switching data determined or calculated in a centralized or distributed manner is transmitted to the switching device of a specific video output device of a specific user.

[0017] In a video broadcasting system, a matching device may be configured to receive a receiving group data set from a presentation video device.

[0018] In other words, a matching device, which may be implemented on the server side or on the user side, in combination with a switching device can ensure that a particular rendered video sequence is received and displayed on a video output device.

[0019] In a video broadcasting system, each staged video sequence can include multiple sections with virtually augmented image content, with the sections of each staged video sequence starting and ending at the same time, allowing for a synchronized switch time from one staged video sequence to another, making the switch between the two staged video sequences unnoticeable to a particular user.

[0020] In a video broadcasting system, a staged video device can be configured to receive a video sequence from a captured real event, generate from said video sequence a plurality of staged video sequences augmented with virtual content and transmit said video sequences to a video distribution device, in the process the staged video device can be part of a television broadcasting system, for example as known from DE 10 2016 119 637 A1.

[0021] The video broadcasting system described above can therefore also be combined with or connected to a television broadcasting system as known from DE 10 2016 119 637 A1.

[0022] Therefore, an auxiliary television broadcasting system, A plurality of television cameras configured to capture camera images and output them as camera signals; At least one production image device or production unit configured to generate a series of production images from the camera images of the plurality of television cameras and output the series of production images as production signals; a plurality of analysis modules, one analysis module assigned to each television camera, the analysis modules being configured to receive camera images in a synchronously buffered manner, the analysis modules being configured to analyze each individual camera image, to calculate camera image metadata and a camera image hash value associated with each camera image, and to output the camera image metadata and the camera image hash value for each camera image asynchronously; a synchronization module connected to the production video device or the production unit and the analysis module, configured to receive the production video from the production video device or the production unit in a synchronous buffering manner, calculate an associated production video hash value for each production video, and receive camera video metadata and an associated camera video hash value for each camera video from the analysis module in an asynchronous buffering manner, further configured to compare the production video hash value with the camera video hash value, and based on the comparison, assign the associated camera video metadata of the camera video to the current production video as production video metadata, and output the production video metadata in a synchronous buffering manner; a multiplication module coupled to the synchronization module and the staged video device or the staged unit, the multiplication module configured to receive the staged video metadata from the synchronization module in a synchronous buffering manner and to receive the staged video from the staged video device or the staged unit in a synchronous buffering manner, the multiplication module further configured to compensate for a time offset between the staged video and associated staged video metadata, generate augmented staged video modifications of at least one subsection of the staged video based on the staged video metadata, such that an augmented staged video is generated, and output the augmented staged video; A supplementary television broadcasting system has also been proposed.

[0023] Such a television broadcasting system comprises a video broadcasting system connectable to a server-side video distribution device, in particular an OTT platform, configured to receive and process multiple staged video sequences captured at real events and augmented with virtual content from a staged video device or a staged unit, and at least one user-side video output device, in particular a playback device for streaming, connectable or connected to the video distribution device and configured to receive at least one of the multiple staged video sequences and output the received staged video sequence or one of the received staged video sequences, in particular for display on a display. In this process, it is envisaged that the video output device includes a switching device configured to automatically switch to another received staged video sequence during the output of one staged video sequence and output the other staged video sequence.

[0024] Further advantages and details of the invention will become apparent from the following description of embodiments with reference to the drawings. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a simplified schematic diagram illustrating an example of a video broadcasting device. [Diagram 2] FIG. 1 is a diagram showing a simplified schematic diagram of functions of a video broadcasting device. [Diagram 3] FIG. 13 is a simplified schematic diagram showing another example of a video broadcasting device. [Figure 4] 2 is a simplified schematic diagram of a further example of a video broadcasting device; [Diagram 5] FIG. 4 is a simplified schematic diagram showing the connection between a known television broadcasting system and the video broadcasting devices of FIGS. 1 to 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] 1 is a schematic, simplified diagram of a video broadcasting device 50. The video broadcasting device 50 includes, inter alia, a video delivery device 52 configured as a so-called over-the-top platform (OTT platform), which is configured to receive from a delivery device 54 a number of staged video sequences LF1, LF2, LF3, LFN captured at a real event and augmented with virtual content, and process these.

[0027] In particular, the staged video device 54 is capable of augmenting each staged video sequence LF1, LF2, LF3, LFN with different virtual content. In this way, each staged video sequence LF1, LF2, LF3, LFN can be augmented with virtual content that is suitable for, for example, a particular group of end users.

[0028] The video broadcasting device 50 further includes at least one user-side video output device 56, particularly a playback device for streaming, which is connectable or connected to the video distribution device 52. The video output device 56 can be, for example, a so-called standard streaming video playback device based on technologies such as MPEG DASH (Dynamic Adaptive Streaming over HTTP) and HLS (http Live Streaming).

[0029] 1 depicts only a single video output device 56. However, it will be appreciated that video delivery device 52 may be connected to multiple video output devices 56. This is particularly illustrated by content delivery network (CDN) component 58, which may be part of video delivery device 52.

[0030] The video output device 56 is further configured to receive a plurality of presentation video sequences LF1, LF2, LF3, LFN, and to output, in particular for display on a display 60, one of the received presentation video sequences LF1. The video output device 56 includes a switching device 62 which is configured to automatically switch to other received performance video sequences LF2, LF3, LFN during output of one performance video sequence LF1, and output the other performance video sequences.

[0031] The video broadcasting system 50 includes a matching device 64, which is configured to transmit switching data SD to the switching device 62. At this time, the switching device 62 is configured to output the performance video sequences LF1, LF2, LF3, and LFN assigned to the switching data SD.

[0032] The matching device 64 is configured to receive a plurality of receiving group data sets EGD1, EGD2, EGD3, EGDN, where the number of receiving group data sets EGD1, EGD2, EGD3, EGDN is equal to the number of presented video sequences LF1, LF2, LF3, LFN augmented with virtual content.

[0033] Each receiving group data set EGD1, EGD2, EGD3, EGDN contains information about the virtual content for which the assigned presentation video sequence LF1, LF2, LF3, LFN is augmented. Furthermore, each receiving group data set EGD1, EGD2, EGD3, EGDN contains information about the user target group for which the corresponding virtual content should be presented.

[0034] In the video broadcasting system 50, the matching device 64 is arranged to generate the switching data SD as a function of the receiving group data sets EGD1, EGD2, EGD3, EGDN.

[0035] Additionally, the matching device 64 is configured to assign one of the receiving group data sets to a user of the video output device 56. The assignment of a particular receiving group data set EGD1, EGD2, EGD3, EGDN to a particular user may be determined based on a comparison of the user data with the receiving group data sets EGD1, EGD2, EGD3, EGDN.

[0036] The video broadcasting system 50 described above with reference to FIG. 1 and its functionality will now be described in detail with reference to FIG.

[0037] 2 shows three staged video sequences LF1, LF2, and LF3. The staged video sequences LF1, LF2, and LF3 each include a plurality of sections A0 to A8 having virtually expanded video content, and the sections A0 to A8 in each staged video sequence LF1, LF2, and LF3 start and end at the same time, as shown by the vertical dashed lines.

[0038] Furthermore, two video output devices 56 are depicted in a simplified manner in FIG. 2 and can be understood as representing users P1 and P2, respectively.

[0039] 2, by using the above-described architecture of the video broadcasting device 50, each of the rendered video sequences LF1, LF2, and LF3 can be divided chronologically into sections A0 to A8 or periods. During these periods, each viewer or user P1, P2 can view augmentation by virtual content composed of sections A0 to A8 of the different rendered video sequences LF1, LF2, and LF3.

[0040] 2, user P1 watches staged video sequence LF2 in section A0, then watches staged video sequence LF1 in sections A1 and A2, watches staged video sequence LF2 (again) in section A3, watches staged video sequence LF3 in sections A4 to A7, and watches staged video sequence LF1 in section A8. The sequence of sections A0 to A8 from staged video sequences LF1, LF2, and LF3 is shown to user P1 by a solid black arrow.

[0041] 2, user P2 views staged video sequence LF3 in sections A0 to A2, then views staged video sequence LF2 in sections A3 and A4, views staged video sequence LF1 in sections A5 and A6, and views staged video sequence LF2 in sections A7 and A8. The sequence of sections A0 to A8 from staged video sequences LF1, LF2, and LF3 is illustrated with a dashed black arrow pointing to user P2.

[0042] The system configuration of the video broadcasting device 50 described above allows the provision of the rendered video sequences LF1, LF, LF3 to be utilized to present the users P1, P2 with the rendered video sequences LF1, LF2, LF3 that best fit their profile, particularly their user data, during each section A0-A8. That is, each user P1, P2 will see a personalized sequence (feed) made up of the most suitable sections from the rendered video sequences LF1, LF2, LF3, where the suitable section A0-A8 is selected from the various rendered video sequences LF1, LF2, LF3 based on the above-mentioned switching data SD having a relationship, particularly, between the user data and the receiving group data set EGD.

[0043] Thus, the video broadcasting system 50 presented herein can be used to address, by means of virtual augmentation, a particular selection or part of a user target group in each section A0 to A8, which can also be called a time interval or period. This selection can vary from section to section. As can be seen from Fig. 2, both users P1, P2 see the same staged video sequence LF2 during section A3, and therefore also see the same virtual augmentation contained in this section A3 from the staged video sequence LF2. In all other sections A0 to A2 and A4 to A8, users P1, P2 see different staged video sequences.

[0044] User P1 may, for example, use a first type of device as video output device 56. User P2 may, for example, use a second type of device as video output device 56. Information regarding the type of device may, for example, be part of the user data.

[0045] For example, the staged video sequence may include virtual augmentations in sections A8, A2, A1, respectively, which relate to a first type of device used by user P1. Thus, the receiving group data set EGD1 includes information such that sections A8, A2, A1 are targeted to users with a first type of device. Thus, by comparing the receiving group data set EGD1 with the user data of user P1, the control data SD for user P1 may be determined such that sections A8, A2, A1 of the staged video sequence LF1 are displayed to user P1.

[0046] Sections A8, A2 and A1 from the staged video sequence are not of interest to user P2, since he uses a second type of device. However, for example, staged video sequence LF1 may contain virtual augmentations in sections A6 and A5, which relate to the second type of device used by user P2. The receiving group data set EGD2 thus contains information such that sections A6 and A5 are targeted to a user with a second type of device. By comparing the receiving group data set EGD2 with the user data of user P2 described above, control data SD for user P2 can be determined such that sections A6 and A5 of staged video sequence LF1 are displayed to user P2.

[0047] For example, in sections A7, A4, A3, and A0, the staged video sequence LF1 includes virtual augmentations that allow it to target additional users using other types of devices.

[0048] FIG. 3 is an illustration similar to FIG. 1, where the matching device 64 is part of the server-side video distribution device 52. In other words, the video distribution device 52 is part of the infrastructure of the OTT platform. In such a technical implementation, the video distribution device 52 needs slightly more computing power to enable the execution of the matching device 64. A separate matching instance needs to be executed for each user P1, P2, and the additional computing power required increases with the number of users P1, P2. However, the matching device 64 can be implemented in such a way that a matching instance for one user uses only a small amount of computing power.

[0049] Figure 4 is an illustration similar to Figures 1 and 3, where the matching device 64 is part of the user-side video output device 56, 56a. For example, a standard streaming video player 56 (MPEG DASH, HLS-based or another technology) may be extended by the matching device 64. Then, this extended player 56a or this extended video output device 56a runs on each terminal device of each user P1, P2.

[0050] Both the design according to Fig. 3 and the design according to Fig. 4 are technically feasible and achievable. In the implementation according to Fig. 3, the video output device 56 (player) can remain standard compliant, i.e. does not include the (proprietary) extension 56a due to the proprietary technology [omitted!] of the matching device 64. This allows the corresponding application (video output device 56 or player) on the terminal device to remain unchanged for all users.

[0051] 5 shows, as an example, a television broadcasting system 10 known from the above-mentioned DE. This television broadcasting system 10 is composed of multiple cameras C1 and C2 and a production unit 12, and the cameras C1 and C2 transmit their camera images as camera image signals to the production unit 12. Typically, such a television broadcasting system 10 includes two or more cameras C1 and C2, which are indicated by three black dots to the right of the camera C2.

[0052] A well-known conventional processing path starts with the cameras C1, C2. A camera signal S1 is captured by a number of cameras C1, C2 (-Cn) and transmitted to the directing unit 12. Within the directing unit 12, a directed video sequence or directed video signal is compiled (or edited) from the camera footage of the individual cameras C1-Cn. The edited directed video signal S3 or the succession of directed footage is technically called world feed and is broadcast. It is pointed out that in this context, also in the case of post-production, such as repetitions, slow motion, interviews, etc., the possibility is given to capture or superimpose augmentations, in particular the same augmentations, on previously live broadcast footage. For this purpose, the directing unit can be configured to enable or disable augmentations by a control module for such post-production, during or after the live broadcast sequence.

[0053] The television broadcasting system 10 can be extended as follows: The camera signal S1 is passed to the analysis modules A1, A2 (up to An, indicated by three black dots under A2). The analysis modules A1, A2 pass the signal S2 to the selection or synchronization module 14. One analysis module A1, A2 is assigned to each camera C1, C2. For each camera image, camera image metadata M1, M2 (up to Mn) is generated by the analysis modules A1, A2 and associated hash values ​​H1, H2 (up to Hn) are calculated. Each set of metadata and hash values, i.e., for example, M1, H1 or M2, H2, is transmitted as a signal S2 by the analysis modules A1, A2 to the synchronization module 14. The synchronization module 14 further receives a staged video signal S3a or staged video from the staged unit. The synchronization module 14 passes modified metadata S4 of the camera currently displayed in the staged video to the multiplication module 16. The multiplication module 16 generates a plurality of extended staged video signals S5 to be broadcast. The analysis modules A1, A2, the synchronization module 14 and the multiplication module 16 are controlled by a control module 18. The control module 18 is controlled via the directing unit 12.

[0054] For completeness, the diagram of Figure 5 shows an optional LED system 20 that transfers video data to the control module 18 via a video transmission protocol, e.g., HDMI or DVI, but is not required for the basic functionality of the television broadcast system 10.

[0055] In such a television broadcasting system 10, the video broadcasting system 50 described above with reference to Figures 1 to 4 can start or be connected with the multiplication module 16 generating a plurality of extended effect video signals S5. The plurality of effect video signals S5 can therefore be called effect video sequences LF1, LF2, and LF3. That is, the effect video device 54 in Figures 1, 3, and 4 is a simplified representation of the components of the television broadcasting system 10 described above, enclosed in a dashed rectangle.

Claims

1. A server-side video distribution device (52) configured to receive and process a plurality of staged video sequences (LF1, LF2, LF3, LFN) captured at a real event and extended with virtual content from a staged video device (54), in particular an OTT platform; at least one user-side video output device (56), in particular a playback device for streaming, connectable or connected to the video distribution device (52) and configured to receive at least one of the multiple staged video sequences (LF1, LF2, LF3, LFN) and to output the received staged video sequence (LF1, LF2, LF3, LFN) or one of the multiple received staged video sequences (LF1, LF2, LF3, LFN), in particular to display the same on a display (60); The video output device (56) is a video broadcasting system (50) including a switching device (62) configured to automatically switch to the received other performance video sequence (LF1, LF2, LF3, LFN) while outputting the one performance video sequence (LF1, LF2, LF3, LFN) and output the other performance video sequence (LF1, LF2, LF3, LFN).

2. a matching device (64) configured to transmit switching data (SD) to said switching device (62); The video broadcasting system (50) according to claim 1, characterized in that the switching device (62) is configured to output one performance video sequence (LF1, LF2, LF3, LFN) assigned to the switching data (SD).

3. The matching device (64) is configured to receive a plurality of receiving group data sets (EGD1, EGD2, EGD3, EGDN); The video broadcasting system (50) of claim 2, characterized in that the number of the receiving group data sets (EGD1, EGD2, EGD3, EGDN) is the same as the number of multiple performed video sequences (LF1, LF2, LF3, LFN) extended with virtual content.

4. 4. The video broadcasting system (50) according to claim 3, characterized in that the matching device (64) is configured to generate the switching data (SD) in response to the receiving group data sets (EGD1, EGD2, EGD3, EGDN).

5. 5. A video broadcasting system (50) as claimed in any one of claims 2 to 4, characterized in that the matching device (64) is configured to assign one of the receiving group data sets (EGD1, EGD2, EGD3, EGDN) to a user (P1, P2) of the video output device (56).

6. 6. The video broadcasting system (50) of claim 5, wherein the matching device (64) is configured to determine an allocation of a particular receiving group data set (EGD1, EGD2, EGD3, EGDN) to a particular user (P1, P2) based on a comparison of user data with the receiving group data sets (EGD1, EGD2, EGD3, EGDN).

7. 7. The video broadcasting system (50) of claim 6, wherein the matching device (64) is adapted to assign the following receiving group data sets (EGD1, EGD2, EGD3, EGDN): [0010] where Dt is the distance measurement over a period of time t. Pi is the user data (profile) of user i. Pa is the received group data set from the content provider. i is an individual user out of all current users I. j is a single rendition video sequence. t is the time segment of one virtual augmentation. a is an augmentation of augmentation collection A.

8. 8. The video broadcasting system (50) according to claim 2, wherein the matching device (64) is a part of the server-side video distribution device (52) or a part of the user-side video output device (56, 56a).

9. A video broadcasting system (50) as described in any one of claims 2 to 8, characterized in that the matching device (64) is configured to receive the receiving group data sets (EGD1, EGD2, EGD3, EGDN) from the production video device (54).

10. Each of the presentation video sequences (LF1, LF2, LF3, LFN) includes a plurality of sections (A0 to A8) having virtually expanded video content; A video broadcasting system (50) according to any one of claims 1 to 9, characterized in that the sections (A0 to A8) in each performance video sequence (LF1, LF2, LF3, LFN) start at the same time and end at the same time.

11. The video broadcasting system (50) of any one of claims 1 to 10, characterized in that the staged video device (54) is configured to receive a video sequence from a captured real event, generate a plurality of staged video sequences (LF1, LF2, LF3, LFN) augmented with virtual content from the video sequence, and transmit the generated video sequences to the video distribution device (52).

Images