Presentation track for edited live streaming
By independently streaming in streaming media services, the problems of live media streaming and user experience customization are solved by independently streaming the track and production effects of high-resolution multi-camera viewing angle live media streaming and user experience customization are achieved, and efficient media streaming and user interaction are achieved.
Patent Information
- Application Number
- CN202410679845.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-05-28
- Filing Date
- 2024-05-29
- Publication Date
- 2024-12-03
AI Technical Summary
The prior art is difficult to effectively provide high-resolution multi-camera viewing angle live media streaming, and existing media streams are difficult to independently manage presentation tracks and production effects, resulting in low compression efficiency of media streams and difficult to customize user experience.
By introducing independent streaming of presentation tracks and production effects in the streaming media service, the presentation track contains media playback instructions and interaction options for user devices, and the production effects include dynamic media element insertion, independent of live media streaming, Implement dynamic updates and user interaction.
Realize live media streaming with high-resolution multi-camera viewing angles, reducing the compression complexity of media streams, and improving user experience customization and media stream compression efficiency.
Smart Images

Figure CN119071513A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates generally to live media streaming, and more particularly to providing a presentation track for editorial live media streaming. Background Art
[0002] Live media consumption is a common use of electronic devices. Users can use many types of electronic devices to access and stream live media content for playback. Many of these electronic devices support high-definition (such as ultra-high definition or 4K) media playback on compatible displays.
[0003] In today's era, it is preferred to stream live events with high resolution. As an example, a user may wish to stream live events not only with multiple camera angles, but also with high resolution (such as ultra high definition or 4K). BRIEF DESCRIPTION OF THE DRAWINGS
[0004] Figure 1 A simplified network diagram is shown in block diagram form according to one or more embodiments.
[0005] Figure 2 An exemplary method for presenting an event broadcast on a user device according to one or more embodiments is shown in flowchart form.
[0006] Figure 3 An exemplary method for generating a live media stream in accordance with one or more embodiments is shown in flowchart form.
[0007] Figure 4 Another simplified network diagram including a production server device is shown in block diagram form according to one or more embodiments.
[0008] Figure 5 An exemplary process for live media streaming of an event including live media stream data and pre-recorded media segments and / or transformed data is illustrated in flowchart form in accordance with one or more embodiments.
[0009] Figure 6 An example system diagram of an electronic device is shown in accordance with one or more embodiments. DETAILED DESCRIPTION
[0010] The present disclosure relates to systems, methods and computer-readable media for providing a live media stream for an event, the live media stream including a presentation track or feed for edited live streaming. Generally disclosed is a technology for providing a live media stream including a stream providing a presentation track to a user device. Additionally, the live media stream includes additional streams (e.g., live audio / video streams, audio / video data feeds, etc.) provided to the user device based at least in part on the presentation track. Disclosed is a technology for providing additional streams of high resolution (e.g., 4K or ultra-high definition (UHD)) from an event. An event may be, for example, a concert, a conference or a sporting event. In some embodiments, the user device may be a mobile computing device, a tablet computing device, a set-top box connected to a high-definition display screen, a head-mounted device, etc.
[0011] According to one or more embodiments, the disclosed technology satisfies the need in the art to provide a presentation track in the form of metadata. In some embodiments, the presentation track can be streamed to a user device as part of a live media stream of an event. Alternatively, the presentation track can be a data feed, also referred to herein as a presentation feed, which can be streamed to a user device as part of a live media stream of an event.
[0012] In one embodiment, the streaming service device provides a graphical user interface to the video editor to create a presentation track during an event (e.g., in real time). The event may be captured by multiple cameras (e.g., 4K cameras), each of which provides an information stream as part of a live media stream. The presentation track may include, for example, a series of commands and instructions on how to display the event on a user device. For example, for an event captured by multiple cameras, the presentation track may include an indication of when the user device switches the camera feed or when to stop streaming information from the first camera and start streaming information from the second camera (e.g., at time X, switching from camera Y to camera Z). The camera feed (e.g., a 4K video feed from a 4K camera) may be provided at original resolution. The presentation track may be encapsulated as a metadata file. In some embodiments, the metadata file may be static metadata, so that the metadata file is predefined based on a pre-captured camera feed. Additionally or alternatively, the presentation track may be a dynamic metadata file that is continuously updated over time and streamed to the user device. In some embodiments, the presentation track may be sent via a streaming protocol (e.g., HLS or Dash protocol). For example, a dynamic metadata file may be continuously updated over the duration of an event, such as a concert or sporting event that lasts for a long period of time (e.g., 2-3 hours). In some embodiments, a presentation track may be streamed to a user device for the duration of the event upon request from the user. Additionally, the presentation track may be streamed to the user device on a frequency band or channel different from the frequency band or channel used to stream information captured by the camera. The presentation track may also specify a call to previously stored or previously streamed video and / or audio information.
[0013] According to one or more embodiments, the presentation track may include one or more production selections, such as actions, events, commands, etc., associated with one or more streams of the live media stream. The user device may provide a graphical user interface or overlay via a display that enables a user to interact with the displayed media content during the presentation or event (e.g., scoreboard, closed captions, etc.). In addition, according to one or more embodiments, the user interface may provide one or more media controls to the user to customize their viewing experience of the event. The media controls may include, but are not limited to, camera viewing angles, volume levels / mixes (e.g., crowd noise with a narrator), audio input selections (e.g., different narrators, language selections), event information, etc.
[0014] According to one or more embodiments, the presentation track may also include instructions for providing one or more production effects during the live media stream. Production effects may include, for example, different types of actions, events, commands, etc. Production effects may include different types of media elements for display during the live media stream, such as camera transitions (e.g., dimming or fading in / out), video clips, or graphics when switching between camera feeds. Additionally or alternatively, production effects may include, but are not limited to, displaying graphic overlays, images, animations, videos, interactive shapes, or combinations thereof. Production effects may, for example, be inserted into the live stream according to the instructions of the presentation track, such as at a specific timestamp or in response to a predefined event (e.g., switching camera angles). Production effects may, for example, be inserted into the live media stream during scene transitions between a first stream from a first camera source and a second stream from a second camera source, during changes in camera angle, camera calibration changes, or similar instant applications.
[0015] According to one or more embodiments, the disclosed technology meets the needs of the art for providing presentation tracks and production effects separated from live data streams to user devices to produce live media streaming events. In an exemplary embodiment, the production effects may be transmitted or streamed to the user device together with the presentation track or on the same frequency band as the presentation track. The production effects may include additional content, such as graphic or text overlays, pre-recorded media clips, etc. Additionally or alternatively, the production effects may include instructions to be applied to the received stream, such as visual processing, conversion or other editing effects. In some embodiments, the production effects may be stored on a data repository of the user device or accessible by the user device. By sending separately, there is no need to hard-code the production effects into the live media stream, such as media streams obtained from different camera sources. Instead, the presentation track will provide instructions on when to apply the production effects to the received media stream during the live media streaming event (e.g., at time t=1, obtain the stream from camera A; at time t=2, provide a fade-out / fade-in transition; at time t=3, obtain the stream from camera B). When production effects are hard-coded into the media stream, the media stream may become difficult to compress due to high contrast and high frequency changes. By providing the production effects separately, the media stream can be provided as a smaller compressed media file.
[0016] According to one or more embodiments, the disclosed technology provides a data stream or data feed for broadcasting an event. The data stream may be provided as an in-band signal, an out-of-band signal, or a combination or variant thereof. For example, in some embodiments, a presentation track as described herein may be provided separately from a live data stream (e.g., a video of a live event broadcast). In this example, the presentation track may be provided as an independent data feed as out-of-band data, and the live data stream may be provided as in-band data. In this example, the presentation track may be streamed to a device (e.g., a client device) on a dedicated channel and kept separate from the live media stream.
[0017] In the following description, for the purpose of explanation, a lot of specific details are set forth in order to provide a thorough understanding of the disclosed concept. As part of this specification, some drawings of the present disclosure represent structures and equipment in the form of block diagrams to avoid blurring the novel aspects of the disclosed embodiments. In this context, it should be understood that reference to reference numerals (e.g., 100) without associated identifiers refers to all instances (e.g., 100a and 100b) of reference numerals with identifiers. In addition, as part of this specification, some drawings of the present disclosure can be provided in the form of flow charts. The blocks in any particular flow chart can be presented in a particular order. However, it should be understood that the specific flow of any flow chart is only used to illustrate an embodiment. In other embodiments, any various components depicted in the flow chart can be deleted, or components can be executed in different orders, or even components can be executed simultaneously. In addition, other embodiments may include additional steps that are not shown as a part of the flow chart. The language used in the present disclosure is mainly selected for readability and guidance purposes, and it may not be selected to depict or limit the disclosed subject matter. Reference in this disclosure to "one embodiment" or "an embodiment" is meant to include a particular feature, structure, or characteristic described in connection with that embodiment in at least one embodiment, and multiple references to "one embodiment" or "an embodiment" should not be construed as necessarily all referring to the same or different embodiments.
[0018] It will be appreciated that in the development of any actual implementation (as in any development project), many decisions must be made to achieve the developer's specific goals (e.g., to meet system and business-related constraints), and that these goals will vary from one implementation to another. It will also be appreciated that such a development effort may be complex and time-consuming, but will nevertheless be a routine undertaking for those of ordinary skill in the art who benefit from the image capture disclosed herein.
[0019] For purposes of this disclosure, media items are referred to as "videos". However, in one or more embodiments, the media items referred to as "videos" may be any type of media items, including but not limited to audio / video media items, radio stations, streaming data items, podcasts, music videos, animations, video graphics, graphic overlays, etc. Additionally, for purposes of this disclosure, the data feeds of user device instructions or indications are referred to as "presentation tracks" or "presentation feeds". However, in one or more embodiments, the data feeds referred to as "presentation tracks" or "presentation feeds" may be any type of data feeds for directing the presentation of a broadcast, including but not limited to commands directing the presentation of a broadcast, camera source data, timestamp data, conversion data, graphics data, or combinations or variations thereof.
[0020] refer to Figure 1 , depicts a simplified block diagram of a streaming media service 100 connected to a content delivery service 120 and a user device 140, for example, via a network 150. The user device 140 may be a multi-function device such as a set-top box, a digital media player, a mobile phone, a tablet computer, a personal digital assistant, a portable music / video player, a wearable device, or any other electronic device that includes a media playback system that enables video content to be displayed on a high-definition screen.
[0021] The streaming media service 100 may include one or more servers or other computing or storage devices that may include various modules and storage devices. Although the streaming media service 100 is depicted as including various components in an exemplary manner, in one or more embodiments, the various components and functions may be distributed on multiple network devices (such as servers, network storage devices, etc.). In addition, additional components may be used, and certain combinations of the functions of any components may be combined. In general, the streaming media service 100 may include one or more memory devices 112, one or more storage devices 114, and one or more processors 116 (such as a central processing unit (CPU) or a graphics processing unit (GPU)). In addition, the processor 116 may include multiple processors of the same or different types. The memory 112 may include one or more different types of memory, which can be used to perform device functions in conjunction with the processor 116. For example, the memory 112 may include cache, ROM and / or RAM. The memory 112 may store various programming modules during execution, including the media management module 102 and the video editing module 104.
[0022] Streaming media service 100 can store media files, media file data, music directory data, video directory data, media or video clips, statistics, graphics data, etc. Additional data may include, but are not limited to, presentation track data, dynamic metadata files, etc. Streaming media service 100 can store such data in media repository 106 in storage device 114. Storage device 114 may include one or more physical storage devices. The physical storage device may be located in a single location, or may be distributed in multiple locations, such as multiple servers. Media files may be marked or tagged for future reuse. Tag data may be in the form of a unique identifier or code for rapid identification and retrieval for presentation on user devices. In one or more embodiments, tag data may include information about media data, such as date, time, length, file type, and / or media content. Additionally or alternatively, tag data may include video clip information, event information, highlight data, etc.
[0023] In some embodiments, the streaming media service 100 may be communicatively coupled to a plurality of cameras 160(a), 160(b), and 160(c). Cameras 160(a), 160(b), and 160(c) may be directly connected to the streaming media service 100, such as via a local area network, network 150, or a combination or variation thereof. The illustration of three cameras is for exemplary purposes only, and the camera group may include less than or more than three cameras. Each camera may be assigned a unique identifier, such as a serial number or an alphanumeric identification code. The cameras may have different capabilities, such as different zoom capabilities, resolution capabilities, connection capabilities, etc. Each camera may include audio and video capture capabilities. In some embodiments, for example, camera 160(a) may be a fixed camera, camera 160(b) may be a roving camera (such as a camera carried by a camera operator during an event), and camera 160(c) may be attached to an unmanned aerial vehicle (such as a drone).
[0024] Returning to the streaming media service 100, the memory 112 includes modules including computer readable code that can be executed by the processor 116 to cause the media service 100 to perform various tasks. As depicted, the memory 112 may include a media management module 102 and a video editing module 104. According to one or more embodiments, the media management module 102 manages media content, such as media content captured by one or more camera devices (e.g., cameras 160(a)-160(c)). The media management module 102 may provide media content based on a request from a remote location such as a content delivery service 120 or a user device 140. The media management module 102 may store the media content on a media repository 106 of a storage device 114 for subsequent retrieval. Portions of the media content (such as media clips, highlights, segments, etc.) may be stored on the media repository 106. Additionally or alternatively, the media content may be stored on a media repository 148 of a storage device 144 of a user device 140.
[0025] The memory 112 also includes a video editing module 104. In one or more embodiments, the video editing module 104 can cause the display of a graphical user interface (GUI) to receive input from a user to edit the video media content displayed by the GUI. The user can interact with the GUI of the video editing module 104 via various methods, including but not limited to physical touch, non-touch gestures and / or retinal tracking. The video editing module 104 can receive audio and video data feeds from multiple sources (such as media storage repository 106, camera 160 (a)-160 (c) or a combination thereof). In some embodiments, the video data feed can be received at a high resolution (e.g., 4K or ultra-high definition). Additionally or alternatively, the video data feed can be reduced to a lower resolution (e.g., 1080p) before being received by the video editing module 104. In some embodiments, the video editing module 104 can reduce the received video data feed to a lower resolution. In some embodiments, the received video data feed can be displayed on a part of the GUI for user viewing and editing. Alternatively, multiple video data feeds can be received and displayed on a dedicated part of the GUI for viewing. The video data feed may be received in real time or near real time while an event (such as a sporting event) is ongoing. During an event, a user may create a presentation track that may include a series of commands and instructions on how to present a media broadcast of the event on a user device. For example, one or more of the commands or instructions may indicate when to switch between video data feeds, when and where to display graphic overlays (e.g., statistics), when to present highlights or instant replays, or a combination thereof. The presentation track may be encapsulated as a metadata file. Additionally or alternatively, the presentation track may be a dynamic metadata file that is continuously updated over time and streamed to a user device. For example, a dynamic metadata file may be continuously updated over the duration of a live event (such as a concert or sporting event that lasts for a long period of time (e.g., 2 hours-3 hours)). As described herein, the presentation track may be streamed directly to a user device (such as user device 140) from a streaming media service 100 or via a content delivery service 120.
[0026] The content delivery service 120 may include one or more servers or other computing or storage devices that may include various modules and storage devices. Although the content delivery service 120 is depicted as including various components in an exemplary manner, in one or more embodiments, the various components and functions may be distributed on multiple network devices (such as servers, network storage devices, etc.). In addition, additional components may be used, and certain combinations of the functions of any components may also be combined. In general, the content delivery service 120 may include one or more memory devices 124, one or more storage devices 122, and one or more processors 126 (such as a central processing unit (CPU) or a graphics processing unit (GPU)). In addition, the processor 126 may include multiple processors of the same or different types. The memory 124 may include one or more different types of memories, which may be used to perform device functions in conjunction with the processor 126. For example, the memory 124 may include cache, ROM and / or RAM. The memory 124 may store various programming modules during execution.
[0027] Content delivery service 120 can store media files, media file data, music directory data, video directory data, media or video clips, statistics, graphics data, etc. Additional data may include, but are not limited to, presentation track data, dynamic metadata files, etc. Content delivery service 120 can store such data in a media repository 128 within storage device 122. Storage device 122 may include one or more physical storage devices. The physical storage device may be located in a single location, or may be distributed in multiple locations, such as multiple servers. Media files may be marked or tagged for future reuse. Tag data may be in the form of a unique identifier or code for rapid identification and retrieval for presentation on a user device. In one or more embodiments, tag data may include information about media data, such as date, time, length, file type, and / or media content. Additionally or alternatively, tag data may include video clip information, event information, highlight data, etc.
[0028] In some embodiments, the content delivery service 120 may be communicatively coupled to a plurality of cameras, such as cameras 160(a), 160(b), and 160(c). Cameras 160(a), 160(b), and 160(c) may be directly connected to the content delivery service 120, such as via a local area network or a wide area network (such as network 150), or a combination or variation thereof. Additionally, the content delivery service 120 may receive video feeds from one or more sources (such as cameras 160(a), 160(b), and 160(c)). In one or more embodiments, the video feeds may be high resolution, such as ultra high definition or 4K. Data from the video feeds may be stored on a storage device, such as a media repository 128 within storage device 122. In some embodiments, the video feeds may be provided to a user device, such as user device 140. For example, one or more video feeds of a broadcast or streaming event may be provided in response to a request from a user device. The request may be made directly by the user device or based on a presentation track. As described herein, a presentation track may provide a series of commands directing the presentation of a broadcast, the commands including camera source data as well as timestamp data. Additionally, transition data, such as graphics to be displayed or fade features, may be provided when switching between one camera source to another during a broadcast. The video data feed may be provided at a high resolution, such as 4K or Ultra HD.
[0029] The user device 140 may include one or more memory devices 142, one or more storage devices 144, and one or more processors 146 (such as a central processing unit (CPU) or a graphics processing unit (GPU)). In addition, the processor 146 may include multiple processors of the same or different types. The memory 142 may include one or more different types of memory, which can be used to perform device functions in conjunction with the processor 146. For example, the memory 142 may include a cache, a ROM, and / or a RAM. The memory 142 may store various programming modules including computer-readable codes that can be executed by the processor 146 to enable the user device 140 to perform various tasks. As depicted, the memory 142 may include a media player 152 and a stream module 154. According to one or more embodiments, the media player 152 may provide media playback of content, such as video data content received from the content delivery service 120, the camera 160, or the streaming media service 100.
[0030] In some embodiments, media player 152 may provide playback as directed by stream module 154. Stream module 154 may receive a presentation track, as described herein, and perform various tasks to provide playback of a live streaming media event broadcast.
[0031] Figure 2An exemplary method for generating a presentation track as described herein is shown in flowchart form. The method may be performed, for example, by a server device such as Figure 1 For the purpose of explanation, the video editing module 104 on the streaming media service 100 of FIG. Figure 1 The following steps are described in the context of . However, various actions may be performed by alternative components. In addition, various actions may be performed in different orders. In addition, some actions may be performed simultaneously, and some actions may not be required, or other actions may be added.
[0032] The flowchart begins at 205, where the streaming media service 100 can initiate an event broadcast. Initiating the event broadcast can include initializing network connections between devices required for the event broadcast, such as cameras 160(a)-160(c), the streaming media service 100, and the content delivery service 120. The event broadcast initialization can also include initializing modules, such as the media management module 102 and the video editing module 104 of the streaming media service 100, and the stream module 154 of the user device 140. In some embodiments, initiating the event broadcast can include causing the cameras 160(a)-160(c) to capture audiovisual data, such as through one or more image sensors.
[0033] The flow chart continues at 210, where the streaming media service 100 may pre-process the video feeds received from the cameras 160 (a)-160 (c). For example, the streaming media service may pre-process by reducing all video feeds of the event broadcast. Additionally or alternatively, the size of the video feed of the event broadcast may be reduced for real-time processing using, for example, a decimation filter, a video compressor, a video downsampler, or a combination thereof. Video feeds may be received from multiple sources, such as cameras 160 (a)-160 (c). In some embodiments, the camera may capture video data using a high-definition camera lens (e.g., a 4K video feed). The video feed may then be reduced to a lower resolution (such as 1080p) for further processing.
[0034] The flow chart continues at 215, where the video editing module 104 of the streaming media service 100 can be used to create or update a presentation track. As described herein, a presentation track can be used to provide an indication of how to display an event broadcast via a display. The video editing module 104 can process a reduced video feed and be used to generate a low-resolution version of an event broadcast, or otherwise create an edited version of audio-visual content or live streaming data. The reduction of the video content feed enables real-time editing of the feed with reduced computing costs. For example, a presentation track can be created based on a reduced data feed, thereby requiring less bandwidth and other computing resources required for high-quality streams (e.g., 4K video streams). Then, based on the indication of the presentation track, the playback of the original video feed (e.g., 4K video streams) is provided to a user device (e.g., user device 140). That is, the presentation track can encode the presentation indication of the video feed. In some embodiments, a graphical user interface (GUI) can be provided to the user to interact with one or more features of the video editing module 104. The interactive features of the GUI may include, but are not limited to, camera source selection, graphic overlay selection (e.g., statistics, sports scores, fun facts), video transition selection, audio feed selection (e.g., language, commentator, crowd noise), ambient dimming, insertion of pre-recorded video clips (e.g., highlights, replays), etc. In some embodiments, the graphic overlays are transmitted separately to the client device for display at certain times during the broadcast (e.g., timed metadata) rather than being "burned" into the video feed. The inserted clips may be retrieved from a number of different locations, such as media repository 106, media repository 128, or media repository 148. In some embodiments, the presentation track may be streamed directly from streaming media service 100 to 140 and dynamically updated over time (e.g., during the event broadcast).
[0035] In some embodiments, in 220, multiple streams may be provided to user device 140. For example, a user may choose to view only a low-resolution version of the event. In this case, only a downscaled version of the event will be streamed to user device 140 directly from streaming media service 100 or via content delivery service 120. Alternatively, the user may choose to receive a high-resolution version of the event broadcast. In this case, a presentation track of the event will be streamed to user device 140 directly from streaming media service 100 or via content delivery service 120. User device 140 will then obtain a media stream based on the presentation track, such as Figure 2 In 225, when the event broadcast ends, the process ends. In 225, when the event broadcast is still occurring, the process continues to 215 to continue the creation of the presentation track and the streaming of the event based on the indication of the presentation track.
[0036] Figure 3 An exemplary method for playing a streaming event is shown in flowchart form. The method may be performed by one or more modules (such as Figure 1 For the purpose of explanation, the media player 152 or streaming module 154 on the user device 140 will be implemented. Figure 1 The following steps are described in the context of . However, various actions may be performed by alternative components. In addition, various actions may be performed in different orders. In addition, some actions may be performed simultaneously, and some actions may not be required, or other actions may be added.
[0037] The flowchart begins at 305, where the user device 140 obtains a presentation track from an external source or location, such as the streaming media service 100. According to one or more embodiments, the presentation track may be obtained from the streaming media service 100, such as Figure 1 As shown. The presentation track may be updated dynamically and continuously. For example, the presentation track may be updated for the duration of the event broadcast. The presentation track may include one or more instructions, instructions, or commands on how to present the event broadcast on user device 140. For example, one or more instructions may include frame-by-frame instructions, such as timing data for video transitions (fading / obscuring), graphic overlays (e.g., statistics, information data), video animations, or a combination thereof. Additionally or alternatively, a command may be provided to indicate which video feed or feeds should be presented on a display of user device 140 via media player 152.
[0038] The flow chart continues at 310, where one or more media streams (e.g., video feeds) may be obtained by the stream module 154 based on the indication of the presentation track. In some embodiments, multiple video feeds may be provided on a display, such as via a split screen interface. The video feeds may be provided directly from a source, such as from one or more cameras 160 (a)-160 (c). Alternatively, the video feeds may be provided from an intermediate server such as a content delivery service 120. In addition, one video feed may be provided from a source such as a camera 160 (a), and another video feed may be provided from an intermediate server such as a content delivery service 120. The indication of the presentation track may indicate not only where to obtain one or more video feeds, but also when to switch video feeds via timing metadata. The video feeds may be provided directly to the user device 140 at a high resolution (e.g., 4K or ultra high definition).
[0039] The flow chart continues at 315, where media playback of the event broadcast is provided, such as via user equipment 140. The playback of the event broadcast can be provided in view of the presentation track, as described herein. Additionally or alternatively, users of the event broadcast can provide their own instructions to customize the event broadcast. In one example, the user can be provided with the option of switching or converting between one or more video feeds to change the camera angle of the event broadcast. In some embodiments, an interactive GUI can be provided to the user to select features (e.g., graphic overlays), switch camera feeds, or even switch audio feeds (e.g., switch between narrators of the event broadcast, switch between languages). Continue at 320, once the event ends, such as when the broadcast event is completed, the process ends. Otherwise, the process continues back to 305, to continue receiving the presentation track. As described herein, the presentation track can be continuously updated. In addition, the presentation track can be continuously streamed to the user equipment 140 for the stream module 154 to process.
[0040] Reference now Figure 4 , depicts a simplified functional block diagram of a production service 400 connected to a streaming media service 100, a content delivery service 120, and a user device 140, for example, via a network 150. The streaming media service 100 may include one or more servers or other computing or storage devices that may include various modules and storage devices. Although the production service 400 is depicted as including various components in an exemplary manner, in one or more embodiments, the various components and functions may be distributed across multiple network devices (such as servers, network storage devices, etc.). In addition, additional components may be used, and certain combinations of the functions of any components may be combined. In general, the production service 400 may include one or more memory devices 412, one or more storage devices 414, and one or more processors 416 (such as a central processing unit (CPU) or a graphics processing unit (GPU)). In addition, the processor 416 may include multiple processors of the same or different types. The memory 412 may include one or more different types of memory, which may be used to perform device functions in conjunction with the processor 416. For example, the memory 412 may include cache, ROM, and / or RAM. The memory 412 may store various programming modules during execution, including the media management module 402 and the video editing module 404 .
[0041] Production service 400 can store media files, media file data, music directory data, video directory data, media or video clips, statistics, graphics data, etc. Additional data may include, but are not limited to, presentation track data, dynamic metadata files, etc. Production service 400 can store such data in media repository 406 in storage device 414. Storage device 414 may include one or more physical storage devices. The physical storage device may be located in a single location, or may be distributed in multiple locations, such as multiple servers. Media files may be marked or tagged for future reuse. Tag data may be in the form of a unique identifier or code, for rapid identification and retrieval in order to present on a user device. In one or more embodiments, tag data may include information about media data, such as date, time, length, file type, and / or media content. Additionally or alternatively, tag data may include video clip information, event information, highlight data, etc.
[0042] In some embodiments, the production service 400 may be communicatively coupled to a plurality of cameras 160, such as Figure 1 Cameras 160(a), 160(b), and 160(c) are shown. Camera 160 may be directly connected to production service 400, such as via a local area network, network 150, or combinations or variations thereof.
[0043] Returning to production service 400, memory 412 includes modules including computer readable code executable by processor 416 to cause production service 400 to perform various tasks. As depicted, memory 412 may include media management module 402 and video editing module 404. According to one or more embodiments, media management module 402 manages media content, such as media content captured by one or more camera devices (e.g., cameras 160(a)-160(c)). Media management module 402 may provide media content based on a request from a remote location such as content delivery service 120 or user device 140. Media management module 402 may store media content on media repository 406 of storage device 414 for subsequent retrieval. Portions of media content, such as media clips, highlights, segments, graphics, animations, scene transitions, video effects, etc., may be stored on media repository 406.
[0044] The memory 412 also includes a video editing module 404, which is similar in function to the video editing module 404 described above with respect to FIG. Figure 1The video editing module 104 described in the video editing module for creating a presentation track. In one or more embodiments, the production service 400 may transmit the presentation track via an out-of-band signal. Various data flow schemes can be implemented through the disclosed technology. For example, one or more main data streams of the broadcast event may be provided as in-band data, while the presentation track or presentation feed may be provided as an independent stream as out-of-band data. For the purposes of this disclosure, in-band data is defined as data transmitted or streamed on a dedicated channel or a main channel. Additionally, out-of-band data is defined as data transmitted or streamed on a channel independent of the main channel. For example, out-of-band data is data transmitted or streamed independently of the channel used by the in-band data. The one or more main data streams may be transmitted, for example, from cameras 160 (a)-160 (c) to one or more endpoints, such as a streaming media service 100, a content delivery service 120, or a user device 140. As described herein, one or more main data streams may be high-resolution (e.g., 4K or ultra-high definition) data streams. For example, one or more primary data streams may be obtained directly from one or more of the plurality of cameras via the streaming media service 100, the content delivery service 120, or a combination or variation thereof. Continuing with this example, the user device 140 may receive an out-of-band data stream including a presentation track. The user device 140 may then obtain a data stream of the in-band stream taking into account a command, indication, or instruction to present the track. The event broadcast is then recreated for display on a display device of the user device 140.
[0045] Reference now Figure 5, an exemplary flow chart 500 is provided, which illustrates a disclosed embodiment including inserting production effects into a live media stream. As described herein, production effects may be provided using a presentation track stored on a local memory of a user device or retrieved from an external storage device. Production effects may include, for example, different types of camera transitions, such as fade-in or fade-out. Additionally or alternatively, production effects may include displaying graphic overlays, images, animations, videos, or combinations thereof. Production effects are generated and displayed on a user device based on instructions of a presentation track, commands embedded in a media stream, and the like. In one example, a first media stream may be displayed 502 at time T1, a production effect may be displayed or applied 504 at time T2, and a second media stream may be displayed 506 at time T3. Different variations may be implemented, such as providing a series of production effects between media streams obtained from different cameras during a live media stream, such as screen transitions, graphic displays, and the like. In a non-limiting example, production effects may be inserted into a live stream as indicated by a presentation track, such as at a specific timestamp or in response to a predefined event (e.g., switching camera perspectives or sources). Production effects can be inserted into a live media stream during a scene transition between a first media stream and a second media stream, during a camera angle change, a camera calibration change, or similar instantaneous scenes during a live event broadcast. In some embodiments, rather than reading image content from the media stream, commands can be embedded into the media stream so that the playback device instantly displays the transition. Additionally, at the start of media playback, a graphic and / or camera calibration grid resource can be provided to help the playback device display scene transitions and / or camera changes.
[0046] In some embodiments, the presentation track and production effects can be transmitted separately from the live media stream obtained from, for example, a camera for capturing live events. In an exemplary embodiment, the production effects can be transmitted or streamed to the user device together with the presentation track or on the same frequency band as the presentation track. By sending separately, there is no need to hard-code the production effects (e.g., fade-to-dark conversion) into the live media stream that is streamed from the camera to the user device. When the production effects are hard-coded into the media stream, the media stream becomes difficult to compress due to high contrast and high frequency changes. By providing the production effects separately, the media stream can be provided as a smaller compressed media file. For example, during the broadcast of a live media event, an image or video conversion can be displayed by the user device based on the indication proposed by the presentation track (e.g., a fade-to-dark conversion is applied at time X or when switching from camera source A to camera source B).
[0047] Reference now Figure 6 , which shows a simplified functional block diagram of an exemplary multifunction electronic device 600 according to one embodiment. The multifunction device 600 may show, for example, Figure 16, and a plurality of other devices. The multifunction electronic device 600 may include a processor 605, a display 610, a user interface 615, graphics hardware 620, device sensors 625 (e.g., proximity sensors / ambient light sensors, accelerometers, and / or gyroscopes), a microphone 630, an audio codec 635, a speaker 640, a communication circuit 645, a digital image capture circuit 650 (e.g., including a camera system), a video codec 655 (e.g., supporting a digital image capture unit), a memory 660, a storage device 665, and a communication bus 670. The multifunction electronic device 600 may be, for example, a digital camera or a personal electronic device such as a personal digital assistant (PDA), a personal music player, a mobile phone, or a tablet computer.
[0048] The processor 605 may execute instructions necessary for implementing or controlling the operation of many functions performed by the device 600 (e.g., generation and / or processing of images or videos such as disclosed herein). The processor 605 may, for example, drive the display 610 and may receive user input from the user interface 615. The user interface 615 may allow a user to interact with the device 600. For example, the user interface 615 may present various forms, such as buttons, keypads, dials, click wheels, keyboards, display screens, and / or touch screens. The processor 605 may also, for example, be a system on a chip, such as those present in a mobile device, and include a dedicated graphics processing unit (GPU). The processor 605 may be based on a reduced instruction set computer (RISC) or a complex instruction set computer (CISC) architecture or any other suitable architecture, and may include one or more processing cores. The graphics hardware 620 may be dedicated computing hardware for processing graphics and / or assisting the processor 605 to process graphics information. In one embodiment, the graphics hardware 620 may include a programmable GPU.
[0049] The image capture circuit 650 may include two (or more) lens assemblies 680A and 680B, each of which may have a separate focal length. For example, relative to the focal length of lens assembly 680B, lens assembly 680A may have a short focal length. Each lens assembly may have a separate associated sensor element 690A and 690B. Alternatively, two or more lens assemblies may share a common sensor element. The image capture circuit 650 may capture still images and / or video images. The output from the image capture circuit 650 may be processed at least in part by the following devices: one or more video codecs 655 and / or processor 605 and / or graphics hardware 620, and / or a dedicated image processing unit or pipeline incorporated in the circuit 650. The images captured in this way may be stored in the memory 660 and / or the storage device 665.
[0050] The sensor and camera circuit system 650 can capture still images and video images that can be processed in accordance with the present disclosure at least in part by the following devices: one or more video codecs 655 and / or processor 605 and / or graphics hardware 620 and / or a dedicated image processing unit incorporated within the circuit 650. The images captured in this way can be stored in memory 660 and / or storage device 665. Memory 660 may include one or more different types of media used by processor 605 and graphics hardware 620 to perform device functions. For example, memory 660 may include memory cache, read-only memory (ROM) and / or random access memory (RAM). Storage device 665 can store media (e.g., audio files, image files, and video files), computer program instructions or software, preference information, device profile information, and any other suitable data. The storage device 665 may include one or more non-transitory computer-readable storage media, including but not limited to disks (fixed hard disks, floppy disks, and removable disks) and tapes, optical media (such as CD-ROMs and digital video disks (DVDs)), and semiconductor storage devices (such as electrically programmable read-only memories (EPROMs) and electrically erasable programmable read-only memories (EEPROMs), etc. The memory 660 and the storage device 665 may be used to tangibly retain computer program instructions or codes organized into one or more modules and written in any desired computer programming language. When executed by, for example, the processor 605, such computer program codes may implement one or more of the methods described herein.
[0051] The scope of the subject matter disclosed in this disclosure should be determined with reference to the appended claims, along with the full scope of equivalents to which the claims are entitled.In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein."
Claims
1. A method for broadcasting an event, the method comprising the following steps using at least one computing device: receiving one or more data feeds including audiovisual information of the event captured by one or more image sensors; presenting a representation of each of the one or more data feeds for the event on a graphical user interface (GUI) of the at least one computing device; receiving one or more selections related to the one or more data feeds from a user of the at least one computing device; generating a presentation feed based on the received one or more selections; as well as The one or more data feeds and the presentation feed are transmitted to one or more receivers.
2. The method of claim 1 , wherein the one or more data feeds are received at a first resolution, the method further comprising: The one or more data feeds are downscaled to a second resolution. 3 . The method of claim 1 , wherein the presentation feed includes one or more indications for displaying the event on a display device.
4. The method of claim 1, wherein the presentation feed comprises one or more instructions for obtaining at least one of the one or more data feeds, wherein each of the one or more instructions comprises a timestamp for processing the instruction.
5. The method of claim 1, wherein the one or more image sensors are included in one or more camera devices.
6. The method of claim 1, wherein the one or more data feeds include video content in ultra high definition or 4K resolution. The method of claim 1 , wherein the presentation feed comprises static metadata.
8. The method of claim 1, wherein the presentation feed includes dynamic metadata.
9. The method of claim 1, wherein the one or more data feeds are transmitted using in-band signaling.
10. The method of claim 1, wherein the presentation feed is transmitted using an out-of-band signal.
11. The method according to claim 1, further comprising: receiving a subsequent selection from the user of the computing device during the event; as well as The presentation feed is dynamically updated based on the subsequent selection.
12. The method of claim 11, wherein the presentation feed is dynamically updated in real time.
13. The method of claim 11, wherein the subsequent selection comprises one or more of: modifying the presentation of audiovisual information of the one or more data feeds, switching between a first image sensor and a second image sensor of the one or more image sensors, selecting one or more graphical overlays to be rendered on the one or more data feeds, selecting more than one of the one or more data feeds to be presented simultaneously, and At least one saved media item is selected for display during the event.
14. The method of claim 1, wherein the one or more selections include selecting a first image sensor of the one or more image sensors.
15. A method comprising, capturing live streaming data from a plurality of sources at a first resolution; downscaling the live streaming data from the plurality of sources to a second resolution; creating an edited version of the live streaming data from the plurality of sources at the second resolution; generating a presentation feed based on the edited version of the live streaming data; as well as The edited version is transmitted at the second resolution, the presentation feed of the edited version is transmitted at the second resolution, and the live streaming data from the plurality of sources is transmitted at the first resolution.
16. The method of claim 15, wherein the presentation feed is transmitted via an out-of-band signal independent of the live streaming data.
17. The method of claim 15, wherein the presentation feed comprises at least one of static metadata and dynamic metadata.
18. A content capture system comprising: capturing video data from a plurality of image sensors at a first resolution; downscaling the captured video data to a second resolution; generating a presentation feed based on the downscaled captured video data, the presentation feed comprising metadata associated with a presentation indication of the captured video data at the first resolution; as well as The presentation feed is transmitted to one or more receivers.
19. The system of claim 18, wherein the first resolution is Ultra High Definition or 4K.
20. The system of claim 18, wherein the presentation feed includes dynamic metadata.
21. The system of claim 18, wherein the presentation feed includes static metadata.
22. The system of claim 18, wherein the presentation feed is generated based on one or more selections from a user of the content capture system related to one or more of the plurality of image sensors.
23. The system of claim 18, wherein the presentation feed is generated based on one or more selections related to one or more production selections related to the captured video data.
24. The system of claim 23, wherein the one or more production selections include one or more of scene transitions, pre-recorded video clips, animations, or combinations thereof.
25. A playback device, comprising: monitor; processor; and A non-transitory computer-readable medium having instructions stored thereon, wherein the instructions, when executed by the processor, cause the playback device to perform a method comprising the following steps: receiving an information stream including a presentation feed of events; obtaining one or more live media streams based on the one or more indications of the presentation feed; The one or more live media streams are displayed on the display based on the one or more indications of the presentation feed.
26. The playback device of claim 25, wherein each of the one or more live media streams represents a different camera perspective of the event.
27. The playback device of claim 25, wherein the presentation feed includes dynamic metadata.
28. The playback device of claim 25, wherein the information stream includes one or more production selections related to the one or more live media streams.
29. The playback device of claim 28, wherein the one or more production selections are presented by the playback device.
30. The playback device of claim 25, wherein the one or more live media streams are obtained from one or more camera devices.
31. The playback device of claim 25, wherein the one or more live media streams are provided in 4K or ultra high definition.
