Telepresence via VR broadcast streams

Abstract

Describes techniques that extend and / or improve the Advanced Television Systems Committee (ATSC) 3.0 television protocol in reliably delivering next-generation broadcast television services. Provides telepresence through over-the-air (OTA) virtual reality (VR) broadcast streams.

Description

Technical Field

[0001] This application relates to technological advancements that are necessarily rooted in computer technology and target digital televisions, specifically related to the Advanced Television Systems Committee (ATSC) 3.0.

Background Art

[0002] The Advanced Television Systems Committee (ATSC) 3.0 standard group is a set of numerous industry technical standards for delivering next-generation broadcast television, as shown in A / 300. ATSC 3.0 supports the provision of a wide range of television services, such as televised video, two-way services, non-real-time data delivery, and tailored advertising for numerous receiving devices from ultra-high-definition televisions to wireless phones. ATSC 3.0 also coordinates between broadcast content (referred to as "Over the Air" or OTA) and related broadband delivery content and services (referred to as "Over the Top" or OTT). ATSC 3.0 is designed to have the flexibility to easily incorporate advancements without having to comprehensively review any related technical standards as technology evolves.

Summary of the Invention

Means for Solving the Problems

[0003] On a PC, tablet, or smartphone, a mouse or touch screen can be used to pan videos with a 180-degree / 360-degree (180° / 360°) field of view and a resolution of 4K or 8K or higher. By using goggles, videos can be panned by hand controller or head movement. VR goggles can detect the natural rotation or tilt of the head to pan the image displayed inside the goggles, creating a sense of presence as if "teleporting".

[0004] This specification assumes that 180 / 360 video can be monoscopic or stereoscopic. The flat displays used herein can present monoscopic or stereoscopic video, with stereoscopic video being used to render 3D video and monoscopic video being typically considered 2D video. As used herein, “flat display” or “flat video” means video where the entire image being processed is typically limited to a 16x9 aspect ratio display, regardless of the receiving resolution, and lacks panning capabilities. Therefore, “flat” video can be considered the “main view” video of 180 / 360 video. The “main view” video can be considered the area where the director or content creator wants to focus the user's attention. “Surround video” is not displayed. The entirety of 180 / 360 video consists of “flat video” + “surround video”.

[0005] One example is the provision of 180° / 360° surround feeds of live news events in addition to the regular flat feed. Live news events such as accidents, tornadoes, demonstrations, and political rallies can be displayed on the goggles as 360° feeds for 180° / 360° spherical representation. In addition to live news events, live "human interest" stories introducing people and organizations can also be displayed. 360° feeds can provide viewers with better ideas about the feed because they allow them to "look around" and see things that cannot be shown in a flat feed. While people without goggles are watching regular flat video on a TV, people wearing goggles can pan the entire scene by panning the 180° / 360° surround feed. On TVs that support 180° / 360° video display, panning can be performed by pressing the up, down, left, and right buttons on the remote control. Video can be monoscopic or stereoscopic. Stereoscopic video can convey a different image to each eye and is therefore called "3D" video. 3D video can be supported by 180° / 360° video, but not all 180° / 360° videos are 3D. This principle also applies to travel content, as well as any show or content filmed in 180° / 360° VR, such as concerts, motorcycle races, snowboarding, or hiking. Many VR videos are filmed monoscopically, for example, to convey the same content to each eye, and often with the focus set to infinity, thereby minimizing motion sickness (nausea) experienced by users in 3D. Monoscopic video can provide a very satisfying "being there" experience and telepresence.

[0006] Accordingly, the digital television device includes at least one receiver including at least one processor programmed with instructions, the processor configuring the instructions to display a flat video of the event on the main display and a 180-degree or 360-degree (180° or 360°) video of the event on a virtual reality (VR) headset.

[0007] In another embodiment, in a digital television capable of receiving a broadcast signal in at least one receiver, the method includes receiving at least a first broadcast video signal. The method includes displaying content in the first broadcast video signal on at least one conventional flat display, and presenting 180° / 360° video of the content on at least one display. The method includes panning the 180° / 360° video on at least one display in response to at least one command.

[0008] In some embodiments, at least one display includes a virtual reality (VR) headset.

[0009] In some embodiments, at least one display includes a flat display.

[0010] In some implementations, 180° / 360° video is received in a first broadcast video stream, and the first broadcast stream does not contain any other video content, and panning involves panning the video on a flat display.

[0011] In another embodiment, the 180° / 360° video is received in a first broadcast video stream, the first broadcast stream does not contain any other video content, and the 180° / 360° video is presented on a flat display as a flat video as a backward-compatible content view.

[0012] In some implementations, 180° / 360° video is received in a first broadcast video stream, the content within the first broadcast video stream is presented on a flat display, and the method includes transmitting the content from the flat display to a VR headset. In other implementations, 180° / 360° video is received in a first broadcast video stream, the content within the first broadcast video stream is presented on a flat display, and the method includes transmitting the content to a VR headset without going through the flat display.

[0013] 180° / 360° video can be received in a first broadcast video stream along with a flat feed of the content within the 180° / 360° video. Alternatively, 180° / 360° video can also be received from an over-the-top (OTT) source.

[0014] In another embodiment, the apparatus includes at least one receiver assembly configured to receive at least a first broadcast video signal and to present the content within the first broadcast video signal on at least one flat display. The receiver assembly is also configured to present 180° / 360° video of the content on at least one display. The assembly is configured to pan the 180° / 360° video on at least one display in response to at least one command.

[0015] Details of this application, both in terms of its structure and operation, can be best understood by referring to the attached drawings, which indicate similar elements with similar reference numerals. [Brief explanation of the drawing]

[0016] [Figure 1] This is a diagram of the Advanced Television System Commission (ATSC) 3.0 system. [Figure 2] This figure shows the components of the device shown in Figure 1. [Figure 3] This diagram shows the first technology for providing telepresence video. [Figure 4] This diagram shows the second technology for providing telepresence video. [Figure 5] The figure below shows a specific system example based on this principle. [Figure 6] This diagram shows an example of logic based on this principle in flowchart format. [Figure 7] This figure shows a receiver system with two displays: one for presenting a flat view of the event, and another for providing a 360-degree VR telepresence view of the event. [Figure 8] This figure shows the first technology for providing 360° telepresence video as pannable flat video on a TV. [Figure 9] This diagram shows a second technology that delivers 360° telepresence video as flat, backward-compatible video on a TV. [Figure 10] This figure shows a third technology that provides 360° telepresence video on TVs and VR goggles. [Figure 11] This figure shows a fourth technology that delivers 360° telepresence video directly onto VR goggles. [Modes for carrying out the invention]

[0017] This disclosure relates to the technological advancements of digital televisions such as Advanced Television Systems Committee (ATSC) 3.0 televisions. System examples herein can include ATSC 3.0 source components and client components connected via broadcast and / or network so as to be able to exchange data with each other. The client components can include one or more computer devices such as portable televisions (e.g., smart TVs, Internet-enabled TVs), portable computers such as laptop and tablet computers, and other mobile devices such as smartphones and further examples described below. These client devices can operate in various operating environments. For example, some of the client computers can adopt an operating system such as the operating system of Microsoft Corporation as an example, or a Unix operating system, or an operating system such as Android (registered trademark) manufactured by Apple Computer or Google. These operating environments can be used to execute one or more browsing programs such as browsers created by Microsoft, Google or Mozilla, or other browsing programs that can access websites hosted by Internet servers described below.

[0018] The ATSC 3.0 source components can include a broadcast transmission component and a server and / or gateway that can include one or more processors that execute instructions to configure the source components to perform data broadcasting and / or data transmission via a network such as the Internet. Examples of the client components and / or local ATSC 3.0 source components can include game consoles such as Sony PlayStation (registered trademark), personal computers, etc.

[0019] Between the client and the server, information can be exchanged via a network. For this purpose and for security, the server and / or the client can include a firewall, a load balancer, a temporary storage, and a proxy, as well as other network infrastructure to enhance authenticity and security.

[0020] As used herein, an instruction means a computer-implemented step for processing information within a system. The instruction can be implemented in software, firmware, or hardware, and can include any type of program step performed by a component of the system.

[0021] The processor can be a single-chip or multi-chip processor that can execute logic by means of various lines such as address lines, data lines, and control lines, as well as registers and shift registers.

[0022] The software modules described by the flowchart and the user interface in this specification can include various subroutines, procedures, etc. Without limiting the present disclosure, the logic disclosed as being executed by a particular module can also be redistributed to other software modules, and / or combined into a single module, and / or utilized within a shareable library. Although the flowchart form can be used, it should be understood that the software can also be implemented as a state machine or other logical method.

[0023] The principles described herein can be implemented as hardware, software, firmware, or a combination thereof, and thus exemplary components, blocks, modules, circuits, and steps will be described from these functional aspects.

[0024] In addition to those suggested above, logic blocks, modules, and circuits may be implemented or run using any combination of general-purpose processors, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), or other programmable logic devices such as application-specific integrated circuits (ASICs), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. Processors may be implemented by a combination of controllers, state machines, or computer devices.

[0025] The functions and methods described below, when implemented in software, can be written in a suitable language such as Hypertext Markup Language (HTML)-5, Java® / Javascript, C#, C++, etc., and can be stored in or transmitted through computer-readable storage media such as Random Access Memory (RAM), Read-Only Memory (ROM), Electrically Erasable and Programmable Read-Only Memory (EEPROM), Compact Disk Read-Only Memory (CD-ROM), or other optical disk storage such as Digital Utility Disks (DVDs), magnetic disk storage, or other magnetic storage devices including removable Universal Serial Bus (USB) thumb drives. A certain connection can constitute computer-readable media. Such connections may include, as an example, wired cables including optical fibers, coaxial cables, digital subscriber lines (DSL), and twisted pair cables.

[0026] Components included in one embodiment can be used in any suitable combination in other embodiments. For example, any of the various components described and / or shown in the figures herein can be combined, replaced, or excluded from other embodiments.

[0027] The phrase "having at least one of A, B, and C (similarly, "having at least one of A, B, or C" and "having at least one of A, B, and C")" includes A only, B only, C only, both A and B, both A and C, both B and C, and / or all of A, B, and C.

[0028] This principle can employ a variety of machine learning models, including deep learning models. Machine learning models based on this principle can utilize various algorithms trained using methods including supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, feature learning, self-learning, and other forms of learning. Examples of such algorithms that can be implemented by computer circuits include one or more neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and RNNs of the type known as long short-term memory (LSTM) networks. Support vector machines (SVMs) and Bayesian networks can also be considered examples of machine learning models.

[0029] Therefore, as understood herein, performing machine learning can involve training a model on training data after accessing training data so that the model can process further data and perform inference. Thus, an artificial neural network / artificial intelligence model trained through machine learning may include an input layer, an output layer, and multiple hidden layers between these, configured and weighted to perform inference about appropriate outputs.

[0030] Referring to Figure 1, an example of an ATSC3.0 source component, denoted as “Broadcasting Station Equipment” 10, may include an over-the-air (OTA) setup 12 that wirelessly broadcasts television data to multiple receivers 14, such as ATSC3.0 televisions, via orthogonal frequency division multiplexing (OFDM) in a one-to-many relationship. One or more receivers 14 may communicate with one or more companion devices 16, such as remote control devices, tablet computers, mobile phones, computer game headsets like Sony PlayStation® headsets, and computer simulation 180° / 360° headsets, via a short-range link 18, which is typically wireless and can be implemented by Bluetooth®, Low Energy Bluetooth, Wireless HDMI, Amimon Wireless HD Video Link, other Near Field Communication (NFC) protocols, infrared (IR), etc.

[0031] Furthermore, one or more of the receivers 14 can also communicate with the over-the-top (OTT) equipment 22 of the broadcasting station equipment 10, typically in a one-to-one relationship, via a wired and / or wireless network link 20 such as the Internet. The OTA equipment 12 can be located in the same place as the OTT equipment 22, or the two pieces of equipment 12 and 22 of the broadcasting station equipment 10 can communicate with each other separately through appropriate means. In any case, the receiver 14 can receive ATSC 3.0 television signals via OTA through a tuned ATSC 3.0 television channel, or receive related content, including television, via OTT (broadband). Note that the computer equipment described in all the figures of this specification may include some or all of the components shown for the various devices in Figures 1 and 2.

[0032] Next, referring to Figure 2, we can see details of the component example shown in Figure 1. Figure 2 shows an example of a protocol stack that can be implemented by a combination of hardware and software. A broadcaster can use the ATSC3.0 protocol stack shown in Figure 2, appropriately modified for the broadcaster side, to transmit a hybrid service distribution that delivers one or more program elements over computer networks (referred to herein as “broadband” and “over-the-top” (OTT)) and wireless broadcasts (referred to herein as “broadcast” and “over-the-air” (OTA)). Figure 2 also shows an exemplary stack including hardware that can be embodied by a receiver.

[0033] Disclosing Figure 2 from the perspective of the broadcasting station equipment 10, one or more processors 200 accessing one or more computer storage media 202, such as any of the memory or storage described herein, can be implemented to provide one or more software applications in the top-level application layer 204. The application layer 204 may include one or more software applications written, for example, in HTML5 / Javascript, that operate in the runtime environment. Applications in the application stack 204 may include, but are not limited to, linear TV applications, interactive service applications, companion screen applications, personalization applications, emergency alert applications, and usage reporting applications. Typically, applications are embodied in software that represents the elements experienced by the viewer, including video coding, audio coding, and the runtime environment. As an example, an application may be provided that allows the user to control dialogue, use alternative audio tracks, and control audio parameters such as normalization and dynamic range.

[0034] Below the application layer 204 lies the presentation layer 206. The presentation layer 206 includes a broadcast audio-video playback device called a media processing unit (MPU) 208 on the broadcast (OTA) side, which decodes the wirelessly broadcast audio-video content when implemented in a receiver and plays it on one or more displays and speakers. The MPU 208 is configured to present video in the International Organization for Standardization (ISO) Based Media File Format (BMFF) data representation 210 and High Efficiency Video Coding (HEVC) format, with audio in, for example, Dolby Audio Compression (AC)-4 format. ISO BMFF is a common file structure for time-based media files, divided into "segments" and presentation metadata. Essentially, each file is a group of nested objects, each with its own type and length. The MPU 208 can access the broadcast-side encrypted media extension (EME) / common encryption (CENC) module 212 to facilitate decryption.

[0035] Figure 2 further illustrates that, on the broadcast side, the presentation layer 206 may include a signaling module that includes either a Video Professional Group (MPEG) Media Transfer Protocol (MMTP) signaling module 214 or a real-time object delivery over unidirectional transport (ROUTE) signaling module 216 for delivering non-real-time (NRT) content 218 accessible to the application layer 204. The NRT content may, but is not limited to, include stored alternative advertisements.

[0036] On the broadband (OTT or computer network) side, if implemented by the receiver, the presentation layer 206 may include one or more Dynamic Adaptive Streaming (DASH) players / decoders 220 via Hypertext Transfer Protocol (HTTP) to decode and play audio-video content from the Internet. For this purpose, the DASH player 220 can access the EME / CENC module 222 on the broadband side. The DASH content can be provided as a DASH segment 224 in ISO / BMFF format. To support various embodiments of the present invention, the DASH segment encodes various media content component types such as audio, video, or text. It is anticipated that there may be multiple video types, such as one video type including legacy main view and a new video type including surround video. The legacy player / decoder decodes the legacy main view-based legacy video type, and the enhanced player pans and decodes the additional video type including surround video.

[0037] The broadband side of the presentation layer 206, like the broadcast side, can include NRT content within file 226 and a signaling object 228 that provides playback signaling.

[0038] Below the presentation layer 206 in the protocol stack lies the session layer 230. The session layer 230 includes either the MMTP protocol 232 or the ROUTE protocol 234 on the broadcast side. Note that the ATSC standard provides an option to use MPEG MMT for transmission, but this is not shown here.

[0039] Session Layer 230 includes the HTTP protocol 236, which can be implemented on the broadband side as HTTP-secure (HTTP(S)). The broadband side of Session Layer 230 may also employ an HTTP proxy module 238 and a Service List Table (SLT) 240. The SLT 240 includes a table of signaling information used to construct a basic service list and provide bootstrap discovery of broadcast content. The "ROUTE signaling" table includes media presentation descriptions (MPDs) delivered via the User Datagram Protocol (UDP) by the ROUTE transport protocol.

[0040] Below the session layer 230 in the protocol stack is the transport layer 242, which establishes low-latency and loss-tolerating connections. The transport layer 242 uses UDP 244 on the broadcast side and Transmission Control Protocol (TCP) 246 on the broadband side.

[0041] The non-restrictive protocol stack example shown in Figure 2 also includes a network layer 248 below the transport layer 242. The network layer 248 uses the Internet Protocol (IP) on both sides for IP packet communication, with multicast distribution typical on the broadcast side and unicast typical on the broadband side.

[0042] Below the network layer 248 lies the physical layer 250, which includes broadcast transmit / receive equipment 252 and (one or multiple) computer network interfaces 254 for communication on their respective physical media. The physical layer 250 may include modulation and demodulation modules to incorporate modulation and demodulation functions, as well as adding forward error correction capabilities to enable error correction at the receiver, and converting Internet Protocol (IP) packets to be suitable for transmission on the relevant medium. The physical layer 250 converts bits to symbols for long-distance transmission and improved bandwidth efficiency. The physical layer 250 typically includes a radio broadcast transmitter on the OTA side that broadcasts data wirelessly using orthogonal frequency division multiplexing (OFDM), and a computer transmit component on the OTT side that transmits data over the internet.

[0043] On the broadband side, DASH Industry Forum (DASH-IF) profiles can be used, transmitted through various protocols (HTTP / TCP / IP) within the protocol stack. Media files within the ISO BMFF-based DASH-IF profile can be used as a distribution, media encapsulation, and synchronization format for both broadcast and broadband distribution.

[0044] Typically, each receiver 14 includes a protocol stack complementary to the broadcasting station's protocol stack.

[0045] The receiver 14 in Figure 1 may include an internet-enabled TV having an ATSC 3.0 TV tuner 256 (equivalent to a set-top box that controls the TV), as shown in Figure 2. The receiver 14 may be an Android®-based system. Alternatively, the receiver 14 may be implemented by a computerized internet-enabled ("smart") phone, tablet computer, notebook computer, and wearable computer device, etc. Nevertheless, it should be understood that the receiver 14 and / or other computers described herein are configured to implement the principle (e.g., to communicate with other devices to implement the principle, to execute the logic described herein, and to perform any other functions and / or operations described herein).

[0046] Accordingly, the receiver 14 can be established by some or all of the components shown in Figure 1 to implement such principles. For example, the receiver 14 may include one or more displays 258 that are implemented by high-definition or ultra-high-definition "4K" or higher flat screens and may or may not be touch-enabled, receiving user input signals via touch on the displays. The receiver 14 may also include one or more speakers 260 for outputting audio according to this principle, and at least one further input device 262, such as an audio receiver / microphone, for inputting audible commands to control the receiver 14. Further examples of the receiver 14 include one or more network interfaces 264 for communicating over at least one network such as the Internet, WAN, LAN, or PAN under the control of one or more processors 266. Thus, the interface 264 may be a Wi-Fi transceiver, which is an example of a wireless computer network interface such as a mesh network transceiver, but is not limited to this. Interface 264 may be, but is not limited to, a Bluetooth® transceiver, a Zigbee® transceiver, an Infrared Communications Association (IrDA) transceiver, a wireless USB transceiver, a wired USB, a wired LAN, a power line, or a Multimedia over Coax Alliance (MoCA). It should be understood that the processor 266 controls the receiver 14 to implement this principle, including other elements of the receiver 14 described herein, such as controlling the display 258 to display images and receive input. Furthermore, the network interface 264 may be, for example, a wired or wireless modem or router, or other suitable interfaces such as a wireless telephone transceiver or the Wi-Fi transceiver described above.

[0047] In addition to the above, the receiver 14 may also include one or more input ports 268, such as a high-definition multimedia interface (HDMI) port or a USB port, for physically connecting to another CE device (using a wired connection), and / or a headphone port for connecting headphones to the receiver 14 and presenting audio to the user through the headphones. For example, the input ports 268 may be connected via wired or wireless to a cable or satellite source of audio video content. Thus, the source may be a separate or integrated set-top box or satellite receiver. Alternatively, the source may be a game console or a disc player.

[0048] The receiver 14 may further include one or more computer memories 270, such as non-transient disk-based storage or solid-state storage, which are embodied in some cases as a standalone device within the receiver chassis, as a personal video recorder (PVR) or video disc player for playing audio-video (AV) programs, or as a removable storage medium, either inside or outside the receiver chassis. In some embodiments, the receiver 14 may also include, but are not limited to, a cell phone receiver, a Global Positioning Satellite (GPS) receiver, and / or an altimeter, a position or location receiver 272 configured to receive geographical location information from, for example, at least one satellite or cell phone tower and provide this information to the processor 266, and / or to determine the altitude at which the receiver 14 is positioned together with the processor 266. However, it should be understood that, in accordance with this principle, another suitable location receiver other than a cell phone receiver, a GPS receiver, and / or an altimeter may also be used to determine the position of the receiver 14 in all three dimensions, for example.

[0049] Continuing the description of the receiver 14, in some embodiments, the receiver 14 may include one or more cameras 274, which may include one or more cameras such as thermal cameras, webcams and other digital cameras, and / or cameras integrated into the receiver 14 and controllable by the processor 266, for collecting photographs / images and / or videos in accordance with the present principle. The receiver 14 may also include a Bluetooth® transceiver 276 or other near-field communication (NFC) element for communicating with other devices using Bluetooth® and / or NFC technology. An example of an NFC element may be a radio frequency identification (RFID) element.

[0050] Furthermore, the receiver 14 may also include one or more auxiliary sensors 278 (such as motion sensors including accelerometers, gyroscopes, cyclometers, or magnetic sensors or inertial measuring units (IMUs) and combinations thereof) that provide input to the processor 266, an infrared (IR) sensor for receiving IR commands from a remote control device, an optical sensor, a velocity and / or cadence sensor, a gesture sensor (for detecting gesture commands), etc. An IR sensor 280 may also be provided for receiving commands from a wireless remote control. A battery (not shown) may also be provided to supply power to the receiver 14.

[0051] The companion device 16 may include some or all of the elements described above in relation to the receiver 14.

[0052] The methods described herein can be implemented as software instructions executed by a processor, a suitably configured application-specific integrated circuit (ASIC) or field-programmable gate array (FPGA) module, or any other convenient method that a person skilled in the art would understand. The software instructions may be embodied in non-temporary devices such as CD-ROMs or flash drives, if adopted. Alternatively, the software code instructions may be embodied in a temporary configuration such as a wireless signal or an optical signal, or through download over the Internet.

[0053] Next, refer to Figure 3. An event camera 302 can capture a live event 300 for broadcast, transmitting actual, normal flat images to one or more receivers 306 via an OTA broadcast transmitter 304. Simultaneously, a movable 180° / 360° (180° / 360°) 4K or 8K or higher camera 308 captures the event 300 in all directions and transmits its output to the receivers 306 via an OTT source 310 such as an internet server. The (180° / 360°) 4K or 8K or higher camera 308 can provide a surround view of the event and can also provide a greater elevation view of the event than that provided by the flat camera 302. As an example, the (180° / 360°) 4K or 8K or higher camera 308 can be mounted on a drone to capture the event 300. The (180° / 360°) 4K or 8K or higher camera 308 can output 180° / 360° video as needed. For example, the (180° / 360°) 4K or 8K or higher video can be accompanied by AC-4 audio. The (180° / 360°) 4K or 8K or higher camera 308 can capture the event 300 in large angle format.

[0054] On the other hand, Figure 4 shows an embodiment in which the output of flat camera 302 and the output of (180° / 360°) 4K or 8K or higher camera 308 are both transmitted to receiver 306 via OTA broadcast 304. The two video streams generated by cameras 302 and 308 can be transmitted in a single physical layer pipe (PLP) or associated PLP, and both video streams can be identified as to what each is for, for example, flat or (180° / 360°) 4K or 8K, with signaling indicating that both video streams are available.

[0055] In another embodiment, the flat camera 302 is omitted from Figure 4, and only the video stream from the (180° / 360°) 4K or 8K or higher camera 308 can be transmitted to the receiver via OTA for processing as described later. In this embodiment, the stream may have a main view using the main video content type.

[0056] Figure 5 shows that a receiver assembly may include an OTA receiver 500, an OTT receiver 502, and a flat display 504 such as a TV. The receiver assembly may further include a virtual reality (VR) headset 506 such as goggles and one or more devices for panning video, such as 180° / 360° video, presented on the VR headset 506. These devices may include point-and-click mechanisms such as a joystick or buttons on a TV remote control device (RC) 508, a touch screen on a smartphone 510, or a touch screen or touchpad 512 on the headset 506. The panning device may also include one or more motion sensors 514 in the headset 506 or RC 508. The headset may also have an RC 508 instead of a touch screen or touchpad 512 to perform panning. A combination of the above may also be used for panning. Each component shown in Figure 5 may include the appropriate components of Figure 2, including a processor, storage device, and network interface.

[0057] Figure 6 shows an example of logic that can be executed by one or more processors in the receiver assembly of Figure 5. Starting from block 600, if the flat event camera 302 is not present, a flat video stream is received via OTA or similar means, or derived from a single (180° / 360°) 4K or 8K or higher video stream. The (180° / 360°) 4K or 8K or higher video stream may have a main view that is essentially a flat view stream. In block 602, this flat stream can be presented on TV 504 in Figure 5, for example.

[0058] The process proceeds to block 604, where a (180° / 360°) 4K or 8K or higher video stream is received via OTA or OTT as described above, and a pan signal is received in block 606. In block 308, the (180° / 360°) 4K or 8K or higher 180° / 360° video can be presented on the VR headset 506 shown in Figure 5.

[0059] Figure 7 shows that by presenting a flat stream of event 300 from Figure 3 on TV 504, and simultaneously presenting a (180° / 360°) 4K or 8K or higher 180° / 360° version of the same event 300 on headset 506, a panning function is enabled, indicated by arrows 700, which allows the wearer of headset 506 to virtually be present in event 300 and "look around" the event as if actually being there by moving their head or operating another panning device such as RC508. Note that panning includes both azimuth panning and elevation panning.

[0060] Figure 8 shows a first technique in which TV 504 receives (180° / 360°) 4K or 8K or higher video 800 via OTA and presents it as flat video on the TV. Viewers can pan the video on TV 504 using any of the aforementioned input devices, such as a joystick or buttons on RC 508 or a touch screen on a smartphone 510.

[0061] Figure 9 shows that TV504 receives (180° / 360°) 4K or 8K or higher video 800 via OTA and renders it as flat on the TV to establish a backward-compatible view without panning.

[0062] Figure 10 shows that TV 504 receives (180° / 360°) 4K or 8K or higher video 800 via OTA, renders it flat for presentation on the TV, and relays the (180° / 360°) 4K or 8K or higher video 800 from the TV to the headset 506 via Bluetooth or Wi-Fi, allowing it to be presented in 180° / 360° on the headset, which can pan the described view.

[0063] Figure 11 shows that the headset 506 can receive (180° / 360°) 4K or 8K or higher video 800 directly via OTA or OTT and present 180° / 360° on the headset, which can pan the described view, without the need for TV 504 to intervene to relay the 180° / 360° video.

[0064] While this principle has been described with reference to several embodiments, these embodiments are not intended to be limiting, and it will be understood that the subject matter claimed herein can also be implemented using a variety of other configurations. [Explanation of symbols]

[0065] Received 600 2D streams. Displayed on TV 602 Received 604 360° stream 606 Received a pan signal. 608 Pan presented on a 3D VR display

Claims

1. In a digital television in which at least one receiver can receive a broadcast signal, Receiving at least the first broadcast video signal, Displaying the content within the first broadcast video signal on at least one flat display, The 180° / 360° video of the aforementioned content is presented on at least one display, A method comprising, wherein the content presented on the at least one flat display is acquired from at least one flat video camera, the 180° / 360° video is acquired from at least one 180° / 360° camera, and the flat video content and the 180° / 360° video are transmitted to the receiver via over-the-air (OTA) through at least one physical layer pipe (PLP) with signaling that both the flat video content and the 180° / 360° video are available.

2. The at least one display includes a virtual reality (VR) headset. The method according to claim 1.

3. The 180° / 360° video is received in a first broadcast video stream, and the first broadcast video stream does not contain any other video of the content. The method according to claim 1.

4. The 180° / 360° video is received in a first broadcast video stream, the first broadcast video stream does not contain any other video of the content, and the 180° / 360° video is presented on the flat display as a flat video as a backward-compatible content view. The method according to claim 1.

5. The 180° / 360° video is received in a first broadcast video stream, the content within the first broadcast video stream is presented on the flat display, and the method includes transmitting the content from the flat display to the VR headset. The method according to claim 2.

6. The 180° / 360° video is received in a first broadcast video stream, the content in the first broadcast video stream is presented on the flat display, and the method includes transmitting the content to the VR headset without going through the flat display. The method according to claim 2.

7. The 180° / 360° video is received in a first broadcast video stream along with a flat feed of the content within the 180° / 360° video. The method according to claim 1.

8. The aforementioned digital television includes an Advanced Television Systems Commission (ATSC) 3.0 receiver, The method according to claim 1.

9. Having received at least the first broadcast video signal, The content within the first broadcast video signal is presented on at least one flat display, The system comprises at least one receiver assembly configured to display 180° / 360° video of the content on at least one display, The apparatus is characterized in that the content is acquired from at least one flat video camera, the 180° / 360° video is acquired from at least one 180° / 360° camera, and the flat video content and the 180° / 360° video are transmitted to the receiver assembly via over-the-air (OTA) through at least one physical layer pipe (PLP), with signaling that both the flat video content and the 180° / 360° video are available, and they identify each other.

10. The at least one display includes a virtual reality (VR) headset. The apparatus according to claim 9.

11. The 180° / 360° video is received in a first broadcast video stream, and the first broadcast video stream does not contain any other video of the content. The apparatus according to claim 9.

12. The 180° / 360° video is received in a first broadcast video stream, the first broadcast video stream does not contain any other video of the content, and the 180° / 360° video is presented on the flat display as a flat video as a backward-compatible content view. The apparatus according to claim 9.

13. The 180° / 360° video is received in a first broadcast video stream, the content within the first broadcast video stream is presented on the flat display, and a command is executable to transmit the content from the flat display to the VR headset. The apparatus according to claim 10.

14. The 180° / 360° video is received in a first broadcast video stream, the content within the first broadcast video stream is presented on the flat display, and a command is executable to transmit the content to the VR headset without going through the flat display. The apparatus according to claim 10.

15. The 180° / 360° video is received in a first broadcast video stream along with a flat feed of the content within the 180° / 360° video. The apparatus according to claim 9.

16. A digital television device, It comprises at least one receiver including at least one processor programmed with instructions, and said instructions are The event's flat video is displayed on the main display. Display 180-degree or 360-degree (180° or 360°) video of the aforementioned event on a virtual reality (VR) headset. The processor is configured as follows: A digital television apparatus characterized in that the flat video is acquired from at least one flat video camera, the 180° or 360° (180° or 360°) video is acquired from at least one 180° / 360° camera, and the flat video content and the 180° or 360° video are transmitted to the receiver via over-the-air (OTA) through at least one physical layer pipe (PLP) with signaling that both the flat video content and the 180° / 360° video are available.

Images