Session Description Protocol (SDP) Signaling of Occlusion-Free Regions in 360-Degree Videoconferencing

Abstract

The method includes receiving a first video input corresponding to a 360-degree video, receiving one or more second video inputs, defining one or more occlusion free regions within the first video input, each occlusion free region indicating an area of ​​the video that is not overlaid by another image or video, signaling location information of the one or more occlusion free regions in a Session Description Protocol (SDP) via a Third Generation Partnership Project (3GPP) overlay attribute, the 3GPP overlay attribute signaling whether a respective 3GPP overlay is intended for an occlusion free region or is dedicated to a particular overlay, sending the SDP to a receiver, and rendering an output video that includes the first video input with the one or more second video inputs overlaid in areas that do not include the one or more occlusion free regions.

Description

[Technical field]

[0001] This disclosure relates to 360 Degree A method for signaling occlude-free regions in video conferencing is provided. Degree Occlusion-free regions in the 360 ​​video contain important information and should not be covered by the overlay. Degree This is the video area.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Patent Application No. 63 / 307,550, filed February 7, 2022, and U.S. Patent Application No. 18 / 073,207, filed December 1, 2022, the disclosures of which are incorporated by reference in their entireties herein. [Background technology]

[0003] The 3rd Generation Partnership Project (3GPP) TS 26.114 defines videoconferencing systems for mobile handsets. In its new release, the specification supports 360 Degree Supports video conferencing with terminals that support video capture and transmission. This standard also supports 360 Degree It also supports adding overlays to the video. Degree The video and its overlays can be rendered together with other two-dimensional (2-d) videos from other remote participants in the conference call.

[0004] The current 5G media streaming architecture defined in 3GPP TS26.114 provides a general framework for video conferencing over mobile networks. A remote participant is 360° away from Room A. DegreeUser B receiving 2-d video from user C and so on. The user would like to see both videos on his / her device. However, if the user wants to make the most of the device display, the 360° view of room A may be Degree The video may need to capture the entire screen of User B's device, and then the 2D video from User C may capture the 360° view of Room A. Degree It needs to be overlaid on top of the video.

[0005] The current standard does not define any method to signal the occlusion-free regions of room A. These regions are defined as 360° views of room A. Degree An area of ​​video that contains important information (participants in the room, or a presentation view) and that should not be blocked by overlaying video from other users (such as User C's 2D video) at the receiving remote terminal. Summary of the Invention

[0006] The following presents a simplified summary of one or more embodiments of the present disclosure in order to provide a basic understanding of such embodiments. This summary is not an extensive overview of all contemplated embodiments, and is not intended to identify key or critical elements of all embodiments or to delineate the scope of any or all embodiments. Its sole purpose is to present some concepts of one or more embodiments of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0007] The present disclosure provides a method for signaling an occlusion-free region.

[0008] According to some embodiments, a method is provided that is executed by at least one processor. The method includes receiving a first video input corresponding to a 360-degree video. The method further includes receiving one or more second video inputs. The method further includes defining one or more occlusion free regions in the first video input, each occlusion free region indicating an area of ​​the video that is not overlaid by another image or video. The method further includes signaling location information of the one or more occlusion free regions in a Session Description Protocol (SDP) via a 3GPP overlay attribute, where the 3GPP overlay attribute indicates a region of the video that is not overlaid by another image or video. Gao Occlude-free area Is it targeted to , which are specific to a particular overlay Or The method further includes transmitting the SDP to a receiver and rendering an output video that includes the first video input with one or more second video inputs overlaid in areas that do not include the one or more occlusion free regions.

[0009] According to some embodiments, the apparatus includes at least one memory configured to store program code and at least one processor configured to read and operate as instructed by the program code. The program code includes a first receiving code configured to cause the at least one processor to receive a first video input corresponding to a 360-degree video. The program code further includes a second receiving code configured to cause the at least one processor to receive one or more second video inputs. The program code further includes a defining code configured to cause the at least one processor to define one or more occlusion-free regions in the first video input, where each occlusion-free region represents an area of ​​the video that is not overlaid by another image or video. The program code further includes a signaling code configured to cause the at least one processor to signal location information of the one or more occlusion-free regions in a Session Description Protocol (SDP) via a 3GPP overlay attribute, where the 3GPP overlay attribute indicates that each 3GPP overlay is associated with an occlusion-free region. Is it targeted to , which are specific to a particular overlay Or The program code further includes a transmitting code configured to cause the at least one processor to transmit the SDP to a receiver. The program code further includes a rendering code configured to cause rendering an output video including the first video input with one or more second video inputs overlaid in areas that do not include one or more occlusion free regions.

[0010] According to some embodiments, a non-transitory computer-readable storage medium has stored thereon instructions that, when executed by at least one processor, cause the at least one processor to receive a first video input corresponding to a 360-degree video. The instructions further cause the at least one processor to receive one or more second video inputs. The instructions further cause the at least one processor to define one or more occlusion-free regions in the first video input, each occlusion-free region indicating an area of ​​the video that is not overlaid by another image or video. The instructions further cause the at least one processor to signal location information of the one or more occlusion-free regions in a Session Description Protocol (SDP) via a 3GPP overlay attribute, where the 3GPP overlay attribute indicates a region of the video that is not overlaid by another image or video. Gao Inclusion Free Area Is it targeted to , which are specific to a particular overlay Or The instructions further cause the at least one processor to transmit the SDP to a receiver. The instructions further cause the at least one processor to render an output video that includes the first video input with one or more second video inputs overlaid in areas that do not include one or more occlusion free regions.

[0011] Additional embodiments will be set forth in the description that follows, and in part will be obvious from the description, and / or may be learned by practice of presented embodiments of the present disclosure. [Brief description of the drawings]

[0012] These and other features and aspects of embodiments of the present disclosure will become apparent from the following description taken in conjunction with the accompanying drawings. [Figure 1] FIG. 1 is a diagram of an example network device in accordance with various embodiments of the present disclosure. [Diagram 2] FIG. 2 is a flow chart of one exemplary process for defining an occlusion-free region, according to various embodiments of the present disclosure. [Diagram 3] FIG. 3 is an illustration of occlusion-free and occluded regions, according to various embodiments of the present disclosure. [Figure 4] FIG. 4 is a diagram of a 360-degree video system according to various embodiments of the present disclosure. [Diagram 5] FIG. 5 is a diagram of a 360-degree video system according to various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] The following detailed description of the illustrative embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings may identify the same or similar elements.

[0014] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit the implementation to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be obtained from the actual lines of implementation. Furthermore, one or more features or components of one embodiment may be incorporated or combined with another embodiment (or one or more features of another embodiment). In addition, in the flowcharts and operational descriptions provided below, it will be understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed simultaneously (at least partially), and the order of one or more operations may be rearranged.

[0015] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not intended to limit the implementation. Thus, the operations and behavior of the systems and / or methods have been described herein without reference to specific software code. It will be understood that software and hardware may be designed to implement the systems and / or methods based on the description herein.

[0016] Although particular combinations of features are recited in the claims and / or disclosed herein, these combinations are not intended to limit the disclosure of possible implementations. Indeed, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim set.

[0017] No element, act, or instruction used herein should be construed as critical or essential unless expressly described as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and may be used interchangeably with "one or more." Where only one item is intended, "one" or similar language is used. Also, as used herein, the terms "has," "have," "having," "include," "including," and the like are intended to be open-ended terms. Furthermore, the phrase "based on" is intended to mean "based, at least in part, on," unless expressly specified otherwise. Furthermore, phrases such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood to include only A, only B, or both A and B.

[0018] 360 Degree When using traditional methods for video conferencing, current systems provide 360 Degree It relies on transcribing the video into a two-dimensional space and then overlaying the two-dimensional drawing with other relevant information. For example, when a user is in a 360° conference with some kind of presentation, current systems will automatically animate the entire conference (360° Degree They must choose to display either the meeting video (video), the presentation (2D video), or some sort of overlapped rendering of the meeting video with the presentation drawn across parts of the meeting. With the increasing occurrence of remote work, the need for collaboration between peers requires better ways to see both the participants and the subject matter of virtual meetings.

[0019] This disclosure provides a method for detecting 360 pixels tagged as occlusion-free regions. DegreeDescribe regions of interest in a video. The location and area of ​​each region is forwarded to the receivers so that each receiver knows that the regions are supposed to remain occlusion-free during rendering with the video and images from other terminals. For example, the faces of each participant in a conference room are visible in the 360-degree view of the conference room. Degree It may be one area of ​​interest in the video.

[0020] These regions of interest are dynamic and may change during a videoconferencing session: for example, a new participant may join the conference room and / or someone else may leave the conference room, and thus the number and / or size of the occlusion-free regions may change during a videoconferencing session.

[0021] In some embodiments, 360 Degree The global coordinate system of the video is used as the reference for any occlusion-free region. An occlusion-free region is defined by its position in that coordinate system and its area in that coordinate system. For example, on the spherical surface The rectangular region of is defined by its center and the yaw and pitch ranges around the center. Similarly, in various coordinate systems, regions can be defined using the parameters of that coordinate system.

[0022] In some embodiments, 360 Degree The occlusion-free regions of the video can be signaled in the SDP via the 3gpp_overlay attribute. An additional flag can be added to indicate whether the determined 3GPP overlay is occlusion-free. Is it targeted to , which is exclusive to a specific overlay Or For example,

number

[0023] The for_additional_overlay flag has the following semantics: 1. If the for_additional_overlay flag is not included or is set to 0, then this overlay has dedicated media identified by the overlay_id value. 2. If for_additional_overlay is included and set to 1, then this overlay is an occlusion-free overlay and receivers can use it to overlay any media desired. child Area Gao As an inclusion-free area Be clear Marked attitude So, This overlay Important Information The loss of do not have.

[0024] The SDP can contain one or more of the above announcements, each of which represents one occlusion-free region. Furthermore, the SDP can be used to update, remove, or add new regions during a session by providing new SDP lines during the session.

[0025] By defining a clear map of occlusion-free regions using SDP, the receiving terminal knows not to render any overlays on those regions. The number, location, and size of each occlusion-free region can be updated during the session. This information can be extracted at the source and then distributed from the source or added in the MRF.

[0026] Figure 1 shows a 360 Degree1 shows an exemplary system 100 according to an embodiment for using the videoconferencing method. The exemplary system 100 can be one of a variety of systems, such as a personal computer, a mobile device, a cluster of computers, a server, an embedded device, an ASIC, a microcontroller, or any other device capable of executing code. A bus 110 connects the exemplary system 100 together so that all components can communicate with each other. The bus 110 connects the processor 120, the memory 130, the storage component 40, the input component 150, the output component 160, and the interface component.

[0027] The processor 120 may be a single processor, a processor with multiple processors therein, a cluster (two or more) of processors, and / or distributed processing. The processor executes instructions stored in both the memory 130 and the storage component 40. The processor 120 acts as a computing device and performs operations for the text normalization apparatus. The memory 130 is a high-speed storage, and retrieval to any of the memory devices may be made possible through the use of a cache memory, which may be closely associated with one or more CPUs. The storage component 40 may be any long-term storage, such as a HDD, SSD, magnetic tape, or any other long-term storage format.

[0028] Input component 150 can be any file type or signal from a user interface component, such as a camera or text capture device. Output component 160 outputs the processed information to a communication interface 170, which can be a speaker or other communication device that can display information to a user or another observer, such as another computing system.

[0029] FIG. 2 shows a flow chart of one example embodiment of a process for conducting a video conference.

[0030] As shown in FIG. 2, the process may include receiving a first video input corresponding to a 360-degree video conference.

[0031] As further shown in FIG. 2, the process may include receiving one or more secondary video inputs.

[0032] As further shown in Figure 2, the process can include defining one or more occlusion-free regions in the first video input. An occlusion-free region refers to an area of ​​the video that is not overlapped by any image or video.

[0033] As further shown in FIG. 2, the process may include signaling location information of the one or more occlusion-free regions in a session description protocol via a 3GPP overlay attribute.

[0034] As further shown in FIG. 2, the process may include transmitting the SDP to a receiver.

[0035] As further shown in FIG. 2, the process may include rendering an output video that includes a first video input with one or more second video inputs overlaid in areas that do not include one or more occlusion-free regions.

[0036] FIG. 3 illustrates an embodiment of an occlusion region 320 and an occlusion-free region 310. On a screen 300, space may be allocated to either an occlusion-free region 310 or an occluded region 320. An occlusion-free region 310 is a region of the screen 300 where content or other information is being drawn / rendered or otherwise output. This occlusion-free region 310 may prevent other content from being rendered over or otherwise overlapped. On the other hand, an occlusion region 320 is marked as a region that has no content, has low priority content, or where other content may be placed over it. An occlusion-free region is defined by its location in the coordinate system and its area in the coordinate system. For example, on the spherical surface The rectangular region of the screen 300 is defined by its center and yaw and pitch ranges around the center. Similarly, in various coordinate systems, regions may be defined using the parameters of the coordinate systems. In some embodiments, a region of the screen 300, the entire screen, or multiple screens may be marked as either an occlusion region or an occlusion-free region.

[0037] Various mechanisms may be used to signal an occlusion-free region. For example, in some embodiments, signaling an occlusion-free region may be to send the coordinates of such a region as an item in a list of occlusion-free regions as part of the session description. In other embodiments, signaling an occlusion-free region may be done by defining a node in the scene description that defines the occlusion-free region and its properties (e.g., transparent and does not contain media objects). In other embodiments, signaling an occlusion-free region may be performed by defining a separate scene description that defines only the occlusion-free region. DegreeThe occlusion-free regions of the video can be signaled in the SDP a=3gpp_occludefree attribute. The video component can have the location and size (extent) of the region. Since the component is defined by 3gpp_occludefree, ITT4RT knows that this signaling is used to signal regions that do not contain actual media but should not be covered.

[0038] In some embodiments, the scene description may include a node for each occlusion-free region. The node texture property may be set to an alpha channel with an opacity of 0 (fully transparent). Alternatively, a new MIME type may be defined for the occlusion-free node. For example, in a glTF scene description, for a texture, if alphaMode=MASK and alphaCutOff=1.1, the object is transparent (not rendered). A new attribute may be added to the glTF specification to explicitly signal these regions as occlusion-free regions.

[0039] Figure 4 shows the 360 ​​in use. Degree FIG. 1 illustrates in detail one embodiment of a video system. In an exemplary embodiment, the video system is 360°. Degree A video presentation is taking place. The video presentation can be, for example, a video conference, a video chat, a video or other information exchange in which visual and audio information is present. In this embodiment, User B 460 is connected to User A 400 through 360. Degree User A receives a 360° video from room A and a 2-d video from user C 450. The user would like to see both videos on his / her device. However, if the user wants to make the most of the device display, he / she may want to see the 360° video from room A. Degree The video may need to capture the entire screen of User B's device, and then a 2D video from User C may be captured of a 360° view of Room A 400. DegreeThe video feed 420 of User B 460 must be overlaid on top of the video. In this embodiment, the video feed 420 of User B 460 is a combination of the video 400 of User A as a background, which then has defined occlusion-free and occlusion regions. These regions of the video 400 of User A are transmitted from the sender to the receiver or shared through other software solutions or devices. Here, User B 460 transmits video information to User C 450 and User A 400. In addition, User A 400 and User C 450 transmit their respective video information between themselves and to User B 460. The occlusion-free regions may be described using a scene description object separate from the regular scene description. This additional scene description only contains information about the occlusion-free regions and is therefore not used for rendering, but provides a map for the occlusion-free regions.

[0040] After receiving information about the other users' video displays, regions are defined as occluded or occlude-free, and then a combination of the two is rendered on user B's screen 410. User B's screen 410, shown in FIG. 4, uses user A's video 400 as a background, with at least one region marked as an occlusion-free region 460 and another region marked as an occlusion region 430. In FIG. 4, occlusion region 430 has user C's video 440 drawn on it. In some cases, it may be necessary to change the placement of one or more screens due to new information: an additional user, more presentation information being drawn, or, for example, a user becoming a focal point or requiring more screen real estate. With the additional information, each user's screen may require the occlusion or occlusion-free regions to be redefined to change and take into account changing circumstances. For example, referring to FIG. 4, if an additional user were to join the presentation, all users' current screens would have to redefine their occlusion or occlusion-free regions and re-render each of their screens to ensure that important information is not occluded.

[0041] Figure 5 shows the 360 ​​in use. Degree FIG. 1 illustrates in detail one embodiment of a video system. In an exemplary embodiment, the video system is 360°. Degree A video presentation is taking place. The video presentation can be, for example, a video conference, a video chat, a video or other information exchange in which visual and audio information is present. In this embodiment, User B 460 is connected to User A 500 through 360. Degree User A 530 receives a 360° video from Room A and a 2-d video from User C 540. The user may want to view both videos on his / her device. However, if the user wants to make the most of the device display, he / she may want to view the 360° video from Room A. Degree The video may need to capture the entire screen of User B's device, and then the 2D video from User C may be captured in Room A. of 360 Degree The video feed 520 of user B must be overlaid on top of the video. In this embodiment, the video feed 520 of user A is a combination of the video 500 of user A as a background, which then has defined occlusion-free and occlusion regions. These regions of the video 500 of user A are transmitted from the sender to the receiver via the MRF 550 or shared via other software solutions or devices. The MRF 550 can be used to create a single scene description for the conversation session. This scene description describes the overall scene for each remote client. The same scene description may optionally include additional nodes to signal occlusion-free regions, or may include a separate root node that includes only nodes of occlusion-free regions.

[0042] Here, user B sends its video information to user C and user A. In addition, users A and C send their respective video information between themselves and to user B. The occlusion-free regions may be described using a scene description object separate from the normal scene description. This additional scene description only contains information about the occlusion-free regions and is therefore not used for rendering, but provides a map for the occlusion-free regions.

[0043] After receiving information about other users' video displays, regions are defined as occlusion or occlusion-free, and then a combination of the two is rendered on User B's screen 510. User B's screen 510, shown in FIG. 5, uses User A's video 500 as a background, with at least one region marked as an occlusion-free region 560 and another region marked as an occlusion region 530. In FIG. 5, the occlusion region 530 has User C's video 440 drawn on it. R In some cases, it may be necessary to change the placement of one or more screens due to new information: an additional user, more presentation information being rendered, or, for example, a user becoming the focus or requiring more screen area. With the additional information, each user's screen may require the occlusion or occlusion-free regions to be redefined to change and take into account the changing circumstances. For example, using FIG. 5, if an additional user were to join the presentation, all users' current screens would have to redefine their occlusion or occlusion-free regions and re-render each of the users' screens to ensure that important information is not occluded.

[0044] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the implementation to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations.

[0045] Some embodiments may relate to systems, methods, and / or computer-readable media at any possible level of technical detail of integration. Furthermore, one or more of the above components described above may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium (or media) having computer-readable program instructions for causing a processor to perform operations.

[0046] A computer readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer readable storage media includes the following: portable computer diskettes, hard disks, random access memories (RAM), read only memories (ROM), erasable programmable read only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disk read only memories (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in a groove having instructions recorded thereon, and any suitable combination of the above. Computer-readable storage media, as used herein, should not be construed as being a transitory signal per se, such as electric waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted through wires.

[0047] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing device.

[0048] The computer readable program code / instructions for performing the operations may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or object-oriented programming languages ​​such as Smalltalk, C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or a connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry to perform an aspect or operation.

[0049] These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing apparatus perform the operations specified in one or more blocks or blocks of the flowcharts and / or block diagrams. These computer readable program instructions may also be stored on a computer readable storage medium that can direct a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that a computer readable storage medium having instructions stored thereon includes an article of manufacture containing instructions that perform aspects of the operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0050] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device and cause the computer, other programmable apparatus, or other device to perform a sequence of operations to create a computer-implemented process, such that the instructions executing on the computer, other programmable apparatus, or other device perform the operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0051] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of instructions, including one or more executable instructions for performing specified logical operations. The methods, computer systems, and computer-readable media may include additional, fewer, different, or differently arranged blocks than those shown in the figures. In some alternative implementations, the operations described in the blocks may occur in a different order than that described in the figures. For example, two blocks shown in succession may in fact be executed simultaneously or substantially simultaneously, or the blocks may sometimes be executed in the reverse order depending on the functionality involved. It is also noted that each block of the block diagrams and / or flowchart diagrams, as well as combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified operations or acts, or executes a combination of dedicated hardware and computer instructions. It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limiting of the implementation. Thus, the operation and behavior of the systems and / or methods are described herein without reference to any specific software code. - It is understood that software and hardware may be designed to implement the systems and / or methods based on the description herein.

Claims

1. 1. A method executed by at least one processor, the method comprising: receiving a first video input corresponding to a 360 degree video; receiving one or more secondary video inputs; defining one or more occlusion free regions in the first video input; Each occlusion-free region represents an area of ​​the video that is not overlaid by another image or video. Steps and signaling location information of the one or more occlusion-free regions in a Session Description Protocol (SDP) via a Third Generation Partnership Project (3GPP) overlay attribute; The 3GPP overlay attribute signals whether the respective 3GPP overlay covers an occlusion-free region or not. Steps and sending the SDP to a receiver; rendering an output video that includes the first video input with the one or more second video inputs overlaid in areas that do not include the one or more occlusion free regions; A method comprising:

2. the position information for each occlusion free region is based on a position of the respective occlusion free region in a coordinate system; The method of claim 1.

3. the one or more second video inputs are 360 ​​degree video or two-dimensional video; The method of claim 1.

4. The method further comprises: updating each occlusion-free region in the SDP; The step of updating each occlusion free region includes: removing or adding new occlusion-free regions during the session; providing one or more new SDP lines during the session; The method of claim 1 , comprising:

5. The size of each of the occlusion-free regions is updated. The method according to claim 4.

6. The method further comprises: adding a flag to the SDP to signal whether a selected 3GPP overlay attribute covers an occlusion free region or not; The method of claim 1 , comprising:

7. modifying the rendering in the output video in response to updates in the SDP; The method according to claim 4.

8. An apparatus comprising: at least one memory configured to store program code; at least one processor configured to read the program code and to act as instructed by the program code; the program code comprising: a first receiving code configured to cause the at least one processor to receive a first video input corresponding to a 360 degree video; second receiving code configured to cause the at least one processor to receive one or more second video inputs; a defining code configured to cause the at least one processor to define one or more occlusion free regions in the first video input; Each occlusion-free region represents an area of ​​the video that is not overlaid by another image or video. A definition code, signaling code configured to cause the at least one processor to signal location information of one or more occlusion-free regions in a Session Description Protocol (SDP) via a Third Generation Partnership Project (3GPP) overlay attribute; The 3GPP overlay attribute signals whether the respective 3GPP overlay covers an occlusion-free region or not. A signaling code; a transmitting code configured to cause the at least one processor to transmit the SDP to a receiver; rendering code configured to cause the at least one processor to render an output video that includes the first video input with the one or more second video inputs overlaid in areas that do not include the one or more occlusion free regions; 13. An apparatus comprising:

9. the position information for each occlusion free region is based on a position of the respective occlusion free region in a coordinate system; 9. The apparatus of claim 8.

10. the one or more second video inputs are 360 ​​degree video or two-dimensional video; 9. The apparatus of claim 8.

11. the program code further includes update code; The update code may include, in the at least one processor: Update each occlusion-free region in the SDP, Remove or add new occlusion-free regions during the session, and causing one or more new SDP lines to be provided during said session; It is configured as follows:

9. The apparatus of claim 8.

12. The size of each of the occlusion-free regions is updated.

12. The apparatus of claim 11.

13. The program code further includes additional code, The additional code causes the at least one processor to: Add a flag to the SDP; signaling whether the selected 3GPP overlay attributes are directed to an occlusion-free region; It is configured as follows:

9. The apparatus of claim 8.

14. In response to the update in the SDP, The program code further comprises: causing the at least one processor to modify a rendering in the output video; It is configured as follows:

12. The apparatus of claim 11.

15. A computer program comprising instructions, When executed by at least one processor, the instructions cause the at least one processor to: receiving a first video input corresponding to a 360-degree video; receiving one or more second video inputs; defining one or more occlusion free regions in the first video input, each occlusion free region representing an area of ​​a video that is not overlaid by another image or video; signaling location information of one or more occlusion-free regions in a Session Description Protocol (SDP) via a Third Generation Partnership Project (3GPP) overlay attribute, the 3GPP overlay attribute signaling whether a respective 3GPP overlay covers an occlusion-free region; sending the SDP to a receiver; and rendering an output video that includes the first video input with the one or more second video inputs overlaid in areas that do not include the one or more occlusion free regions. A computer program that causes

16. the position information for each occlusion free region is based on a position of the respective occlusion free region in a coordinate system; 16. A computer program product as claimed in claim 15.

17. the one or more second video inputs are 360 ​​degree video or two-dimensional video; 16. A computer program product as claimed in claim 15.

18. The instructions further include causing the at least one processor to: updating each occlusion-free region in the SDP; Updating each occlusion-free region is Removing or adding new occlusion-free regions during a session, and providing one or more new SDP lines during said session; Including, 16. A computer program product as claimed in claim 15.

19. The size of each of the occlusion-free regions is updated.

20. A computer program product as claimed in claim 18.

20. The instructions further include causing the at least one processor to: adding a flag to the SDP to signal whether the selected 3GPP overlay attribute is directed to an occlusion-free region; 16. A computer program product as claimed in claim 15.

Images