Method, apparatus and program for signaling occlusion-free regions in 360 videoconferencing - Patents.com

Abstract

1. A technique for defining an occluded-free region in a 360 video conference, the technique comprising: receiving a first video input, the first video input being a 360 degree video conference; receiving one or more second video inputs; defining one or more regions in the first video input as occluded-free regions that do not overlap with any other images or videos; transmitting the one or more occluded-free regions to a receiver; and rendering an output video including the first video input overlaid with the one or more second video inputs in areas that do not include the one or more occluded-free regions.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. patent application Ser. No. 63 / 275,795, filed November 4, 2022, and U.S. patent application Ser. No. 17 / 973,301, filed October 25, 2021, the disclosures of which are incorporated by reference in their entireties herein.

[0002] This disclosure provides a method for signaling occluded-free regions in 360 video conferencing, which are regions of a 360 video that should not be covered by any overlays, since the occluded-free regions in a 360 video contain important information. [Background technology]

[0003] 3GPP TS26.114 defines a video conferencing system for mobile handsets. This specification supports video conferencing with terminals that support the capture and transmission of 360 video. The standard also supports adding overlays to the 360 ​​video. The 360 ​​video and corresponding overlays may be rendered together with other 2-d videos from other remote participants in the conference call.

[0004] The current 5G media streaming architecture defined in 3GPP TS26.114 provides a general framework for video conferencing over mobile networks. During a video conference, a remote participant can receive 360-video from a first room and 2-d video from another user. The user may want to see both videos on his / her terminal. However, if the user wants to make the most of the device's display, the 360-video from the first room must occupy the entire screen of the other user's device, and then the 2-d video from the other user must be overlaid on top of the video from the first room.

[0005] The current standard does not define a method to signal the occlusion-free regions of the first room: these are the areas of the 360 ​​video that should have important information about the first room (participants in the room, or the presentation display) and that should not be occluded by overlaying video from other users at the receiving remote terminal. Summary of the Invention [Means for solving the problem]

[0006] The following presents a simplified summary of one or more embodiments of the present disclosure in order to provide a basic understanding of such embodiments. This summary is not an extensive overview of all contemplated embodiments, and is not intended to identify key or critical elements of all embodiments or to delineate the scope of any or all embodiments. Its sole purpose is to present some concepts of one or more embodiments of the present disclosure in a simplified form as a prelude to the more detailed description that is presented later.

[0007] The present disclosure provides a method for signaling occlusion-free regions in a 360 video conference.

[0008] According to an exemplary embodiment, a method for defining an occluded-free region in a 360-degree video conference executed by at least one or more processors. The method includes receiving a first video input corresponding to the 360-degree video conference. The method further includes receiving one or more second video inputs. The method further includes defining one or more regions in the first video input as occluded-free regions that do not overlap with any other image or video. The method further includes transmitting the one or more occluded-free regions to a receiver and rendering an output video including the first video input overlaid with the one or more second video inputs in the regions that do not include the one or more occluded-free regions.

[0009] According to an exemplary embodiment, an apparatus for defining an occluded-free region in a 360-degree video conference includes at least one memory configured to store computer program code and at least one processor configured to access the computer program code and operate as directed by the computer program code. The computer program code includes a first receiving code configured to cause the at least one processor to receive a first video input corresponding to a 360-degree video conference. The computer program code further includes a second receiving code configured to cause the at least one processor to receive one or more second video inputs. The computer program code further includes a defining code configured to cause the at least one processor to define one or more regions in the first video input as occluded-free regions that do not overlap with other images or videos. The computer program code further includes transmitting code configured to cause the at least one processor to transmit the one or more occluded-free regions to a receiver, and rendering code configured to cause the at least one processor to render an output video including the first video input overlaid with the one or more second video inputs in areas that do not include the one or more occluded-free regions.

[0010] According to an exemplary embodiment, a non-transitory computer-readable medium having stored thereon computer instructions that, when executed by at least one processor, cause the at least one processor to perform a method. The method includes receiving a first video input corresponding to a 360-degree video conference. The method further includes receiving one or more second video inputs. The method further includes defining one or more regions in the first video input as occluded-free regions that do not overlap with any other image or video. The method further includes transmitting the one or more occluded-free regions to a receiver and rendering an output video that includes the first video input overlaid with the one or more second video inputs in the regions that do not include the one or more occluded-free regions.

[0011] Additional embodiments are set forth in the description that follows, and in part will be apparent from the description, and / or may be learned by practice of presented embodiments of the present disclosure.

[0012] The above and other features and aspects of the embodiments of the present disclosure will become apparent from the following description taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0013] [Figure 1] FIG. 2 is a diagram of an exemplary network device in accordance with various embodiments of the present disclosure. [Diagram 2] 1 is a flow diagram of an example process for defining an occlusion-free region in accordance with various embodiments of the present disclosure. [Diagram 3] 1 is a schematic illustration of an occluded free region and an occluded region, according to various embodiments of the present disclosure. [Figure 4] 1 is a schematic diagram of a 360 video system in accordance with various embodiments of the present disclosure. [Diagram 5] 1 is a schematic diagram of a 360 video system in accordance with various embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The following detailed description of the exemplary embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings may identify the same or similar elements.

[0015] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the embodiments to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the embodiments. Moreover, one or more features or components of one embodiment may be incorporated or combined with another embodiment (or one or more features of another embodiment). Furthermore, in the flow charts and descriptions of the operations provided below, it should be understood that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed (at least partially) simultaneously, and one or more operations may be reordered.

[0016] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limiting of the embodiment. Thus, the operation and behavior of the systems and / or methods have been described herein without reference to any specific software code. It should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0017] Although particular combinations of features are recited in the claims and / or disclosed herein, these combinations are not intended to limit the disclosure of possible embodiments. Indeed, many of these features can be combined in ways not specifically recited in the claims and / or disclosed herein. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible embodiments includes each dependent claim in combination with all other claims in the claim set.

[0018] No element, act, or instruction used herein should be construed as critical or essential unless expressly described as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and may be used interchangeably with "one or more." When only one item is intended, the term "one" or similar words are used. Also, as used herein, the terms "has," "have," "having," "include," "including," and the like are intended to be open-ended terms. Furthermore, the phrase "based on" is intended to mean "based at least in part on," unless otherwise noted. Furthermore, phrases such as "at least one of [A] and [B]" or "at least one of [A] or [B]" should be understood as including only A, only B, or both A and B.

[0019] When using traditional methods for video conferencing with 360 video, current systems rely on transcribing the 360 ​​video into a two-dimensional space and overlaying that two-dimensional view with other relevant information. For example, if a user is participating in a 360 meeting with some kind of presentation, current systems must choose to display either the meeting (360 video), the presentation (2D video), or some kind of rendering that overlaps the meeting video with the presentation drawn over a portion of the meeting. Due to the increasing occurrence of remote work, the need for collaboration between colleagues requires a better way to see both the participants and the subject of a virtual meeting.

[0020] 1 illustrates an exemplary system 100 of one embodiment for using the 360 ​​videoconferencing method. The exemplary system 100 may be one of a variety of systems, such as a personal computer, a mobile device, a cluster of computers, a server, an embedded device, an ASIC, a microcontroller, or any other device capable of executing code. A bus 110 connects the exemplary system 100 together so that all components can communicate with each other. The bus 110 connects the processor 120, the memory 130, the storage component 140, the input component 150, the output component 160, and the interface component.

[0021] The processor 120 may be a single processor, a processor with multiple processors therein, a cluster of processors (two or more), and / or distributed processing. The processor executes instructions stored in both the memory 130 and the storage component 140. The processor 120 acts as a computing device and performs operations for the text normalizer. The memory 130 is a high-speed storage and may enable lookup to any memory device by using a cache memory that may be closely associated with one or more CPUs. The storage component 140 may be one of any long-term storage such as a HDD, SSD, magnetic tape, or any other long-term storage format.

[0022] Input component 150 may be any file type or a signal from a user interface component such as a camera or text capture device. Output component 160 outputs the processed information to a communication interface 170, which may be a speaker or other communication device and can display the information to a user or another observer, such as another computing system.

[0023] 2 shows a flow diagram of an example embodiment of a process for conducting a video conference. The operations detailed in the process include receiving an input video 200, defining an occluded-free region 210, and defining an occluded region 220. The defined regions 210 and 220 are transmitted to a receiver 230. Rendering 240 is then performed to render an output video having some combination of occluded-free and occluded videos. The process then determines whether there is a change 250 to restart the process upon receiving the input video 200. If it is determined that there is no change, the process ends (260).

[0024] FIG. 3 illustrates an embodiment of an occluded region 320 and an occluded-free region 310. On a screen 300, space may be allocated to either an occluded-free region 310 or an occluded region 320. An occluded region 310 is a region of the screen 300 where content or other information is being drawn / rendered or otherwise output. This occluded region 310 may not have other content rendered on it or otherwise overlapped therewith. On the other hand, an occluded region 320 is marked as a region that has no content, has low priority content, or where other content may be placed on it. An occluded-free region is defined by its location in that coordinate system and its area in that coordinate system. For example, a spherical rectangular region is defined by its center and its yaw and pitch ranges around the center. Similarly, in various coordinate systems, a region may be defined using the parameters of that coordinate system. In some embodiments, a region of the screen 300, an entire screen, or multiple screens may be marked as either an occluded region or an occluded-free region.

[0025] Various mechanisms may be used to signal occluded-free regions. For example, in some embodiments, signaling an occluded-free region may be to send the coordinates of such a region as part of the session description, as an item in a list of occluded-free regions. In other embodiments, signaling an occluded-free region may be done by defining a node in the scene description that defines the occluded-free region and its properties (e.g., transparent, does not contain media objects). In other embodiments, signaling an occluded-free region may be done by defining a separate scene description that defines only the occluded-free region. The occluded-free region of a 360 video may be signaled in the SDP a=3gpp_occludefree attribute. The video component may have the location and size (extent) of the region. Since this component is defined by 3gpp_occludefree, ITT4RT knows that this signaling is used to signal a region that does not contain actual media, but should not be covered.

[0026] In some embodiments, the scene description may include a node for each occluded-free region. The texture property of the node may be set to an alpha channel with opacity 0 (fully transparent). Alternatively, a new MIME type may be defined for the occluded-free node. For example, in a glTF scene description, if alphaMode=MASK and alphaCutOff=1.1 for a texture, the object is transparent (not rendered). A new attribute may be added to the glTF specification to explicitly signal these regions as occluded-free regions.

[0027] FIG. 4 details one embodiment of a 360 video system in use. In an exemplary embodiment, a 360 video presentation is taking place. The video presentation may be, for example, a video conference, a video chat, a video, or other information exchange in which visual and audio information is present. In this embodiment, User B 460 receives a 360 video from User A 400 and a 2-d video from User C 450. The user may want to see both videos on his terminal. However, if the user wants to make the most of the device's display, the 360-video of Room A must occupy the entire screen of User B's device, and then the 2-d video from User C must be overlaid on top of the 360-video of Room A 400. In this embodiment, User B 460's video feed 420 is combined with User A's video 400 as a background, with occluded-free and occluded regions defined. These regions of User A's video 400 are transmitted from the transmitter to the receiver, or shared through other software solutions or devices. Here, User B 460 sends its video information to User C 450 and User A 400. In addition, User A 400 and User C 450 send each of their individual video information between themselves and to User B 460. The occluded-free regions may be described in a scene description object separate from the normal scene description. This additional scene description is not used for rendering, since it only contains information about the occluded-free regions, but it does provide a map of the occluded-free regions.

[0028] After receiving information about the other users' video displays, regions are defined as occluded or occluded-free, and then a combination of both is rendered on user B's screen 410. User B's screen 410, shown in FIG. 4, uses user A's screen 400 as a background, with at least one region marked as occluded-free 460 and another region marked as occluded 430. In FIG. 4, in the occluded region 430, user C's video 440 is drawn on top of the region. In some cases, the arrangement of one or more of the screens may need to be changed due to additional users, more presentation information to be drawn, or new information, such as, for example, a user becoming the focus or otherwise requiring more screen space. With the information added, each user's screen must redefine the occluded or occluded-free regions to account for the change in circumstances. For example, referring to FIG. 4, if an additional user joins the presentation, all users' current screens must re-render each of the users' screens by redefining the occluded or occluded-free regions to ensure that important information is not occluded.

[0029] FIG. 5 details one embodiment of a 360 video system in use. In an exemplary embodiment, a 360 video presentation is taking place. The video presentation may be, for example, a video conference, a video chat, a video, or other information exchange in which visual and audio information is present. In this embodiment, User B 460 receives a 360 video from User A 500 and a 2-d video from User C 540. The user may want to see both videos on his terminal. However, if the user wants to make the most of the device's display, the 360-video of room A needs to take up the entire screen of User B's device, and then the 2-d video from User C must be overlaid on top of the 360-video of room A. In this embodiment, User B's video feed 520 is combined with User A's video 500 as a background, with occluded-free and occluded regions defined. These regions of User A's video 500 are sent from the transmitter to the receiver via the MRF 550, or shared via other software solutions or devices. The MRF 550 may be used to create a single scene description for a conversation session, which describes the entire scene to each remote client. The same scene description may optionally include additional nodes for signaling occluded-free regions, or may include a separate root node that includes only nodes for occluded-free regions.

[0030] Here, user B sends their video information to user C and user A. In addition, users A and C send each of their individual video information between themselves and to user B. The occluded-free regions may be described in a scene description object separate from the normal scene description. This additional scene description contains only information about the occluded-free regions and is therefore not used for rendering, but provides a map of the occluded-free regions.

[0031] After receiving information about the other users' video displays, regions are defined as occluded or occluded-free, and then a combination of both is rendered on user B's screen 510. User B's screen 510, shown in FIG. 5, uses user A's screen 500 as a background, with at least one region marked as occluded-free 560 and another region marked as occluded 530. In FIG. 5, in the occluded region 530, user C's video 440 is drawn over the region. In some cases, the arrangement of one or more of the screens may need to be changed due to additional users, more presentation information to be drawn, or new information, such as, for example, a user becoming the focus or otherwise requiring more screen space. With the information added, each user's screen must redefine the occluded or occluded-free regions to account for the change in circumstances. For example, using FIG. 5, if an additional user joins the presentation, all users' current screens must redefine the occluded or occluded-free regions and re-render each of the users' screens to ensure that important information is not occluded.

[0032] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the embodiments.

[0033] Some embodiments may relate to a system, a method, and / or a computer-readable medium at any possible level of integration of technical details. Furthermore, one or more of the above-mentioned components may be implemented as instructions stored on a computer-readable medium and executable by at least one processor (and / or may include at least one processor). The computer-readable medium may include a computer-readable non-transitory storage medium having computer-readable program instructions for causing a processor to perform operations.

[0034] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves with instructions recorded on them, and any suitable combination of the above. As used herein, a computer-readable storage medium should not be construed as a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over electrical wires.

[0035] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to the respective computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing / processing device.

[0036] The computer readable program code / instructions for performing the operations may be either assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk or C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, an electronic circuit, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can execute computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuit to perform an aspect or operation.

[0037] These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to manufacture a machine such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, perform the operations specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may be stored in a computer-readable storage medium that can direct a computer-programmable data processing apparatus and / or other apparatus to operate in a particular manner, such that the computer-readable storage medium having instructions stored therein comprises an article of manufacture including instructions that perform aspects of the operations specified in the flowcharts and / or block diagram blocks.

[0038] The computer-readable program instructions may also be loaded into a computer, other programmable data processing apparatus, or other device, and cause a sequence of operations to be performed on the computer, other programmable apparatus, or other device, such that the instructions executing on the computer, other programmable apparatus, or other device perform the operations specified in the flowchart and / or block diagram blocks, to generate a computer-implemented process.

[0039] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer-readable media according to various embodiments. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of an instruction, including one or more executable instructions for performing the specified logical operations. The methods, computer systems, and computer-readable media may include additional, fewer, different, or differently arranged blocks compared to those shown in the figures. In some alternative implementations, the operations shown in the blocks may occur in a different order than the order shown in the figures. For example, two blocks shown in succession may in fact be executed simultaneously or substantially simultaneously, or the blocks may be executed in reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, as well as combinations of blocks in the block diagrams and / or flow diagrams, may be implemented by a dedicated hardware-based system that performs the specified operations or acts, or executes a combination of dedicated hardware and computer instructions.

[0040] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not intended to limit the embodiments. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code, and it will be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0041] The above disclosure also encompasses the embodiments listed below.

[0042] (1) A method for defining an occluded-free region in a 360 degree video conference, the method being executed by at least one processor and including the steps of receiving a first video input corresponding to a 360 degree video conference; receiving one or more second video inputs; defining one or more regions in the first video input as occluded-free regions that do not overlap with any other images or videos; transmitting the one or more occluded-free regions to a receiver; and rendering an output video including the first video input overlaid with the one or more second video inputs in areas that do not include the one or more occluded-free regions.

[0043] (2) The method according to feature (1), wherein the occluded-free region is defined by a position of the occluded-free region in a coordinate system.

[0044] (3) The method of any one of features (1) or (2), wherein the one or more second video inputs are 360-degree video or 2-D video.

[0045] (4) The method of any one of features (1) to (3), wherein the occlusion-free region is dynamic and changes during the video conferencing session.

[0046] (5) The method of any one of features (1) to (4), further comprising: responding to a change in at least one piece of information in the input video by altering a rendering in the output video.

[0047] (6) The method of any one of features (1) to (5), wherein the signaling of the occluded-free region includes one or more of the following operations: sending coordinates of one or more occluded-free regions as part of a session description as items in a list of occluded-free regions; defining a node of a scene description that defines one or more occluded-free regions and properties of one or more occluded-free regions; defining a separate scene description that defines only the occluded-free regions.

[0048] (7) The method according to any one of features (1) to (6), wherein the information of each occluded-free region is updated during a session, and new occluded-free regions are added or existing occluded-free regions are removed.

[0049] (8) An apparatus for defining an occluded-free region in a 360 degree video conference, the apparatus comprising: at least one memory configured to store computer program code; and at least one processor configured to access the computer program code and operate as directed by the computer program code, the computer program code including: a first receiving code configured to cause the at least one processor to receive a first video input corresponding to a 360 degree video conference; a second receiving code configured to cause the at least one processor to receive one or more second video inputs; a defining code configured to cause the at least one processor to define one or more regions in the first video input as occluded-free regions that do not overlap with any other image or video; a transmitting code configured to cause the at least one processor to transmit the one or more occluded-free regions to a receiver; and a rendering code configured to cause the at least one processor to render an output video including the first video input overlaid with the one or more second video inputs in areas that do not include the one or more occluded-free regions.

[0050] (9) The apparatus of feature (8), wherein the occlusion-free region is defined by a position of the occlusion-free region in a coordinate system.

[0051] (10) The apparatus of feature (8) or (9), wherein the one or more second video inputs are 360-degree video or 2-D video.

[0052] (11) The apparatus of any one of features (8) to (10), wherein the occlusion-free region is dynamic and changes during the video conferencing session.

[0053] (12) The apparatus of any one of features (8) to (11), further comprising response code configured to cause the at least one processor to respond to a change in at least one information in the input video by altering a rendering in the output video.

[0054] (13) The apparatus of any one of features (8) to (12), wherein the signaling code configured to cause at least one processor to signal the occluded-free region further causes the at least one processor to perform one or more of the following operations: sending coordinates of the one or more occluded-free regions as part of a session description as items in a list of occluded-free regions; defining a node of a scene description that defines one or more occluded-free regions and properties of the one or more occluded-free regions; defining a separate scene description that defines only the occluded-free region.

[0055] (14) The device of any one of features (8) to (13), wherein information about each occluded-free region is updated during a session, and new occluded-free regions are added or existing occluded-free regions are removed.

[0056] (15) A non-transitory computer-readable medium having stored thereon computer instructions that, when executed by at least one processor, cause the at least one processor to receive a first video input corresponding to a 360-degree video conference, receive one or more second video inputs, define one or more regions in the first video input as occluded-free regions that do not overlap with any other images or videos, transmit the one or more occluded-free regions to a receiver, and render an output video including the first video input overlaid with the one or more second video inputs in areas that do not include the one or more occluded-free regions.

[0057] (16) The non-transitory computer-readable medium of feature (15), wherein the occlusion-free region is defined by a position of the occlusion-free region in a coordinate system.

[0058] (17) The non-transitory computer-readable medium of feature (15) or (16), wherein the one or more second video inputs are 360-degree video or 2-D video.

[0059] (18) The non-transitory computer-readable medium of any one of features (15) to (17), wherein the occlusion-free region is dynamic and changes during the video conferencing session.

[0060] (19) The non-transitory computer-readable medium of any one of features (15) to (18), further causing the at least one processor to respond to a change in at least one information in the input video by modifying a rendering in the output video. (20) The non-transitory computer-readable medium of any one of features (15) to (19), wherein the signaling of the occluded-free region includes one or more of the following operations: sending coordinates of one or more occluded-free regions as part of a session description as items in a list of occluded-free regions; defining a node of a scene description that defines one or more occluded-free regions and properties of one or more occluded-free regions; defining a separate scene description that defines only the occluded-free regions. [Explanation of symbols]

[0061] 100 Systems 110 Bus 120 processors 130 Memory 140 Storage Components 150 Input Components 160 Output Components 170 Communication Interface 200 Input Video 210 Occlusion-free area 220 Occluded Area 230 Receiver 300 screens 310 Occluded Free Area 320 Occluded Area 400 screens 410 screens 420 Video Feed 430 Occluded Area 440 Videos 460 Occluded Free 500 Videos 510 screens 520 Video Feed 530 Occluded Area 560 Occluded Free

Claims

1. 1. A method for defining an occlusion-free region in a 360 video conference, the method being executed by at least one processor and comprising: receiving a first video input corresponding to a 360 degree video conference; receiving one or more secondary video inputs; defining one or more regions in the first video input as occlusion-free regions that do not overlap with any other image or video; transmitting the one or more occlusion-free regions to a receiver; rendering an output video including the first video input overlaid with the one or more second video inputs in areas not including the one or more occluded-free areas; A method comprising:

2. The method of claim 1 , wherein the occlusion-free region is defined by a position of the occlusion-free region in a coordinate system.

3. The method of claim 1 , wherein the one or more second video inputs are 360-degree video or 2-D video.

4. The method of claim 1 , wherein the occlusion-free region is dynamic and changes during a videoconferencing session.

5. responding to a change in at least one information in the first video input by modifying the rendering in the output video; The method of claim 1 further comprising:

6. The step of signaling the occlusion-free region comprises the following operations: i) transmitting coordinates of the one or more occluded free regions as part of a session description as an item in a list of occluded free regions; ii) defining nodes of a scene description that define said one or more occluded free regions and properties of said one or more occluded free regions; iii) defining a separate scene description that defines only the occlusion-free region; The method of claim 1 , comprising one or more of:

7. The method of claim 1 , wherein the information of each occluded-free region is updated during a session, and new occluded-free regions are added or existing occluded-free regions are removed.

8. An apparatus for defining occlusion-free regions in a 360° video conference, configured to perform the method according to any one of claims 1 to 7.

9. A program which, when executed by at least one processor, causes the at least one processor to perform the method according to any one of claims 1 to 7.

Images