Video conference apparatus using spatial virtual reality environment, video conference method, and computer program

Abstract

To provide a video conference device for holding a round-table video conference, a video conference method, and a computer program for executing the video conference method.SOLUTION: A video conference device (100) reproduces videos (110a, 110b) of a plurality of video conference participants in a plurality of video reproduction areas (120a, 120b, 220a to 220c, 420a to 420d, 470a to) or video reproduction spaces disposed in spatial virtual reality environments (120200300400450,,,). 470h. The videoconference device renders said spatial virtual reality environments (120200300400450) with video playing areas (120a, 120b, 220a - 220c, 420a - 420d, 470a - 470h) or video playing spaces into an overall image (112) for a single videoconference participant.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] SUMMARY OF THE INVENTION An embodiment according to the present invention relates to a videoconferencing device. A further embodiment according to the invention relates to a videoconferencing method. A further embodiment according to the invention relates to a computer program for performing the videoconferencing method. Generally, embodiments in accordance with the present invention relate to concepts for conducting round-table video conferences. [Background technology]

[0002] Video conferencing systems are becoming increasingly popular, especially during the COVID-19 pandemic.

[0003] Today, in such applications, participants' video streams are mostly represented as individual tiles on the screen, and if there are too many participants, some of the video streams may be hidden or may appear to change depending on who is currently speaking.

[0004] An example of a video conference is shown in Figure 5. Microsoft has introduced a "together mode" for TEAMS, where participants are tied to fixed seats, like in a lab room. However, this is also just a flat layout. An example is shown in Figure 6.

[0005] In view of this situation, it is desirable to create a concept that provides an improved user experience when participating in a video conference. Summary of the Invention

[0006] An embodiment in accordance with the present invention creates a videoconferencing device configured to play back video of multiple videoconference participants in multiple video playback areas or video playback spaces arranged in a spatial (e.g., three-dimensional or spatially three-dimensional) virtual reality environment. The videoconferencing device is configured to render the spatial (e.g., three-dimensional) virtual reality environment having the video playback areas and / or video playback spaces (e.g., from the perspective of a virtual camera at a virtual camera position) into an overall (or composite) image (e.g., as seen by a virtual camera (122) representing the participant's field of view), for example, for a particular (or single or individual) videoconference participant (e.g., observer).

[0007] The video playback area is typically the surface of a monitor or canvas in a cinema, the virtual playback screen in this context. The video playback space is a set of 3D data with position and light field / image texture information from a scene typically captured by multiple cameras and / or 3D sensors, processed and represented as a point cloud, multi-view images, or light field data. The video playback space can provide various perspective and depth information for the captured objects or scenes by a renderer.

[0008] This embodiment of the present invention is based on the idea that an improved user experience in video conferencing can be provided by playing back videos of multiple video conference participants in multiple video playback areas or on or in video playback spaces (or video playback rooms) arranged in a virtual reality environment, and rendering a spatial virtual reality environment having video playback areas or video playback spaces to obtain an overall picture (or comprehensive image) for a specific (or single or individual) video conference participant. It has been found that rendering videos of multiple video conference participants in a spatial virtual reality environment provides a much better sense of engagement to video conference participants viewing the rendered overall or comprehensive image compared to traditional tiled arrangements. In particular, it has been found that video conference participants can focus on the video conference participant(s) of their current interest and experience an impression similar to that obtained from participating in an actual on-site conference.

[0009] For example, the placement of the video playback area and / or video playback space in a spatial virtual reality environment allows videoconference participants to select their viewpoint and / or their line of sight and / or their focal point (e.g., in the sense of zoom). Furthermore, it has been found that placing the video playback area and / or video playback space in a "natural" location within a three-dimensional scene (rather than as flat tiles within a rectangular grid) makes videoconferences more realistic and therefore often more informative. In contrast, the traditional "flat tile" format has been found to be confusing and tiring for many people, and concentration often is lost relatively quickly when using traditional techniques.

[0010] In conclusion, playing videos of multiple video conference participants in multiple video playback areas or video playback spaces arranged in a spatial virtual reality environment and depicting the spatial virtual reality environment with the video playback areas or video playback spaces enables a significantly improved user experience, further facilitating user interaction (e.g., selecting a desired field of view) and helping to reduce loss of concentration.

[0011] In a preferred environment, multiple video playback areas or multiple video playback spaces have different orientations (e.g., have different spatial orientations or are located in different non-parallel planes). Selecting different orientations for different video playback areas and / or different video playback spaces can improve the "spatial" impression of video conference participants. Furthermore, by rendering the images of different video conference participants on or in video playback areas or video playback spaces with different orientations, "natural" user interactions, such as changes in gaze direction, result in changes in the overall image (or overall picture) available to a given video conference participant, which fully matches that participant's "natural" expectations. Therefore, the user experience can be more natural and less tiring compared to conventional video conference systems.

[0012] In a preferred embodiment, the videoconferencing device is configured to adjust the observer position (e.g., the position of the virtual camera) in response to a user input. The videoconferencing device is also configured to render a spatial (three-dimensional) virtual reality environment having a video playback area and / or a video playback space in response to the observer position, e.g., to obtain a two-dimensional image of the spatial or three-dimensional virtual reality environment having the video playback area and / or the video playback space from the perspective of an observer at the observer position. The user input may be a pressed key, an action on a computer mouse, or a rotation of the user's head or a change in the gaze direction of the user's eyes under observation.

[0013] Thus, images of a spatial virtual reality environment with images of multiple videoconference participants included in the video playback area and / or video playback space can be rendered from different perspectives (e.g., defined by variable observer positions). Thus, by making the observer position adjustable, the observer position can be adjusted, for example, depending on who is currently speaking and / or depending on user preferences. As a result, the spatial virtual reality environment can be rendered from variable observer positions, thereby focusing on the currently most relevant speakers and (possibly) their respective environments. Furthermore, varying the observer position can, for example, correspond to blending between different cameras in a TV studio, giving videoconference participants the impression of familiarity. Thus, for example, adjusting the observer position depending on user input (alternatively or additionally depending on who is currently speaking) can provide a positive user experience.

[0014] In a preferred embodiment, the videoconferencing device is configured to adjust an observer's gaze direction (e.g., a virtual camera's gaze direction) in response to a user input, and to render a spatial (e.g., three-dimensional) virtual reality environment having a video playback area or video playback space in response to the observer's gaze direction. For example, the spatial virtual reality environment can be rendered to obtain a two-dimensional image of the spatial or three-dimensional virtual reality environment having a video playback area or video playback space from the perspective of an observer at an observer position, with a field of view in the observer's gaze direction. By allowing the videoconference participant (which may also be an observer) to adjust the observer's gaze direction, the videoconference participant can obtain a particularly realistic impression. For example, by allowing the observer's gaze direction (which is taken into account in rendering the overall image or comprehensive image) to be adjusted, the videoconference participant can focus their attention on one or more other videoconference participants (or images from one or more other videoconference participants) in a natural way.

[0015] For example, the adjustment of the observer gaze direction may correspond to the rotation of the videoconference participant's head, thus giving the videoconference participant the impression of sitting in a common room or around a common table with other videoconference participants, which is therefore particularly realistic and comfortable for the videoconference participant.

[0016] In a preferred embodiment, the videoconferencing device is configured to selectively render one or several video playback areas and / or one or several video playback spaces associated with other users in response to user input. Thus, a user can select whether they currently want to view the video of one other user (e.g., a videoconference participant) rendered on the respective video playback areas and / or video playback spaces of the space or three-dimensional virtual reality environment. Thus, a videoconference participant can "zoom in" on one other videoconference participant or "zoom out" to view video playback areas or video playback spaces showing the video of multiple other videoconference participants. As a result, a videoconference participant can focus their attention on a single other videoconference participant (e.g., by "zooming in" to view only the video of the single other videoconference participant and, optionally, the environment of the video playback area or video playback space associated with this single other videoconference participant). However, the user can also "zoom out" and view multiple other videoconference participants simultaneously. Thus, the user has the flexibility to selectively focus their attention on a single other videoconference participant when desired and observe multiple videoconference participants in other situations.

[0017] Furthermore, by placing a video playback area or video playback space in which the images of multiple video conference participants are displayed in a spatial virtual reality environment, transitions between the field of view of a single other video conference participant and the field of view of multiple other video conference participants can be performed in a visually pleasant manner, for example by expanding and / or contracting the field of view, and possibly by (smooth) changes in the observer position and / or observer gaze direction (which may be associated with the video conference participant at whom the drawing is currently being performed).

[0018] In a preferred embodiment, the video conferencing device is configured to arrange the video playback areas or video playback spaces along a contour (e.g., along a closed contour or line). In this way, for example, overlapping (e.g., occlusion) of video playback areas or video playback spaces associated with different video conference participants can be avoided. Furthermore, such arrangements of video playback areas or video playback spaces can closely resemble seating arrangements in many "real-world" conference situations.

[0019] Furthermore, arranging the video playback areas and / or video playback spaces along a contour can facilitate the selection of variable observer positions and / or observer gaze directions for the currently considered video conference participant, since the arrangement of the video playback areas or video playback spaces associated with different video conference participants obviates the need for complex (e.g., three-dimensional) selection of observer positions. Rather, for example, when each video playback area or video playback space is arranged along a contour, a small number of parameters may be sufficient to select an observer position and / or observer gaze direction, and thereby a desired view on the video of the other video conference participants. For example, a single (one-dimensional) movement parameter may be sufficient to describe a position along the contour, such that a very simple human interaction device for a user interface may be sufficient to select an observer position for the currently considered video conference participant.

[0020] In a preferred embodiment, the video conferencing device is configured to arrange the video playback area or video playback space along a circular, oval, square, or rectangular shape. Arranging the video playback area and / or video playback space along such shapes can model a "real" conference environment in which video conference participants sit around a circular, oval, square, or rectangular table. Thus, a "natural" user experience can be achieved.

[0021] Furthermore, by arranging the video playback area and / or video playback space along such a closed contour in the spatial virtual reality environment, the selection of the viewer position and / or viewer gaze direction can be performed in a particularly easy and efficient manner, since it is often sufficient to position the viewer (i.e., the video conference participant on whom the overall image is drawn) on the contour or within the area bounded by the contour (e.g., a circle, oval, square or rectangle). Therefore, the user interface for inputting the viewer position and / or viewer gaze direction can be simple, since the range of "reasonable" viewer positions is typically limited by such an arrangement of the video playback area or video playback space.

[0022] In a preferred embodiment, the video conferencing device is configured to position (or render) the video playback area and / or video playback space at a user's position (e.g., at a seating position / e.g., at a seat at or around a table / e.g., at a standing position around a table or counter) within a spatial (or three-dimensional) virtual reality environment. Thus, the virtual reality environment reflects a typical real-life conference environment and therefore provides a highly advantageous user impression. In particular, the video playback area or video playback space, in which the images of other video conferencing participants are displayed (or rendered) (e.g., as textures), is positioned according to user expectations arising from a "real" conference experience. Thus, the video conferencing device provides video conferencing participants with a "look and feel" similar to an "in-office" conference. This improves the user's impression and helps maintain concentration.

[0023] In a preferred embodiment, the videoconferencing device is configured to obtain one or more observation parameters (e.g., observation point parameters or "viewpoint" parameters and / or rotation angles and / or scaling factors) (e.g., from a user and / or via user input and / or from a user interface) and render a spatial (e.g., three-dimensional) virtual reality environment having a video playback area or video playback space according to the one or more observation parameters. By allowing a user (e.g., a videoconference participant) to input one or more observation parameters via the user interface, the user can adapt the field of view in the spatial virtual reality environment to his or her expectations. Thus, the user can, for example, determine his or her observer position and / or his or her observer gaze direction, and therefore which other videoconference participant or participants he or she wishes to focus on. The one or more observation parameters can be obtained in several ways. For example, the user can input the one or more observation parameters using a (manual) user interface (e.g., using a touchscreen, using keys, or using a pointing device such as a computer mouse). Alternatively, the one or more observation parameters can be derived from a user's movements, a user's gaze direction, a user's position, etc. To obtain the one or more observation parameters based on the videoconference participants' movements, various technical means can be applied, such as position trackers, eye-gaze trackers, head trackers, etc. Thus, the videoconference participants can manually or automatically adapt the one or more observation parameters according to their needs.

[0024] In a preferred embodiment, the observation parameters include a rotation angle (e.g., a rotation angle of an observer, observer camera, or virtual camera located at the center of the virtual reality environment or at the center around which the playback area or video playback space is arranged) and / or a scaling factor. By using the rotation angle as an observation parameter, the viewing direction of the virtual camera (and potentially also the position of the virtual camera) can be easily determined.

[0025] For example, the rotation angle can define the viewing direction of a virtual camera, from which a global image (or a comprehensive image) is generated. However, for example, if the position of the virtual camera (which may be equal to the observer position) is adapted to move along a predetermined line or along a predetermined closed contour (such as a circle, ellipse, square, or rectangle), the rotation angle can also define the position of the virtual camera. However, the rotation angle may optionally simultaneously define the viewing direction of the virtual camera (i.e., the observer viewing direction) and the position of the virtual camera (i.e., the observer position). This is due to the fact that, for example, for each position of the virtual camera, there may be an associated viewing direction of the virtual camera (e.g., towards the center of the contour along which the virtual camera can move), and therefore the viewing direction of the virtual camera (i.e., the observer viewing direction) can be related to the position of the virtual camera (i.e., the observer position).

[0026] In conclusion, the rotation angle is a very efficient viewing parameter since it can fully define the viewer position and / or viewer gaze direction in the form of a single scalar value that can be easily obtained from a user interface.

[0027] Furthermore, the magnification factor has been found to also constitute a very efficient viewing parameter, as it allows the user (video conference participant) to choose between focusing on a single other video conference participant or a "wide field of view" that includes images of multiple different other video conference participants.

[0028] In a preferred embodiment, the video conferencing device is configured to render a spatial (or three-dimensional) virtual reality environment having a video playback area or video playback space as viewed from a predetermined (e.g., fixed) observation point (e.g., the center of the virtual reality environment or the center around which the video playback area or video playback space is arranged) with an adjustable viewing direction and / or an adjustable scaling factor. For example, the rendering may be such that a user can select the viewing direction and / or scaling factor and thus, for example, which other video conference participants' video inputs are within their field of view or how many video conference participants' video inputs are within their field of view.

[0029] It has been found that rendering a spatial virtual reality environment from a predetermined (e.g., fixed) observation point allows for particularly easy user control, since there is no need to adjust the observation point. Furthermore, it has been found that a user (video conference participant) can nevertheless select a desired field of view by inputting (e.g., via a user interface) a selection of a gaze direction and / or a selection of a magnification factor. Because both gaze direction information and magnification factor information are typically only one-dimensional, both the selection of the gaze direction and the selection of the magnification factor may be performed using simple input means. Furthermore, it has been found that the use of a fixed, predetermined observation point matches well with the user experience in on-site conferences, where conference participants are typically stationary (e.g., sitting in their seats). Therefore, such a type of control provides a particularly good user experience.

[0030] In this regard, it should be noted that various choices of the predetermined observation point are possible. The predetermined observation point can be, for example, at the center of the virtual reality environment or at the center around which the video playback area or video playback space is arranged. However, alternatively, the predetermined observation point can also be on (or near) a contour (e.g., a circle, oval, square, or rectangle) along which the video playback area or video playback space is arranged (which can, for example, correspond to the seats arranged at a table). However, both choices of the predetermined observation point have been shown to provide a good user experience.

[0031] In a preferred embodiment, the videoconferencing device is configured to render a spatial (or three-dimensional) virtual reality environment with a video playback area or video playback space (and optionally also with an adjustable viewing direction and / or an adjustable magnification factor) as seen from a variable observation point, where the variable parameter or variable observation point is determined by, for example, one of the observation parameters.

[0032] This embodiment is based on the finding that, despite deviations from typical situations in on-site meetings, having a variable observation point may be advantageous for some users. Rendering a spatial virtual reality environment having a video playback area and / or video playback space as viewed from a variable observation point allows the video conference participant for whom the rendering is performed to have a good view of all conference participants. For example, it may be advantageous to change the observation point when another video conference participant is speaking who is seated relatively far (virtually) from the video conference participant for whom the virtual reality environment is rendered.

[0033] In an on-site (real-world) conference, another conference participant sitting far away can only be seen (and sometimes not very clearly) from a long distance, but by changing the observation point in such a situation, a much better image of the "far away" video conference participant can be provided in the rendered "big picture" or overall image.

[0034] Similarly, changing the observation point can be particularly advantageous, for example, when another videoconference participant is speaking who is sitting beside the videoconference participant whose image is being drawn. While a "real-world" conference participant usually does not have the opportunity to see the "front" of the person next to them, by changing the observation point when the person next to them is speaking in the conference, i.e., by using a variable observation point, it is possible to see the "front" of the person next to them. Therefore, in some situations, the impression may be even better than in an in-person conference. However, it should be noted that many other useful scenarios for using a variable observation point are also possible.

[0035] In a preferred embodiment, the video conferencing device is configured to adjust a variable observation point (e.g., the position of the virtual camera) in response to observation information (e.g., obtained via a user interface) or in response to angle information (or rotation angle information) that may be obtained by the user interface, for example. For example, the variable observation point may be adjusted (or may be adjustable) to be selectable or displaceable by the user along a curve in the virtual reality environment (e.g., along a curve on which a video playback area or video playback space is located).

[0036] By making the variable observation point adjustable according to observation or angle information (which may be provided by the user via a user interface, for example), users (video conference participants) have great freedom to select their "favorite" observation points, and thus the rendered overall or complete image can be better adapted to the user's preferences.

[0037] In a preferred embodiment, the video conferencing device is configured to adjust the variable observation point (e.g., the position of the virtual camera) in response to observation information (e.g., obtained via a user interface) or in response to angle information (or rotation angle information) that may be obtained via the user interface, so that the variable observation point (e.g., the position of the virtual camera) is selectable or displaceable by a user (e.g., via a user interface) along a curve in the virtual reality environment (e.g., along a curve on which a video playback area or video playback space is positioned).

[0038] By allowing a user to easily move a variable observation point along a curve (e.g., along a line), for example, there can be a predetermined path (curve) along which the variable observation point can be moved. Because a position along the predetermined curve in the virtual reality environment can be defined by a single (one-dimensional) variable, movement along this curve (which may be a predetermined path) can be controlled, for example, by one-dimensional user input. Thus, for example, the curve of the virtual reality environment can be predefined such that each point along the curve provides a good view of the three-dimensional (spatial) virtual reality environment.

[0039] Furthermore, the video conferencing device may be configured to adjust the observer gaze direction depending on the observer position information, for example so that the observer gaze direction, which specifies the gaze direction in the drawing, is automatically adapted to the observer position.

[0040] Thus, for example, for each position along a (predefined) curve in a virtual reality environment, a corresponding observer gaze direction may be defined, eliminating the need for a user to independently select an observation point and an observer gaze direction. Rather, a single (scalar) parameter entered via a user interface may be sufficient to define both a variable observation point (movable along a predetermined curve) and an observer gaze direction. For example, a relationship between a position along a (predefined) curve and an observer gaze direction may be defined to provide a good user experience for each observation point along the curve.

[0041] Furthermore, it should be noted that movement along a given curve in a virtual reality environment may be pseudo-continuous, e.g., without "hard" switching between multiple positions that provide substantially different rendered overall images, or may be stepwise (e.g., each step brings the focus to a different video playback area or video playback space).

[0042] In conclusion, by jointly adjusting the variable observation point and the observer gaze direction based on the observer position information, simple user control over the viewpoint of the rendered overall image is provided, which helps to achieve a good user experience.

[0043] Further in conclusion, an embodiment provides a videoconferencing device, the videoconferencing device configured to adjust a variable observation point (e.g., a position of a virtual camera) in response to observer position information (e.g., obtained via a user interface), such that, for example, the observation point is selectable or displaceable by a user along a curve in the virtual reality environment (e.g., along a curve on which a video playback area or video playback space is positioned). Furthermore, in this embodiment, the videoconferencing device is configured to adjust an observer gaze direction in response to the observer position information, such that, for example, the observer gaze direction, which specifies the viewing direction in the rendering, is automatically adapted to the observer position.

[0044] In a preferred embodiment, the video conferencing device is configured to adjust the observer gaze direction in response to the observer position information so that the observer gaze direction is perpendicular to the direction in which the variable observation point moves with a tolerance of + / - 10 degrees, or so that the observer gaze direction is perpendicular to the curve in which the variable observation point moves with a tolerance of + / - 10 degrees.

[0045] It has been found that aligning the observer's gaze direction so that it is substantially perpendicular to the direction or curve along which the variable observation point moves provides a pleasant user experience. Furthermore, when the curve along which the variable observation point moves is the curve along which the video playback area or video playback space is located, the adaptation of the observer's gaze direction as described above allows for a good view of the video of all video conference participants. For example, an efficient adjustment mechanism using such a concept can be provided for the front view of various video playback areas or video playback spaces (and consequently the video from various video conference participants). This achieves a good balance between user convenience and user impression.

[0046] In a preferred embodiment, the video conferencing device is configured to adjust the observer gaze direction in response to the observer position information so that the observer gaze direction is always directed to the inside of a curve, for example a closed curve or a contour, for example a circle or an ellipse, along which the variable observation point moves in response to the observer position. Alternatively, the observer gaze direction may always be directed (at least approximately) to the center of the curve (for example a closed curve or contour / for example a circle or an ellipse) along which the variable observation point moves in response to the observer position information.

[0047] By always directing your gaze inside the curve, or at least approximately to the center of the curve, you can portray the field of view of the videoconference participant sitting on the other side. Thus, the field of view of the other videoconference participant will be able to consistently provide the impression that the other videoconference participant is sitting on the other side of the table. This is also considered a comfortable impression, as it reflects many real-world conferencing experiences.

[0048] Furthermore, by using this concept, a sufficient distance between the observer position and the depicted image playback area or image playback space is ensured, resulting in a good user impression.

[0049] In a preferred embodiment, the videoconferencing device is configured to adjust the observer gaze direction in response to the observer position information so as to direct the observer gaze direction toward a video playback area facing the variable observation point or toward a video playback space facing the variable observation point. Thus, a field of view on one or more opposing video playback spaces or video playback areas is provided, thereby enabling a good view of one or more other videoconference participants while keeping user control of the actual observer position and observer gaze direction reasonably simple. In particular, the user does not need to separately adjust the observer gaze direction, but instead is provided with an automatically adjusted advantageous observer gaze direction.

[0050] In a preferred embodiment, the video conferencing device is configured to render audio signals (e.g., audio signals) from one or more video conference participants according to the arrangement of video playback areas or video playback spaces (e.g., video tiles) associated with the video conference participants in a spatial (e.g., three-dimensional) virtual reality environment (e.g., to render the audio signals as originating from respective positions in the video playback areas or video playback spaces).

[0051] Thus, the user has a good auditory impression, and the origin of the voices of the various video conference participants can correspond to, for example, the direction from which the video conference participants are looking. In other words, the voices of the various video conference participants can be depicted as originating from, for example, the location where the video playback area or video playback space associated with the video conference participants is located. As a result, a consistent perception of the speech of the other video conference participants and the video of the various video conference participants is provided.

[0052] Another embodiment according to the present invention provides a videoconferencing method comprising playing back video of a plurality of videoconference participants in a plurality of video playback areas or video playback spaces arranged in a spatial (three-dimensional) virtual reality environment.

[0053] This embodiment of the present invention is based on the same considerations as the videoconferencing device described above. A good user experience can be achieved by playing back videos of multiple videoconference participants in multiple video playback areas arranged in a spatial (three-dimensional) virtual reality environment, or in a video playback space arranged in a spatial (three-dimensional) virtual reality environment. Furthermore, it should be noted that the videoconferencing method may be optionally supplemented with any of the features, functions, and details disclosed herein, individually and in combination.

[0054] Another embodiment according to the present invention provides a computer program for performing a videoconferencing method when the computer program is executed on a computer. The computer program is based on the same considerations as the videoconferencing device and the videoconferencing method described above. It should be noted that the computer program may be optionally supplemented with any of the features, functions and details disclosed herein, individually and in combination.

[0055] Embodiments according to the present invention will now be described with reference to the accompanying drawings. [Brief explanation of the drawings]

[0056] [Figure 1] 1 shows a block schematic diagram of a videoconferencing device according to an embodiment of the present invention; [Figure 2] 1 shows a schematic diagram of a spatial virtual reality environment that can be used in an embodiment according to the present invention; [Figure 3] 1 shows a schematic diagram of a spatial virtual reality environment that can be used in an embodiment according to the present invention; [Figure 4a] 1 shows a schematic diagram of a spatial virtual reality environment that can be used in an embodiment according to the present invention; [Figure 4b] 1 shows a schematic diagram of a spatial virtual reality environment that can be used in an embodiment according to the present invention; [Figure 5] 1 shows a schematic diagram of a conventional video conferencing application. [Figure 6] 1 shows a schematic diagram of another conventional video conferencing application. DETAILED DESCRIPTION OF THE INVENTION

[0057] 1. Video conference device according to Fig. 1 FIG. 1 shows a schematic diagram of a videoconferencing device 100 according to one embodiment of the present invention.

[0058] The video conference device 100 can receive, for example, video information 110a, 110b of multiple video conference participants and provide a composite image 112 that is rendered based on the video information 110a, 110b.

[0059] The videoconferencing device 100 is configured to, for example, play back video of multiple videoconference participants described by video information 110a, 110b in multiple video playback areas or multiple video playback spaces 120a, 120b arranged in the spatial (e.g., three-dimensional) virtual reality environment 120. For example, the videoconferencing device 100 may include (e.g., in stored form) a description or definition of the spatial virtual reality environment 120. This definition of the spatial virtual reality environment may include, for example, definitions of the locations of the video playback areas or video playback spaces 120a, 120b.

[0060] Videoconferencing device 100 may include a rendering 140 configured to render spatial (e.g., three-dimensional) virtual reality environment 120 having video playback areas or video playback spaces 120a, 120b (e.g., from the perspective of a virtual camera at virtual camera position 122) into an “overall” or overall image 112 that may be intended for, e.g., a particular videoconference participant (or an individual videoconference participant or a single videoconference participant). For example, rendering 140 may receive information defining the spatial virtual reality environment, information describing the position of virtual camera 122, and information describing the virtual camera's viewing direction (the information describing the virtual camera's position may be considered an observer position, and the virtual camera's viewing direction may be considered an observer viewing direction).

[0061] Thus, rendering 140 may generate an image (e.g., overall image 112) of a virtual reality environment described (or defined) by appropriate information when viewed from the perspective of virtual camera 122. For example, rendering 140 may use video information 110a, 110b of various video conference participants as a texture for their associated video playback areas or as a texture for their associated video playback spaces. In other words, video information 110a of a first conference participant may be displayed (or rendered) at a position in first video playback area 120a, taking into account the orientation (or spatial alignment) of the first video playback area. Similarly, video information 110b of a second video conference participant may be displayed (or rendered) at a position in second video playback area 120b, taking into account the spatial orientation (or alignment) of second video playback area 120b.

[0062] In other words, different spatial orientations of different video playback areas 120a, 120b may be taken into account, including, for example, consideration of viewpoint transformations of video playback areas 120a, 120b and, as a result, video content rendered on video playback areas 120a, 120b. Furthermore, additional three-dimensional rendering effects, such as changes in size depending on the distance from the virtual camera, occlusion effects, or shadow effects, may also be taken into account by rendering 140. Thus, rendering 140 may provide overall image information 112, for example, based on a description (or definition) of a spatial virtual reality environment, based on information describing the position and viewing direction of the virtual camera (or observer), and also taking into account the video information 110a, 110b of various video conference participants. As a result, rendering 140 can provide an overall image in which the video information of various video conference participants is displayed on spatially arranged video playback carriers (or in a spatially arranged video playback space), and the position and / or viewing direction of the virtual camera (or observer) can be adjusted.

[0063] Thus, the overall image typically reflects a view of the spatial virtual reality environment from a viewpoint that may be predefined in some embodiments but may be adjustable under user control in other embodiments. The overall image can be rendered to create the impression that the (other) videoconference participants, whose images are displayed in video playback areas 120a, 120b, appear to be positioned in a three-dimensional (spatial) scene, e.g., at various orientations. For example, the overall image can provide the user with the impression that the other videoconference participants are sitting (or standing) in various non-parallel positions, e.g., the impression of a conference table (e.g., a round table) around which the other videoconference participants are positioned. Thus, a comfortable user experience can be provided for users of videoconferencing devices who may, for example, adjust the position and / or viewing direction (i.e., the observation position or viewer viewing direction) of the virtual camera.

[0064] The (optional) user interface 150 may receive the (optional) user input 152 and adjust the observer position and / or observer gaze direction 122a (corresponding to the position and / or orientation of the virtual camera 122) according to the user input. Various types of user input may be received and processed by the user interface, for example to derive the observer position and / or observer gaze direction and / or observer magnification factor from the user input.

[0065] Generally speaking, the user interface 150 may be adapted to provide a good view of one or more other videoconference participants (whose images are displayed in the video playback area and / or video playback space) and to facilitate as much as possible the user's selection of an appropriate viewing position and / or viewing gaze direction that meets the user's expectations. For example, the user interface may receive observer position information as user input, which may be one-dimensional observer position information describing the observer position, for example, along a line or along a closed contour. However, the observer position information may also be two- or three-dimensional information that defines the observer position within the spatial virtual reality environment using two or three (e.g., 3D adjustable) coordinates. Furthermore, the observer position information may alternatively be limited to a set of discrete values ​​specifying the observer position, for example, from among a predefined set of distinct observer positions that may be spaced apart until recognized as distinct observer positions.

[0066] Alternatively or additionally, observer gaze direction information may be received from a user via a user interface and may define the observer gaze direction (in other words, the gaze direction of virtual camera 122).

[0067] In some embodiments, both observer position information and observer gaze direction information may be received via the user interface to enable independent adjustment of the observer position and observer gaze direction. However, in other embodiments, only one of the observer position information and the observer gaze direction information may be received from the user via the user interface, and the user interface 150 may, for example, derive the user observer position information from the observer gaze direction information, or vice versa. Thus, a user's specification of the observer gaze direction may result in the user interface determining the observer position information, or a user's definition of the observer position information may result in the user interface determining the observer gaze direction. Thus, the user need only set one of the two parameters (observer position, observer gaze direction), and the user interface 150 may automatically select the other parameter. This is particularly advantageous, for example, when the user position information is limited to a one-dimensional (scalar) value specifying, for example, a position along a line, a curve, or a (closed) contour. Alternatively, if the observer gaze direction information entered by the user is limited to a single scalar value defining, for example, an azimuth angle (while using a fixed elevation angle), it may be advantageous to derive observer position information from the observer gaze direction information.

[0068] It should be noted that such one-dimensional (scalar) values ​​can be entered in a very simple manner via the user interface, and the use of one-dimensional (scalar) values ​​avoids the risk of overburdening the user, who should primarily focus on the discussion within the conference and not on the technical control of the videoconferencing equipment.

[0069] Alternatively or additionally, the user interface may receive scaling information from the user describing whether the user wants to enlarge the image of a particular other videoconference participant, or whether the user wants to zoom out, e.g., to view a wider field of view of the spatial virtual reality environment and / or to view the images of multiple videoconference participants.

[0070] Furthermore, it should be noted that the observer position information may be input in the form of, for example, a "viewpoint parameter," and the observer gaze direction information may be input in the form of, for example, an "angle parameter."

[0071] In conclusion, the observer position input via the user interface, and / or the observer gaze direction input via the user interface, and / or the scaling information input via the user interface may be used by the rendering 140 to provide a "total image" 112 from a viewpoint defined by the observer position, the observer gaze direction, and the (optional) scaling information.

[0072] Further (optional) details are provided below. Additionally, videoconferencing device 100 optionally includes audio drawing 160. Audio drawing 160 may, for example, draw audio signals associated with videoconference participants. For example, audio drawing 160 may perform audio drawing based on an assumption that each audio signal originates from a location where a video associated with the videoconference participant is displayed within spatial virtual reality environment 120. Furthermore, audio drawing 160 may, for example, use information regarding an observer position and / or an observer gaze direction (e.g., of the videoconference participant whose overall image and audio content are being drawn). Thus, audio drawing 160 may, for example, draw audio signals associated with other videoconference participants based on the relative positions of the videoconference participant whose overall image and audio signals are being drawn and the other videoconference participants.

[0073] Thus, audio depiction 160 may, for example, depict the audio signals of other video conference participants so that they appear to originate from the location where the video of the other video conference participants is being displayed, and thus audio depiction 160 may serve to match the visual perception of the other video conference participants with the auditory perception of what the other video conference participants are saying.

[0074] Thus, the videoconferencing device 160 may optionally provide rendered audio information 162, which may include one or more channel signals.

[0075] In conclusion, since images of other video conference participants are shown in a three-dimensional scene (spatial virtual reality environment), the video conferencing device 100 provides a significantly improved user experience when participating in a video conference, making the perception of the video conference more realistic and helping video conference participants maintain their concentration.

[0076] 2. Details of the spatial virtual reality environment and the adjustment of observer position, observer line of sight, and zoom information In the following, some optional details regarding the spatial virtual reality environment and the adjustment of the observer position, the observer gaze direction, and the scaling information are described. It should be noted that the details described below may be optionally incorporated into any of the embodiments disclosed herein.

[0077] 2 shows a schematic diagram of a (spatial) virtual reality environment 200. The virtual reality environment 200 includes, for example, a "round table" situation.

[0078] Seating positions are arranged around a circular table. In other words, in spatial virtual reality environment 200, video playback areas and / or video playback spaces may be arranged at seating positions, for example, around a circular table (or around an oval table, a square table, a rectangular table, or a table having any other shape, such as a U-shape). For example, video playback areas 220a, 220b, and 220c may correspond to video playback areas 120a and 120b and may be arranged around the table (e.g., with a tolerance of + / - 20° of the maximum extension or diameter of the table) so as to be visible from a camera position, which may be located, for example, at the center of the table. In other words, video playback areas or video playback spaces 220a, 220b, and 220c are arranged along a contour that may be defined by the perimeter of table 230. Thus, the contour along which video playback areas 220a-220c are arranged may be circular, but may alternatively be oval, square, rectangular, or any other geometric shape. It should further be noted that video playback areas 220a-220c are positioned at user positions within the spatial virtual reality environment, which may be, for example, seating positions around table 230. As an example, if the number of video conference participants is less than the number of chairs around table 230, empty chairs may also be defined by the spatial virtual reality environment. Alternatively, the number of user positions and / or table size may be adapted (e.g., automatically) to the number of video conference participants.

[0079] 2, the observer position (or virtual camera position) 240 may be fixed, for example, at the center of the table 230 or around the center of the table 230. Therefore, it may not be necessary for the user to define the observer position (or virtual camera position), but rather it may be sufficient to define the observer viewing direction 240a (or virtual camera viewing direction). Using such a concept, participants in a video conference, to which a comprehensive image is drawn, may be able to see the images of all video conference participants equally well.

[0080] Figure 3 shows a schematic diagram of another spatial virtual reality environment 300 that may be used in embodiments disclosed herein. Similar to the spatial virtual reality environment 200 according to Figure 2, a video playback area (not shown in Figure 3) may be positioned, for example, at a seating position (i.e., a user position) along the edge of a table (e.g., a round table) 330. The placement of the video playback area in the spatial virtual reality environment 300 of Figure 3 may be the same as the placement of the video playback area in the spatial virtual reality environment 200 of Figure 2, for example.

[0081] However, the movement of the observation point (or virtual camera) may be different. For example, the virtual camera (or observation point) 340 may be movable along a line, a contour, or a circle (or an oval, square, or rectangle) 342, for example, according to user input provided by a user interface. For example, a first position of the virtual camera is indicated by reference numeral 340a, a second position of the virtual camera 340 is indicated by reference numeral 340b, and a third position of the virtual camera 340 is indicated by reference numeral 340c. The movement between the different positions 340a, 340b, 340c may be gradual, for example, in relatively large steps that cause a significant change in the rendered overall image, or may be quasi-continuous (providing a smooth transition of the rendered overall image). As can be seen, the virtual camera may move, for example, along a line (or closed contour) 350 (for example, along the edge of the console 330) along which the video playback area may be located. Furthermore, the viewing direction 342a, 342b, 342c of the virtual camera 340 may be automatically adjusted, for example, to point towards the center 344 of the table or the center 344 of the contour 350 along which the virtual camera moves. For example, the viewing direction of the virtual camera 340 may be adapted by the user interface, for example, so that the viewing direction of the virtual camera (corresponding to the observer viewing direction) is perpendicular to the direction along which the variable observation point moves within a tolerance of + / - 10 degrees, or so that the viewing direction of the virtual camera is perpendicular to the curve 350 along which the virtual camera 340 (or variable observation point) moves within a tolerance of + / - 10 degrees. The variable observation point (i.e., the location of the virtual camera) may be adjusted by the user interface 150 in response to observer position information, and the observer viewing direction (i.e., the viewing direction of the virtual camera) may be adjusted in response to observer position information, for example.

[0082] Therefore, the position and viewing direction of the virtual camera 340 may be controlled, for example, by either angular information (from which both the virtual camera position and the virtual camera viewing direction may be derived by a user interface) or linear position information (from which both the virtual camera position and the virtual camera viewing direction may be derived by a user interface). As a result, very simple control of the virtual camera position and the virtual camera viewing direction is provided. In such an embodiment, it becomes possible in a very simple manner to always orient the observer viewing direction inside the curve 350 (e.g., a closed curve or contour) along which the variable observation point moves in response to the observer position information. Alternatively, it is also possible in a very simple manner to ensure that the viewing direction is always oriented toward the center of the curve along which the variable observation point moves in response to the observer position information. Therefore, the observer viewing direction may be, for example, directed toward the video playback area opposite the variable observation point, or toward the video playback space opposite the variable observation point. As a result, a good user experience can be achieved.

[0083] FIG. 4a shows a schematic diagram of a spatial virtual reality environment that can be used in embodiments according to the present invention. As can be seen in FIG. 4, multiple video playback spaces 420a, 420b, 420c, and 420d used to display the videos of different video conference participants are arranged along a closed contour 410, which may be circular or elliptical in shape. It is clear that the surfaces of the video playback areas 420a through 420d are arranged at different orientations in the spatial virtual reality environment 400. As a result, perspective effects are taken into account when depicting the spatial virtual reality environment as seen from the point of a virtual camera having an associated virtual camera gaze direction. Thus, the videos of different video conference participants may be distorted depending on the relationship between the video playback areas and the respective positions and orientations of the virtual cameras.

[0084] 4b shows a schematic diagram of another spatial virtual reality environment 450 that can be used in embodiments according to the present invention. As can be seen, video playback areas 470a-470h are arranged along a square or rectangle 460 that may correspond to the perimeter of a square or rectangular table. The virtual camera position may be variable, for example, within the area in which the video playback areas are arranged. Thus, spatial virtual reality environment 450 may correspond, for example, to a real-life conference situation in which conference participants are seated along the perimeter of a table.

[0085] However, it should be noted that different spatial virtual reality environments are of course possible, and the video playback areas may be arranged in various ways, for example irregularly or in ways that may cause occlusions.

[0086] In conclusion, the spatial virtual reality environments 200, 300, 400, 450 can all be used in embodiments of the present invention to help achieve a good user experience.

[0087] 3. Further Aspects and Embodiments According to one aspect of the present invention, an arrangement (or device) is proposed in which 360-degree virtual reality technology is combined with a videoconferencing stream.

[0088] According to one embodiment, this is particularly interesting for so-called roundtable conferences. For this purpose, a virtual camera (e.g., virtual camera 240) is placed as a viewpoint in the center of a roundtable (e.g., roundtable 230). Each participant (e.g., a videoconference participant, for whom an overall image is depicted) can then choose their own viewing direction, as in 360-degree VR applications (see, for example, https: / / ibc2020.digitalmedia.fraunhofer.de). Using a mouse (e.g., a computer mouse), they can move left and right, zoom in and out, and thereby direct their attention (e.g., visual attention) to the respective conversation partner. Participants are shown (or superimposed) at one seat position (or at each seat position). The background can be designed as a virtual reality room.

[0089] An example is shown in Figure 3. A further development (or alternative) could be to position the camera on a circular path or a curved path of different shapes (e.g. on a circle 350) according to the 360-degree viewing direction in the virtual reality room. The observer may then, for example, "drive" (e.g. control the position using a user interface) along the curved path. This has the advantage that the camera can take a "natural" position on the other side of the table. The viewing angle (or viewpoint) and focal length are, for example, adapted (e.g. automatically by the user interface) to the natural perception.

[0090] Several aspects of the present invention will be described below. One embodiment in accordance with the present invention provides for the combination of a 360-degree virtual reality presentation with video tiles (eg, video playback areas) from a videoconferencing system.

[0091] According to one embodiment, participants can adjust viewpoint parameters (such as rotation angle and / or magnification factor) themselves.

[0092] According to another aspect, in an extended variant, the virtual camera is in different virtual positions depending on the rotation angle.

[0093] Implementation Alternatives While some aspects have been described in terms of apparatus, it will be apparent that these aspects also represent descriptions of corresponding methods, with blocks or apparatus corresponding to method steps or features of method steps. Similarly, aspects described in terms of method steps also represent descriptions of corresponding blocks, components, or features of corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0094] The encoded audio signals of the present invention can be stored on a digital storage medium or can be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0095] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM or flash memory, on which electronically readable control signals are stored, which cooperates (or can cooperate) with a programmable computer system to execute the respective methods. Thus, the digital storage medium may be computer-readable.

[0096] Some embodiments according to the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system to perform one of the methods described herein.

[0097] Generally, embodiments of the present invention can be implemented as a computer program product having program code that operates to perform one of the methods when the computer program product is run on a computer, and the program code may be stored on, for example, a machine-readable carrier.

[0098] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0099] In other words, an embodiment of the inventive methods is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0100] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transitory.

[0101] A further embodiment of the inventive method is, therefore, a data stream or sequence of signals representing the computer program for performing one of the methods described herein, which data stream or sequence of signals may for example be adapted to be transmitted via a data communications connection, for example via the Internet.

[0102] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0103] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0104] Further embodiments according to the invention comprise an apparatus or system configured to transmit (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may for example be a computer, a portable device, a memory device, etc. The apparatus or system may for example comprise a file server for transmitting the computer program to the receiver.

[0105] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0106] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0107] The devices described herein, or any components of the devices described herein, may be implemented at least in part in hardware and / or software.

[0108] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0109] Any of the methods described herein, or any of the components of the apparatus described herein, may be implemented at least in part by hardware and / or software.

[0110] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore the intention to be limited only by the scope of the appended claims and not by the specific details presented by way of description and illustration of the embodiments herein.

Claims

1. A video conference device (100), the video conferencing device is configured to play back videos (110a, 110b) of a plurality of video conference participants in a plurality of video playback areas (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or video playback spaces arranged in a spatial virtual reality environment (120, 200, 300, 400, 450); The video conferencing device is configured to render the spatial virtual reality environment (120, 200, 300, 400, 450) having the video playback area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or video playback space into an overview image (112) for a single video conference participant.

2. The orientations of the plurality of video playback areas (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or video playback spaces are different. The videoconferencing device (100) of claim 1.

3. the videoconferencing device adjusts an observer position in response to a user input; The spatial virtual reality environment (120, 200, 300, 400, 450) having the image reproduction area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or the image reproduction space is configured to be rendered according to the observer position. A videoconferencing device (100) according to any one of claims 1 to 2.

4. the video conference device adjusts the viewer's line of sight in response to a user input; The spatial virtual reality environment (100, 200, 300, 400, 450) having the image reproduction area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or the image reproduction space is configured to be rendered according to the viewer's line of sight. A videoconferencing device (100) according to any one of claims 1 to 3.

5. configured to selectively render one or several video playback areas (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or one or several video playback spaces respectively associated with other users in response to user input; A videoconferencing device (100) according to any one of claims 1 to 4.

6. The video playback area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or the video playback space is configured to be arranged along a contour (350, 410, 460). A videoconferencing device (100) according to any one of claims 1 to 5.

7. The video playback area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or video playback space is configured to be arranged along a circle (350, 410), an ellipse, a square, or a rectangle (460). A videoconferencing device (100) according to any one of claims 1 to 6.

8. configured to position the video playback area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or video playback space at a user's location within a spatial virtual reality environment (120, 200, 300, 400, 450); A videoconferencing device (100) according to any one of claims 1 to 7.

9. a virtual reality system configured to acquire one or more observation parameters (152) and render the spatial virtual reality environment (120, 200, 300, 400, 450) having the video playback area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or video playback space according to the one or more observation parameters; A videoconferencing device (100) according to any one of claims 1 to 8.

10. the observation parameters (152) include a rotation angle and / or a magnification factor; A videoconferencing device (100) according to any one of claims 1 to 9.

11. configured to render the spatial virtual reality environment (120, 200, 300, 400, 450) having the video playback area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or video playback space as viewed from a predetermined observation point (240) using an adjustable viewing direction and / or an adjustable scaling factor; A videoconferencing device (100) according to any one of claims 1 to 10.

12. The spatial virtual reality environment (120, 200, 300, 400, 450) is configured to render a video playback area (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or the playback space as viewed from a variable observation point (340a, 340b, 340c), A videoconferencing device (100) according to any one of claims 1 to 10.

13. configured to adjust the variable observation points (340a, 340b, 340c) in response to observation information or angle information; The videoconferencing device (100) of claim 12.

14. the videoconferencing device is configured to adjust the variable observation points (340a, 340b, 340c) in response to observation or angle information, the variable observation points (340a, 340b, 340c) being selectable or displaceable by the user along a curve (350, 410, 460) within the virtual reality environment (100, 30, 400, 450); The videoconferencing device (100) of claim 13.

15. configured to adjust the variable observation points (340a, 340b, 340c) in response to observer position information; and adjusting the viewer's line of sight (342a, 342b, 342c) in accordance with the viewer position information. A videoconferencing device (100) according to any one of claims 12 to 14.

16. the video conference device adjusts the viewer's line of sight (342a, 342b, 342c) in accordance with the viewer position information; the observer line of sight (342a, 342b, 342c) is perpendicular to the direction in which the variable observation point moves with a tolerance of + / - 10 degrees; or the observer gaze direction (342a, 342b, 342c) is perpendicular to the curve (350, 410, 460) along which the variable observation point moves, with a tolerance of + / - 10 degrees; The videoconferencing device (100) of claim 15.

17. the video conference device adjusts the viewer's line of sight (342a, 342b, 342c) in accordance with the viewer position information; The viewer's line of sight (342a, 342b, 342c) always faces the inside of a curve (350, 410, 460) along which the variable observation point (342a, 342b, 342c) moves in response to the viewer position information, or The observer's line of sight (342a, 342b, 342c) is always directed toward the center (344) of a curve (350, 410, 460) along which the variable observation point (342a, 342b, 342c) moves in response to the observer position information. Videoconferencing device (100) according to claim 15 or 16.

18. the video conference device is configured to adjust the viewer's line of sight (342a, 342b, 342c) in accordance with the viewer position information; the observer's line of sight (342a, 342b, 342c) is directed toward an image reproduction area (120a, 120b, 420a-420d, 470a-470h) on the opposite side of the variable observation point (340a, 340b, 340c), or toward an image reproduction space on the opposite side of the variable observation point; 18. Videoconferencing device (100) according to claim 15, 16 or 17.

19. configured to render audio signals from one or several video conference participants according to the arrangement of the video playback areas (120a, 120b, 220a-220c, 420a-420d, 470a-470h) or video playback spaces associated with the video conference participants within the spatial virtual reality environment (120, 200, 300, 400, 450), A videoconferencing device (100) according to any one of claims 1 to 18.

20. and playing back video of a plurality of video conference participants in a plurality of video playback areas or video playback spaces arranged in a spatial virtual reality environment. Videoconferencing methods.

21. A computing program for carrying out the method of claim 20 when the program is run on a computer.

Images