Systems, methods, and programs for implementing a personal camera that adapts to both local and remote surroundings

Abstract

To provide a system, a method, and a program for implementing a smart camera compatible with both co-located and remote surroundings.SOLUTION: The computerized system includes a processing unit and a memory. The system works in conjunction with a real-time video conferencing stream containing a user video. The memory receives a real-time video conferencing stream with a user video to the computerized system. The system embodies a computer executable command set for performing a method including the steps of: detecting and isolating a background in the received real-time video conference stream from the user; and replacing the isolated background with a background received from a second user's system or a pre-recorded background.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The disclosed embodiments relate generally to smart camera systems, and more particularly to systems, methods, and programs that implement a personal "chameleon" smart camera that adapts to both local and remote surroundings. [Background technology]

[0002] As will be appreciated by those skilled in the art, despite recent work on smart spaces, today much knowledge work is performed outside of traditional office environments, for example in places like the home, in cafes / restaurants (see FIG. 1d), while traveling on trains (see FIGS. 1a and 1c) or buses (FIG. 1b), and in cubicles in open offices.

[0003] As will be further appreciated by those skilled in the art, conducting web conferences in these public spaces is problematic: video backgrounds and reflective areas of the video (e.g., eyeglasses) may contain private and / or confidential information that may be inadvertently disclosed to unauthorized persons, and audio channels may suddenly become cluttered with undesirable ambient voices, sounds, and noises, such as a barista in a restaurant calling a person's name or the loud rumble of a passing train. Furthermore, past studies have shown that the lack of shared visual context in video conferences can undermine participants' conversational foundations and degrade the user experience.

[0004] In view of the above and other shortcomings of the prior art, therefore, there is a need for new and improved systems and methods for implementing smart cameras that are compatible with both co-located and remote environments. Such inventive cameras have many applications, including video conferencing where participants are outside of a traditional office environment. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] U.S. Patent No. 9,313,454 [Patent Document 2] U.S. Patent No. 7,783,075 [Patent Document 3] U.S. Patent No. 5,812,787 [Non-patent literature]

[0006] [Non-Patent Document 1] FAN, Q., et al., A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing, ICCV'17, 2017, 30 pgs. [Non-Patent Document 2] FUSSEL, SR, et al., Coordination of Communication: Effects of Shared Visual Context on Collaborative Work, CSCW'00 Proceedings of the 2000 ACMConference on Computer Supported Cooperative Work, 2000, pp. 21-30. [Non-Patent Document 3] ISOLA, P., et al., Image-to-Image Translation with Conditional Adversarial Networks, 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017, 17 pgs. [Non-Patent Document 4] KAUFF, P., et al., An immersive 3D video-conferencing system using shared virtual team user environments, CVE702, September 30-October 02, 2002, Bonn, Germany, 9 pgs. [Non-Patent Document 5] RADFORD, A., et al., Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks, 2015, arXiv:1511.06434, 16 pgs. [Non-Patent Document 6] SONG, S., et al., Im2Pano3D: Extrapolating 360 Structure and Semantics Beyond the Field of View, IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2017, 18 pgs. [Non-Patent Document 7] github.com, justadudewhohacks / face-api, 15 pgs., retrieved 12 / 3 / 19, URL:https: / / github.com / justadudewhohacks / face-api.js [Non-Patent Document 8] github.com, jeelizFaceFilter, 14 pgs., retrieved 12 / 3 / 19, URL:https: / / github.com / jeeliz / jeelizFaceFilter Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention aims to provide a system, method and program for implementing a smart camera that is compatible with both local and remote surroundings. [Means for solving the problem]

[0008] According to one aspect of the embodiments described herein, there is provided a computerized system incorporating a processing unit and a memory, the system operative in association with a real-time videoconferencing stream comprising video of a user, the memory embodying a set of computer-executable instructions that cause the computerized system to perform a method comprising receiving the real-time videoconferencing stream comprising video of the user, detecting and separating a background in the received real-time videoconferencing stream from the user, and replacing the separated background with a background received from a system of a second user or a pre-recorded background.

[0009] In one or more embodiments, the background is a pre-recorded background retrieved from a database.

[0010] In one or more embodiments, the background is generated automatically.

[0011] In one or more embodiments, the background is recorded by the second user in response to a prompt by the second user's system.

[0012] In one or more embodiments, detecting and isolating the background in the real-time videoconferencing stream includes detecting a user's face.

[0013] In one or more embodiments, the user's face is detected by identifying feature points in video frames of a real-time videoconferencing stream.

[0014] In one or more embodiments, the method further includes learning the user's face.

[0015] In one or more embodiments, the user's face is learned at the start of the video conference.

[0016] In one or more embodiments, the user's face is learned prior to the video conference.

[0017] In one or more embodiments, isolating the background in the real-time videoconferencing stream includes identifying a user based on a learned face of the user and separating the background from the identified user.

[0018] According to another aspect of the embodiments described herein, there is provided a computerized system incorporating a processing unit and a memory, the system operative in association with a real-time videoconferencing stream comprising a video of a user, the memory embodying a set of computer executable instructions that cause the computerized system to perform a method including receiving the real-time videoconferencing stream comprising a video of a user, finding and isolating reflections in the received real-time videoconferencing stream, and modifying the isolated reflections in the received real-time videoconferencing stream.

[0019] In one or more embodiments, modifying the isolated reflection includes darkening the isolated reflection.

[0020] In one or more embodiments, the isolated reflection is the user's eyeglasses.

[0021] In one or more embodiments, modifying the isolated reflection includes replacing the isolated reflection with a new reflection using the second user's video conference stream.

[0022] According to another aspect of the embodiments described herein, there is provided a computer-implemented method executed in association with a computerized system incorporating a processing unit and a memory, the system operating in association with a real-time videoconferencing stream comprising video of a user, the method including receiving the real-time videoconferencing stream comprising video of the user, detecting and separating a background in the received real-time videoconferencing stream from the user, and replacing the separated background with a background received from a system of a second user or a pre-recorded background.

[0023] In one or more embodiments, the background is a pre-recorded background retrieved from a database.

[0024] In one or more embodiments, the background is generated automatically.

[0025] In one or more embodiments, the background is recorded by the second user in response to a prompt by the second user's system.

[0026] In one or more embodiments, detecting and isolating the background in the real-time videoconferencing stream includes detecting a user's face.

[0027] In one or more embodiments, the user's face is detected by identifying feature points in video frames of a real-time videoconferencing stream.

[0028] In one or more embodiments, the method further includes learning the user's face.

[0029] In one or more embodiments, the user's face is learned at the start of the video conference.

[0030] In one or more embodiments, the user's face is learned prior to the video conference.

[0031] In one or more embodiments, separating the background in the real-time videoconferencing stream includes identifying a user based on a learned face of the user and separating the background from the identified user.

[0032] Additional aspects related to the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the invention. Aspects of the present invention may be realized and attained by means of the elements and combinations of various elements and aspects particularly pointed out in the following detailed description and the appended claims.

[0033] It should be understood that both the foregoing and the following descriptions are exemplary and explanatory only and are not intended to limit the claimed invention or its application in any manner. [Brief description of the drawings]

[0034] [Figure 1a] This shows an example of knowledge work in nature (on a train). [Figure 1b] This shows an example of knowledge work in nature (bus). [Figure 1c] This shows an example of knowledge work in nature (on a train). [Figure 1d] This shows an example of knowledge work in nature (cafe). [Diagram 2] 1 illustrates an exemplary operating sequence of a "Chameleon" smart camera embodiment of the present invention that adapts to both local and remote surroundings. [Figure 3a] 13 shows exemplary image processing results from a "Chameleon" smart camera embodiment of the present invention that adapts to both local and remote surroundings. [Figure 3b] 13 shows exemplary image processing results from a "Chameleon" smart camera embodiment of the present invention that adapts to both local and remote surroundings. [Figure 3c] 13 shows exemplary image processing results from a "Chameleon" smart camera embodiment of the present invention that adapts to both local and remote surroundings. [Figure 3d] 13 shows exemplary image processing results from a "Chameleon" smart camera embodiment of the present invention that adapts to both local and remote surroundings. [Figure 3e] 13 shows exemplary image processing results from a "Chameleon" smart camera embodiment of the present invention that adapts to both local and remote surroundings. [Figure 4] 1 illustrates an exemplary operational sequence of a “Chameleon” smart camera embodiment of the present invention configured to automatically mute or unmute the microphone based on the user's head pose and the user's detected lip movements. [Diagram 5] 1 illustrates an exemplary embodiment of a computer system that can be used to implement the inventive techniques described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain and illustrate the principles of the present technology.

[0036] In the following detailed description, reference is made to the accompanying drawings, in which identical functional elements are designated with like numerals. The aforementioned accompanying drawings show by way of example, and not by way of limitation, specific embodiments and implementations consistent with the principles of the present invention. These implementations have been described in sufficient detail to enable those skilled in the art to practice the invention, and it should be understood that other implementations may be utilized, and structural changes and / or substitutions of various elements may be made without departing from the scope and spirit of the present invention. Therefore, the following detailed description should not be taken in a limiting sense. Additionally, the various embodiments of the present invention described may be implemented in the form of software running on a general purpose computer, in the form of specialized hardware, or a combination of software and hardware.

[0037] Despite recent research on smart spaces, today a great deal of knowledge work is conducted in nature (outside of traditional office environments), e.g., at home, in coffee shops, on trains, buses, airplanes, and of course in cubicles in crowded open offices. See Figures 1a, 1b, 1c, and 1d. Conducting web conferencing in nature is problematic: the video background and reflective areas in the video (e.g., glasses) can feature private and / or sensitive intellectual property and objects that users do not want streamed (e.g., a barista serving coffee, moving passersby), and the audio channel can suddenly become cluttered with undesirable sounds (e.g., a barista calling your name, a passing train).

[0038] To address these and other problems of the prior art, in accordance with one aspect of the embodiments described herein, a system and method is provided that implements a personal "chameleon" camera that is adaptable to both local and remote surroundings. An exemplary operational sequence of the described personal "chameleon" camera is shown in FIG.

[0039] In one or more embodiments, the described techniques implemented in the aforementioned smart camera first learn the user's face (step 201 in FIG. 2), identify the user's face on the camera, pose it in real time (step 202 in FIG. 2), and remove the user's background in real time (step 203) and replace it with the background of the other participants (step 204 in FIG. 2), thereby enhancing the immersive conference experience for video conference participants.

[0040] One embodiment described above performs real-time correction of identified reflective areas in the video by further identifying and correcting certain reflective areas in the video, such as glasses, mirrors, etc., to obscure potentially sensitive information, and to remove glare and / or create a sense of presence of video conference participants by inserting the reflections of such participants' surroundings into the reflective areas detected in real-time in the received video stream. See step 205 of FIG. 2.

[0041] Additionally, one or more embodiments detect various facial features of the user in the live video stream, such as the user's lips. Additionally, the user's head pose is also detected. Based on such detection results, the system is configured to automatically mute and unmute the user's microphone based on the user's detected lip movements and head orientation. This is done so that the system does not register external noises and only captures sound when the user is actually speaking. As will be appreciated by those skilled in the art, such features are advantageous as they reduce distraction to other video conference participants.

[0042] In one or more embodiments, the embodiments described above further immerse all participants in the same environment, enhancing co-presence, without distracting elements, by replacing the background of a user's live video stream with a background similar to the surroundings of the other participants in the video conference call, including replacing reflective areas of the screen with a reflection of the other person's stream, either their face or what is currently being shared (e.g., a screen sharing session).

[0043] Figures 3a, 3b, 3c, 3d and 3e show exemplary video stream processing results by the "Chameleon" smart camera embodiment of the present invention adapted to both co-located and remote surroundings. Figure 3a shows the original video stream.

[0044] In one or more embodiments, the smart camera system described above detects the user's face and facial orientation in a live video stream. In one embodiment, the described system uses FaceAPI or jeelizFaceFilter face detection software, which are well known in the art. The aforementioned software system performs keypoint detection to identify the user's facial keypoints in the live video stream. Face detection and recognition are then performed using the detected keypoints.

[0045] Once the user's facial region is detected, the camera system embodiment described above separates the user's face and upper body in the video stream from the remaining areas of the video that are labeled as background. The detected face and upper body of a person separated from the background is shown in Figure 3b. In one embodiment, the system then fills in the non-person regions (video background) with background (surroundings) from other conference participants, as shown in Figure 3c.

[0046] In one or more embodiments, the system is configured to detect the face of a particular user who initiates a web conference. To this end, the system performs a face learning operation on the user who is in front of the smart camera when the video conference is initiated. In an alternative embodiment, the system may be configured to pre-learn the user's face when the system is installed or configured, as is done with modern smartphones. After the face learning procedure, the system performs a face recognition operation on the live video stream using the information acquired during the face learning phase.

[0047] In one or more embodiments, when another person enters the field of view of the smart camera described above, his or her image is also removed along with the background to avoid distracting other conference participants. For example, when a spouse or child enters the camera's view, the system automatically removes their face along with the user's background to avoid distracting other video conference participants.

[0048] As is well known in the art, video conference participants often refer to objects outside the field of view of the camera they are using, which can lead to conversation breakdowns. See S. Song, A. Zeng, AX Chang, M. Savva, S. Savarese, and T. Funkhouser. “lm2Pano3D: 360° Estimation of Structure and Semantics Beyond the Field of View,” CVPR.2018. To address this, one embodiment of the described camera leverages a 360° camera to insert participants into a more complete representation of the remote scene.

[0049] In one or more embodiments, the background of other participants in the video can be replaced with a front view of the user's environment to provide a more immersive experience. This can be done by using a separate camera (either built-in to the user's device or a separate device such as a smartphone) to capture the front view while another camera is pointed at the user for the video conference.

[0050] Because built-in front cameras are not steerable and / or laptop screens may be held at poor angles, an initial front-view video sequence can be recorded before the meeting begins. This pre-meeting front-view recording allows users facing a wall to temporarily position their front camera to see more interesting content.

[0051] In one or more embodiments, the system is further configured to utilize a 360-degree camera to help reconstruct a front view of the user's environment. In this implementation, indoor scene images can be collected by a 360-degree video camera in several different locations, such as a cafe, a restaurant, an office, etc. These images can be stored in a remote database, and a complete 360-degree panoramic view of each indoor scene can be stored and retrieved. This 360-degree panoramic view can be further processed to find the most similar part to the query image (i.e., the user's background). The content in the opposite direction of the most similar part of the panoramic view can be used as the background.

[0052] An alternative embodiment uses a deep learning-based generative model, e.g., A Radford, L. Metz, S. Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks," arXiv:1511.06434 [cs.LG] and P. Isola, J. Zhu, T. Zhou, AA Efros, "Image-to-Image Translation with Conditional Adversarial Networks," arXiv:1611.07004 [cs.CV], to generate a visually plausible front view of a user's environment given a background view in the user's video frames. Such a model can be trained on a sufficient amount of collected indoor scene images (in the form of 360-degree panoramic representations) to learn prior contextual information about the room layout and potentially indoor objects such as tables, windows, etc.

[0053] As will be appreciated by those skilled in the art, approximately 64% of American adults wear glasses. This percentage is even higher for computer-related workers. Therefore, reflections of glasses are commonly seen in live video streams during meetings supported by laptops and smartphones. The presence of such reflections in live meeting streams has several drawbacks. The first drawback is privacy. As webcam resolution improves (especially on smartphones), reflective materials can potentially be analyzed to extract intellectual property. Secondly, said reflections then result in a distraction and thus a poor user experience for other participants in the meeting. To solve both of the above problems, one embodiment of the present invention replaces the reflective area with a darker version that keeps the eyes back. See Fig. 3d and Fig. 3e. In one embodiment, the present invention is also configured to add new reflections using the streams of other participants to make them feel more connected (see each other).

[0054] In one embodiment, the camera described above is configured to automatically mute or unmute the microphone based on the user's head pose and the user's detected lip movements. Figure 4 shows an exemplary operation sequence of a "Chameleon" smart camera embodiment of the present invention configured to automatically mute or unmute the microphone based on the user's head pose and the user's detected lip movements. The system detects the user's head pose and lip movements in step 401.

[0055] In one embodiment, if the user's face is straight (neither left nor right) and the user's lips are moving, the microphone is unmuted (unless forcibly muted by the user). See step 402. If the user's head is facing in any direction other than a straight posture (within a predefined margin or threshold), the microphone is muted again, thus accommodating the fact that the user may be speaking to someone else (e.g., a barista approaching and engaging the user). See step 404. Similarly, if the user's lips stop moving, the microphone is muted as well. See step 403.

[0056] Exemplary Computer System Embodiments 5 illustrates an exemplary embodiment of a computer system 500 that may be used to implement the techniques described herein. In one or more embodiments, computer 500 may be implemented within a mobile computing device form factor known to those skilled in the art. In alternative embodiments, computer 500 may be implemented based on a laptop or notebook computer. In yet another embodiment, computer 500 may be a specialized computing system.

[0057] The computer 500 may include a data bus 504 or other interconnection or communication mechanism for communicating information across and between various hardware components of the computer 500, and a central processing unit (CPU or simply processor) 501 coupled to the data bus 504 for processing information and performing other computational and control tasks. The computer 500 also includes a memory 512, such as a random access memory (RAM) or other dynamic storage device, coupled to the data bus 504 for storing various information as well as instructions executed by the processor 501. The memory 512 may also include a persistent storage device, such as a magnetic disk, optical disk, solid-state flash memory device, or other non-volatile solid-state storage device.

[0058] In one or more embodiments, memory 512 may be used for storing temporary variables or other intermediate information during execution of instructions by processor 501. Optionally, computer 500 may further include a read only memory (ROM or EPROM) 502 or other static storage device coupled to data bus 504 for storing static information and instructions for processor 501, such as firmware required for the operation of computer 500, a basic input / output system (BIOS), and various configuration parameters of computer 500.

[0059] In one or more embodiments, the computer 500 may further incorporate a camera 510 for capturing still images and real-time video. Additionally, the computer 500 may incorporate a microphone 511 for picking up audio and a microphone control module 509 for automatically muting and unmuting the microphone 511 based on the user's head pose.

[0060] In one or more embodiments, the computer 500 may further include a communication interface, such as a network interface 505 coupled to the data bus 504. The network interface 505 may be configured to establish a connection between the computer 500 and the Internet 524 using at least one of a WIFI interface 507 and a cellular network (GSM or CDMA) adapter 508. The network interface 505 may be configured to provide two-way data communication between the computer 500 and the Internet 524. The WIFI interface 507 may operate in accordance with 802.11a, 802.11b, 802.11g, and / or 802.11n protocols, as well as Bluetooth protocols, known to those skilled in the art. In the illustrated implementation, the WIFI interface 507 and the cellular network (GSM or CDMA) adapter 508 send and receive electrical or electromagnetic signals that carry digital data streams representing various types of information.

[0061] In one or more embodiments, the Internet 524 typically provides data communication to other network resources via one or more sub-networks. Thus, the computer 500 can access various network resources anywhere on the Internet 524, such as remote media servers, web servers, other content servers, and other network data storage resources. In one or more embodiments, the computer 500 is configured via the network interface 505 to send and receive messages, media, and other data, including application program code, over the various networks that comprise the Internet 524. In the Internet example, when the computer 500 functions as a network client, it can request code or data for an application program running on the computer 500. Similarly, various data or computer code can be sent to other network resources.

[0062] In one or more embodiments, the functions described herein are implemented by the computer 500 in response to the processor 501 executing one or more sequences of one or more instructions contained in the memory 512. Such instructions may be read into the memory 512 from another computer-readable medium. Execution of the sequences of instructions contained in the memory 512 causes the processor 501 to perform the various process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement embodiments of the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware circuitry and software.

[0063] The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to the processor 501 for execution. The computer-readable medium is just one example of a machine-readable medium, which may carry instructions for implementing any of the methods and / or techniques described herein. Such media may take many forms, including but not limited to non-volatile media and volatile media.

[0064] Common forms of non-transitory computer readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes or other magnetic media, CD-ROMs, other optical media, punch cards, paper tapes, other physical media with patterns of holes, RAM, PROM, EPROM, Flash EPROM, flash drives, memory cards, other memory chips or cartridges, or other media from which a computer can read. Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to the processor 501 for execution. For example, the instructions may initially be carried on a magnetic disk from a remote computer. Alternatively, the remote computer may load the instructions into its dynamic memory and transmit the instructions over the Internet 524. In particular, computer instructions may be downloaded into the memory 512 of the computer 500 from said remote computer over the Internet 524 using a variety of network data communication protocols well known in the art.

[0065] In one or more embodiments, the memory 512 of the computer 500 may store any of the following software programs, applications, and / or modules:

[0066] 1. An operating system (OS) 513, which may be a mobile operating system for implementing basic system services and managing various hardware components of the computer 500. Exemplary embodiments of the operating system 513 are well known to those skilled in the art and may comprise any now known or later developed mobile operating system. A network communications module 514 may further be provided for enabling network communications using the network interface 505.

[0067] 2. The software modules 515 may comprise, for example, a set of software modules executed by the processor 501 of the computer 500 to cause the computer 500 to perform certain functions, such as processing the camera video stream in real time using a video processing module 516 and muting or unmuting the microphone using a microphone control module 517.

[0068] 3. The data storage 518 may be used, for example, to store various parameters and thresholds 519.

[0069] And it should be understood that the processes and techniques described herein are not inherently related to any particular apparatus, but may be implemented by any suitable combination of components. Moreover, various types of general-purpose apparatus can be used in accordance with the teachings described herein. It may also prove advantageous to construct specialized apparatus to perform the method steps described herein. Although the present invention has been described with reference to specific examples, these examples are intended in all respects to be illustrative and not restrictive. Those skilled in the art will appreciate that many different combinations of hardware, software, and firmware will be suitable for practicing the present invention. For example, the software described may be implemented in a variety of programming or scripting languages, such as assembler, C / C++, Objective-C, perl, shell, PHP, Java, and other programming or scripting languages ​​now known or later developed.

[0070] Moreover, other implementations of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. Various aspects and / or components of the described embodiments may be used alone or in any combination in systems and methods implementing a personal "chameleon" smart camera that adapts to both local and remote surroundings. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the invention being indicated by the following claims.

Claims

1. A computerized system comprising a processing unit and a memory, A computerized system, the system operating in conjunction with a real-time video conferencing stream containing a user's video, the memory embodying a set of computer-executable instructions that cause the computerized system to perform a method including the steps a, b and c of: a. receiving the real-time videoconference stream including the video of a first user; b. detecting and isolating a first background in the received real-time videoconferencing stream from the first user; c) replacing the separated first background with a second background received from a second user's system;

2. The system described in claim 1, wherein the second background is a pre-recorded background retrieved from a database.

3. The system described in claim 1, wherein the second background is automatically generated.

4. The system of claim 1, wherein the second background is recorded by the second user in response to a prompt by the system of the second user.

5. The system of claim 1 , wherein detecting and isolating a background in the real-time videoconferencing stream comprises detecting a face of the user.

6. The system of claim 5 , wherein the user's face is detected by identifying a plurality of feature points in video frames of the real-time videoconferencing stream.

7. The system of claim 5 , further comprising learning a face of the user.

8. The system of claim 7 , wherein the user's face is learned at the start of a video conference.

9. The system of claim 7 , wherein the user's face is learned prior to the video conference.

10. 8. The system of claim 7, wherein isolating the background in the real-time video conferencing stream comprises: identifying the user based on a learned face of the user; and isolating the background from the identified user.

11. 1. A computer-implemented method performed in association with a computerized system comprising a processing unit and a memory, the system operating in association with a real-time video conferencing stream comprising a video of a user, the computer-implemented method comprising the steps a), b) and c) of: a. receiving the real-time videoconference stream including the video of a first user; b. detecting and isolating a first background in the received real-time videoconferencing stream from the first user; c) replacing the separated first background with a second background received from a second user's system; 12. The computer-implemented method of claim 11, wherein the second background is a pre-recorded background retrieved from a database.

13. The computer-implemented method of claim 11, wherein the second background is generated automatically.

14. The computer-implemented method of claim 11, wherein the second background is recorded by the second user in response to a prompt by the system of the second user.

15. The computer-implemented method of claim 11 , wherein detecting and isolating a background in the real-time videoconferencing stream includes detecting a face of the user.

16. The computer-implemented method of claim 11 , wherein the user's face is detected by identifying a plurality of feature points in video frames of the real-time videoconferencing stream.

17. On the computer, a. receiving a real-time video conference stream including a video of a first user; b. detecting and isolating a first background in the received real-time videoconferencing stream from the first user; c. replacing the separated first background with a second background received from a system of a second user; A program to execute each step.

Images