Video call device using virtual camera

Abstract

A video call device using a virtual camera of the present invention comprises: a virtual camera unit installed in a mobile communication terminal that includes a physical camera, a video storage unit storing one or more video files or still image files, and one or more video call applications, the virtual camera unit being configured to cause a virtual camera to be recognized as a hardware device identical to the physical camera; a camera switching unit that selectively connects the physical camera or the virtual camera unit to the video call application; an image processing unit that processes an image to be outputted to the virtual camera unit; and a user interface via which a user selects, from the video storage unit, a video file or a still image file to be processed by the image processing unit, and a camera control unit that controls the selected video file or still image file stored in the video storage unit to be outputted to the image processing unit for processing according to the video file or still image file selected via the user interface, and instructs the camera switching unit to selectively transmit an output of the physical camera or the virtual camera unit to the video call application.

Description

Video call device using a virtual camera

[0001] The present invention relates to a video call device using a virtual camera, and more specifically, to a video call device using a virtual camera that can provide an image reproduced by a virtual camera as an input image instead of an image from a real camera, regardless of the type of video call app, when performing a video call through various video call apps that require camera input images.

[0002] Recently, in addition to the video calling application programs pre-installed on mobile devices (hereinafter simply referred to as "apps"), a wide variety of video calling apps have been proposed and are being actively used, such as KakaoTalk's FaceTalk, LINE's video calling function, Google's Duo, and Apple's FaceTime. Furthermore, various video conferencing apps utilizing the physical camera of a mobile device or a webcam connected to a desktop PC or laptop PC have been proposed and are being actively used. Hereinafter, video calling apps and video conferencing apps will be collectively referred to as "video calling apps," and the explanation will proceed using mobile devices as examples of the devices used therein.

[0003] Meanwhile, video calls via such conventional video call apps (including multi-party video calls such as group calls) are performed by transmitting video and audio input through a physical camera and microphone equipped on a mobile communication terminal (since the present invention focuses only on video, the explanation will proceed using only video as an example) through the same video call app on the other party's mobile communication terminal.

[0004] However, due to the protection of personal privacy or commercial needs such as entertainment or fun, there may be a demand to avoid transmitting video input from a physical camera directly to the other party's video call app, or to replace the video input from the physical camera with a video of one's own choosing to transmit to the other party's video call app.

[0005] In this regard, a technology for replacing an image input through a real camera with another pre-prepared image and transmitting it through the other party's mobile communication terminal is proposed in the following prior art 1 and 2 for the purpose of measuring service quality for video calls.

[0006] However, the technology proposed as Prior Art 1 below is a technology primarily used in 3G (WCDMA) mobile communication networks, and performs video calls in a low-speed data environment according to the H.324M protocol defined by the ITU-T. It replaces video input from a real camera by inputting video through a video call emulator or a video call interface provided by a mobile communication terminal. Since the technology proposed as Prior Art 1 was developed to suit the 3G (WCDMA) environment where only video call services using H.324M existed, it has problems such as having to be redeveloped to suit other video call services even if they exist.

[0007] FIG. 1 is a block diagram schematically illustrating a device for measuring the quality of a video call service in real time as proposed in the following prior art 2. As shown in FIG. 1, assuming a situation in which a first terminal (310) and a second terminal (320) conduct a video call in the prior art 2, the first terminal (310) can generate a media stream by encoding and packetizing an input signal (312), which is a signal for a video captured through a camera (311), through an encoder and a packetizer (313), and then transmit the generated media stream to the second terminal (320) through a network (170). At this time, the second terminal (320) can decode and depacketize the media stream received from the first terminal (310) through a decoder and a depacketizer (321) to generate an output signal (322) and then output it through an output unit (323) so that video and audio for a video call can be output from the second terminal (320).

[0008] Meanwhile, to measure the quality of the video call service, the first terminal (310) may transmit the original video stored in the first storage (314) to the second terminal (320) instead of the input signal (312), and in this process, the original video may be processed through the original video processing unit (315). For example, since the frame rate and / or resolution of the video call service may not match the pre-generated original video, the first terminal (310) may process the original frames included in the original video through the original video processing unit (315) so that the frame rate and resolution of the original video correspond to the frame rate and resolution of the video call service for which the quality is to be measured. As a more specific example, if the frame rate of the video input through the camera is 15 fps (frames per second) and the frame rate of the original video is 30 fps, the original video can be replaced by skipping one frame at a time, that is, if the indices of the original frames included in the original video are assigned as '1, 2, 3, 4, 5, 6, 7, 9', the original frames of '1, 3, 5, 7, 9' can be replaced with the frames of the video input through the camera.

[0009] However, according to the aforementioned prior art 2, each of the various video call apps must provide its own API (Application Programming Interface) to connect an original image processing unit (315) for replacing the image of a real camera with the original image, and an encoder and packetizer (313), although not illustrated. Accordingly, there was a problem of duplicate development costs incurred in creating and linking a dedicated API for each video call app according to its type and characteristics in order to provide a function to transmit a replacement image instead of the image input through the real camera.

[0010] Above all, there was a problem in that if a specific video call app did not provide a function to transmit an alternative video instead of the video input through the physical camera, users who wanted to make a video call through it had no choice but to use only the video input through the physical camera.

[0011] The main objective of the present invention is to provide a video call device using a virtual camera that can provide an image reproduced by a virtual camera as an input image instead of an image from a real camera, regardless of the type of video call app, when performing a video call through various video call apps that require camera input images.

[0012] According to one feature of the present invention for achieving the aforementioned purpose, a video call device using a virtual camera is provided, comprising: a virtual camera unit installed in a mobile communication terminal equipped with a real camera, an image storage unit storing one or more video or still image files, and one or more video call apps, and which recognizes the virtual camera as a hardware device such as the real camera; a camera switching unit that selectively connects the real camera and the virtual camera unit to the video call app; an image processing unit that processes an image to be output to the virtual camera unit; a user interface that receives a video or still image to be processed by the image processing unit from a user in the image storage unit, and a camera control unit that controls the video or still image stored in the image storage unit to be output to the image processing unit and processed as set through the user interface, and controls the camera switching unit to selectively transmit the output of the real camera and the virtual camera unit to the video call app.

[0013] According to another feature of the present invention, a video call device using a virtual camera is provided, comprising: a virtual camera unit installed in a mobile communication terminal equipped with a real camera, an image storage unit storing one or more video or still image files, and one or more video call apps, and which recognizes the virtual camera as a hardware device such as the real camera; an image processing unit that synthesizes an image input through the virtual camera unit and an image captured by the real camera and transmits them to the video call app; a user interface that receives a video or still image to be output to the virtual camera unit from a user in the image storage unit, and a camera control unit that controls the video or still image stored in the image storage unit to be output to and processed by the image processing unit as set through the user interface.

[0014] The image processing unit processes video or still images selected by the user at the same frame rate and resolution as a real camera.

[0015] The video processing unit repeatedly plays the video if the video call time is longer than the length of the video stream provided through the virtual camera unit.

[0016] According to the video call device using a virtual camera of the present invention, during a video call, an image selected by the user is transmitted through the virtual camera, or the image selected by the user becomes a background image through the virtual camera, and a person image of the user extracted from images captured by a real camera is composited with the background image as a foreground image and transmitted to the other party's terminal, thereby enabling the protection of personal privacy and allowing it to be used according to commercial needs such as entertainment or fun.

[0017] FIG. 1 is a block diagram schematically illustrating a device for measuring the quality of a video call service in real time, proposed as prior art 2.

[0018] FIG. 2 is a functional block diagram of a video call device using a virtual camera according to an embodiment of the present invention.

[0019] FIG. 3 is a flowchart for explaining a video call method based on an embodiment of FIG. 2.

[0020] FIG. 4 is a functional block diagram of a video call device using a virtual camera according to another embodiment of the present invention.

[0021] FIG. 5 is a flowchart for explaining a video call method based on an embodiment of FIG. 4.

[0022] FIG. 6 is a figure for explaining the effect of a video call device through a virtual camera of the present invention.

[0023] Hereinafter, a preferred embodiment of the video call device using a virtual camera according to the present invention will be described in detail with reference to the attached drawings.

[0024] FIG. 2 is a functional block diagram of a video call device using a virtual camera according to an embodiment of the present invention, and for convenience, only the configuration related to the video call on the sending side is illustrated. As shown in FIG. 2, the functional configuration of a video call device using a virtual camera according to an embodiment of the present invention may largely comprise a real camera (100), a video storage unit (150) in which various video content, such as video or image (still image) files, that can be provided to a mobile communication terminal of the other party through a virtual camera is stored, various video call apps (400), and a virtual camera app (200) of the present invention.

[0025] In the above configuration, a physical camera (100) and a video storage unit (150), which is a memory, may be basically provided in a mobile communication terminal. One or more video call apps (400) that may be installed in the mobile communication terminal may include various video call apps installed by the user in addition to the video call apps basically provided by the mobile communication terminal, such as KakaoTalk's FaceTalk, LINE's video call function, Google's Duo, and Apple's FaceTime as described above, and may also include various video conferencing apps. Each of these video call apps (400) may be composed of a camera interface (410), an encoder (420), and a packetizer (430) as in the past.

[0026] Meanwhile, the virtual camera app (200) of the present invention may specifically comprise a virtual camera unit (240), a camera switching unit (250) that selectively connects a real camera (100) and the virtual camera unit (240) to various video call apps (400), an image processing unit (230), a user interface (UI) (220) that receives a video or still image to be processed by the image processing unit (230) from a user in an image storage unit (150) and sets the playback order or the number of repetitions, and a camera control unit (210).

[0027] First, the virtual camera unit (240) is a component that enables various video call apps (400) to recognize the virtual camera as a hardware device such as a real camera (100), and can be implemented, for example, in the Hardware Abstraction Layer (HAL) of the Android Architecture.

[0028] Here, the Hardware Abstraction Layer (HAL) is intended to provide a clean and consistent interface by concealing the specific characteristics and complex internal structure of the hardware, thereby enabling various video call applications to discover and use the hardware of the mobile communication terminal through a simple and abstracted API, regardless of the type of underlying hardware (in this invention, a camera).

[0029] In addition to this, the virtual camera unit (240) can be implemented in the framework layer of iOS and macOS or developed as a driver for Windows.

[0030] Next, the camera switching unit (250) selectively transmits a video signal captured through the real camera (100) or a video signal input through the virtual camera unit (240) to the camera I / F (interface) of various video call apps (400). The video processing unit (230) processes the video or still image of the video storage unit (150) selected or set by the user of the virtual camera app (200) through the user interface (220) into the same form as the input image of the real camera (100), for example, with the same frame rate and resolution, and transmits it to the virtual camera unit (240). For example, if a single still image file is selected, the video processing unit (230) processes the still image into a video form having the same frame rate and resolution as the video input through the real camera (100) until the video call ends, and transmits it to the virtual camera unit (240). When a user selects a video file having a specific frame rate and resolution through the user interface (220), the video file is processed into a video stream having the same frame rate and resolution as the video input through the real camera (100) and transmitted to the virtual camera unit (240). At this time, if the video call time is longer than the length (time) of the video stream processed by the video processing unit (230), the video can be played repeatedly, whereas if it is short, playback can be stopped.

[0031] Finally, the camera control unit (210) controls the video or still image stored in the image storage unit (150) to be output to the image processing unit (230) for processing as input or set through the user interface (220), and controls the switching operation between the real camera (100) and the virtual camera unit (240) to selectively transmit the output of the real camera (100) and the virtual camera unit (240) to the camera interface (410) of the video call app (400).

[0032] The user interface (220), image processing unit (230), camera switching unit (250), and camera control unit (210) can be implemented, for example, in the application layer of the Android architecture.

[0033] In this way, the video selected or set by the user during the video call process passes through the virtual camera unit (240) and is transmitted in real time to the other party's terminal via a wired / wireless communication network as if it were a video captured by a real camera (150), thereby protecting personal privacy or being used for commercial purposes such as entertainment or fun.

[0034] FIG. 3 is a flowchart for explaining a video call method based on an embodiment of FIG. 2, and unless otherwise stated, it is performed primarily by a camera control unit (210). As illustrated in FIG. 3, according to a video call method using a virtual camera according to an embodiment of the present invention, in step S100, it is determined whether the virtual camera app (200) is executed, and if it is executed, the process proceeds to step S110 to determine whether a virtual camera on command is input from the user. If a virtual camera on command is input in step S110, in step S120, the camera switching unit (250) is controlled to connect the virtual camera unit (240) to the video call app (400), and in step S130, it is determined whether a video call is currently in progress. The process of executing and connecting the video call may be performed before the virtual camera app (200) is executed or after the virtual camera app is executed. If a video call is in progress at step S130, proceed to step S140 to transmit the video processed by the video processing unit (230) as set by the user to the other party's terminal via the virtual camera unit (240), and in the subsequent step S150, determine whether the video call has ended. If the video call has not ended at step S150, step S140 is repeated, whereas if it has ended, the program is terminated. Meanwhile, the camera control unit (210) receives a selection of a video or still image to be used for the video call from the user via the user interface (220) before or after the execution of step S100, and one or more videos or still images may be selected during this process. In addition, during this process, the playback order, playback length, or number of repetitions of the video or still image may be selected by the user.

[0035] If the virtual camera is not turned on in step S110, that is, if the default setting physical camera (100) is turned on, the process proceeds to step S160 to determine whether a video call is in progress. If the video call is in progress in step S160, the process proceeds to step S170 to transmit the video captured by the default setting physical camera (100) to the other party's terminal, and in the subsequent step S180, it is determined whether the video call has ended. If the video call has not ended in step S180, step S170 is repeated, whereas if it has ended, the program is terminated.

[0036] FIG. 4 is a functional block diagram of a video call device using a virtual camera according to another embodiment of the present invention, and for convenience, only the configuration related to the video call on the sending side is illustrated. As shown in FIG. 4, the functional configuration of a video call device using a virtual camera according to another embodiment of the present invention may largely comprise a real camera (100), a video storage unit (150) in which various video content, such as video or still image files, that can be provided to the other party's mobile communication terminal through the virtual camera is stored, various video call apps (400), and a virtual camera app (200') of the present invention.

[0037] In the above configuration, a physical camera (100) and a video storage unit (150), which is a memory, may be basically provided in a mobile communication terminal. One or more video call apps (400) that may be installed in the mobile communication terminal may include various video call apps installed by the user in addition to the video call apps basically provided by the mobile communication terminal, such as KakaoTalk's FaceTalk, LINE's video call function, Google's Duo, and Apple's FaceTime as described above, and may also include various video conferencing apps. Each of these video call apps (400) may be composed of a camera interface (410), an encoder (420), and a packetizer (430) as in the past.

[0038] Meanwhile, the virtual camera app (200') of the present invention may specifically comprise a virtual camera unit (240'), an image processing unit (230'), a user interface (220') that receives a video or still image to be processed by the image processing unit (230') from a user in an image storage unit (150) and sets the playback order or the number of repetitions, and a camera control unit (210').

[0039] First, the virtual camera unit (240') is a component that allows various video call apps (300) to recognize the virtual camera as a hardware device such as a real camera (100).

[0040] Next, the video processing unit (230') uses a video or still image from the video storage unit (150) selected or set by the user of the virtual camera app (200') through the user interface (220') as the background image, extracts only the image of the user from the images captured by the real camera (100) as the foreground image, composites it with the background image, and transmits it to the video call app (400). In this process, the background image input through the virtual camera unit (240') can be processed to have the same form as the image of the user extracted after being captured by the real camera (100), for example, with the same frame rate and resolution. At this time, if the video call time is longer than the length (time) of the video stream provided through the virtual camera unit (240'), the video can be played repeatedly, whereas if it is short, playback can be stopped.

[0041] Finally, the camera control unit (210') transmits the output of the image processing unit (230') to the camera interface (410) of the video call app (400) and controls the output of a video or still image stored in the image storage unit (150) to the virtual camera unit (240) as input or set through the user interface (220'). The user interface (220'), the image processing unit (230'), and the camera control unit (210') can be implemented, for example, in the application layer of an Android structure.

[0042] In this way, a video selected or set by the user during the video call process is combined with a video of the user's person captured by a real camera through a virtual camera unit (240) and transmitted in real time to the other party's terminal, thereby protecting personal privacy or being used according to commercial needs such as entertainment or fun.

[0043] FIG. 5 is a flowchart for explaining a video call method based on an embodiment of FIG. 4, and unless otherwise stated, it is performed primarily by a camera control unit (210'). As illustrated in FIG. 5, according to a video call method using a virtual camera according to an embodiment of the present invention, first, in step S200, it is determined whether a virtual camera app (200') has been executed; if it has been executed, the process proceeds to step S210, where a video or still image to be used as a background in a video call is selected by the user through a user interface (220'), and in this process, one or more videos or still images may be selected. In addition, in this process, the playback order, playback length, or number of repetitions of the videos or still images may be selected by the user.

[0044] Next, in step S220, it is determined whether a video call is currently in progress. The process of initiating and connecting the video call may be performed before the virtual camera app (200') is executed or after the virtual camera app (200') is executed. If a video call is in progress in step S220, the process proceeds to step S230, where the image input through the virtual camera unit (240') and processed according to the user's settings through the image processing unit (230') is used as the background image. Additionally, the image in which the user's person image, extracted by the image processing unit (230') from the images captured by the real camera (100), is composited into the foreground image is transmitted to the other party's terminal through the video call app (400). Next, in step S240, it is determined whether the video call has ended. If the video call has not ended, step S230 is repeated, whereas if it has ended, the program is terminated.

[0045] In the above configuration, the technique for extracting a user's image from an image captured by a real camera (100) is already known, and since the extraction process is not included in the technical concept of the present invention, a detailed description is omitted.

[0046] FIG. 6 is a diagram illustrating the effect of a video call device using a virtual camera according to the present invention. As shown in FIG. 6, in the past, as shown in (a), only video captured by a real camera was transmitted to the other party's terminal during the video call process, so unwanted video of my surroundings was also transmitted, which was insufficient in terms of considering personal privacy.

[0047] However, according to the video call device using a virtual camera of the present invention, as illustrated in (b), during the video call process, an image selected by the user is transmitted through the virtual camera, or the image selected by the user becomes a background image through the virtual camera, and a person image of the user extracted from an image captured by a real camera is composited with the background image as a foreground image and transmitted to the other party's terminal, thereby enabling protection of personal privacy and allowing it to be used according to commercial needs such as entertainment or fun.

[0048] The video call device using the virtual camera of the present invention is not limited to the aforementioned embodiments and may be implemented with various modifications within the scope permitted by the technical concept of the present invention. For example, the image processing unit (230') may use a video or still image from the image storage unit (150), which is selected or set by the user of the virtual camera app (200') through the user interface (220'), as the foreground image, and composite an image captured by the real camera (100) with the background image and transmit it to the video call app (400).

Claims

1. Installed in a mobile communication terminal equipped with a physical camera, an image storage unit storing one or more video or still image files, and one or more video call apps, A virtual camera unit that recognizes the virtual camera as a hardware device similar to a real camera; A camera switching unit that selectively connects a real camera and a virtual camera unit to a video call app; An image processing unit that processes an image to be output to a virtual camera unit; A user interface that receives video or still images to be processed in the image processing unit from the user in the image storage unit, and A video call device using a virtual camera comprising a camera control unit that controls video or still images stored in an image storage unit to be output to an image processing unit for processing as set through a user interface, and controls a camera switching unit to selectively transmit the output of a real camera and a virtual camera unit to a video call app.

2. Installed in a mobile communication terminal equipped with a physical camera, an image storage unit storing one or more video or still image files, and one or more video call apps, A virtual camera unit that recognizes the virtual camera as a hardware device similar to a real camera; A video processing unit that combines video input through a virtual camera unit with video captured by a real camera and transmits it to a video call app; A user interface that receives settings from the image storage unit for a video or still image to be output to the virtual camera unit from the user, and A video call device using a virtual camera comprising a camera control unit that controls video or still images stored in an image storage unit to be output to an image processing unit for processing as set through a user interface.

3. In claim 1 or 2, A video call device using a virtual camera, characterized in that the image processing unit processes a video or still image selected by the user at the same frame rate and resolution as a real camera.

4. In claim 1 or 2, A video call device using a virtual camera, characterized in that the video processing unit repeatedly plays the video when the video call time is longer than the length of the video stream provided through the virtual camera unit.

Images