Image capture method and system during video calls

Abstract

This disclosure relates to a method for capturing images during a video call, which is performed by at least one processor of a first user terminal. [Solution] The method for capturing images during a video call includes the steps of: starting a video call between a first user terminal associated with a first user and a second user terminal associated with a second user, both included in a chat room of an instant messaging application; entering a multi-user photo-taking mode while the video call is in progress; generating a first chroma-key image based on an image captured by the image sensor of the first user terminal in the multi-user photo-taking mode; and transmitting the first chroma-key image to the second user terminal, wherein the first chroma-key image is an image in which the background area, excluding the person area from the captured image, has been changed to a first color.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to a method and system for capturing an image during a video call, and specifically, to a method and apparatus that generate a chroma-key image based on an image captured by an image sensor in a multi-user photographing mode, transmit the generated chroma-key image to other user terminals, and output the generated composite image. BACKGROUND ART

[0002] With the proliferation of mobile devices such as recent smartphones and the development of the Internet, instant messaging applications and the like that enable not only voice calls but also video calls between multiple users are widely used. Meanwhile, the demand for so-called "photobooths" that print and provide photographs taken by multiple people together offline is steadily increasing.

[0003] However, since the image capture function provided in existing video call service providers only captures the screen at a specific moment during a video call, it is difficult to provide users with images to which various formats, concepts, photo themes and the like are applied. Therefore, existing image capture services have a problem that it is difficult to provide users with satisfactory images compared to the case where photographing is performed in a "photobooth".

[0004] In addition, even if original images captured at each user terminal are transmitted to a specific user terminal to generate a composite image at the specific user terminal in order to provide high-quality images, not only is high bandwidth required to transmit / receive the original captured images, but also much resources and time are required for image processing. Accordingly, there is a demand for the introduction of a method capable of reducing the consumption of communication / computing resources while realizing high-quality images. PRIOR ART DOCUMENTS PATENT DOCUMENTS

[0005] [Patent Document 1] Korean Published Patent No. 10-2020-0145420 [Overview of the project] [Problems that the invention aims to solve]

[0006] This disclosure provides a method for capturing images during a video call, a computer-readable non-temporary recording medium for recording command words, and a system (device) for solving the aforementioned problems. [Means for solving the problem]

[0007] This disclosure can be implemented in a variety of ways, including methods, systems (apparatus), or computer-readable non-temporary recording media on which instructions are recorded.

[0008] An image capture method during a video call, executed by at least one processor of a first user terminal according to one embodiment of the present disclosure, includes the steps of: initiating a video call between a first user terminal associated with a first user and a second user terminal associated with a second user, both included in a chat room of an instant messaging application; entering a multi-user photo capture mode while the video call is in progress; generating a first chroma-key image based on an image captured by the image sensor of the first user terminal in the multi-user photo capture mode; and transmitting the first chroma-key image to the second user terminal, wherein the first chroma-key image is an image in which the background area, excluding the person area from the captured image, has been changed to a first color.

[0009] One embodiment of the present disclosure further includes the step of outputting a composite image, which is synthesized using a first chroma key image containing a first user and a second chroma key image containing a second user, onto the display of the first user's terminal.

[0010] In one embodiment of the present disclosure, the step of generating a first chroma key image includes the step of recognizing a person region in a captured image using a person recognition model, and the step of changing the background region, which is the region outside the person region recognized from the image, to a first color.

[0011] In one embodiment of the present disclosure, the step of changing the background area to a first color includes the step of changing the background area to a first color in response to receiving a first user input of selecting a first color.

[0012] In one embodiment of the present disclosure, the step of changing to a first color includes the step of determining whether the similarity between a predetermined second color and the color contained within the person area is greater than or equal to a predetermined threshold, and the step of determining a first color whose similarity to the color contained within the person area is less than a predetermined threshold, wherein the first color is different from the second color.

[0013] In one embodiment of the present disclosure, the step of changing to a first color includes the step of outputting a message requesting a change to the second color on the display of a first user terminal in response to determining that the similarity between a predetermined second color and the color contained within the person area is greater than or equal to a predetermined threshold, and the step of receiving a second user input to change the second color to the first color.

[0014] In one embodiment of the present disclosure, the step of changing to a first color includes the steps of calculating an average color value related to the person area and determining the first color based on the average color value.

[0015] In one embodiment of the present disclosure, the step of changing to a first color includes the steps of calculating the distance between each of a plurality of color values ​​related to the person area and an arbitrary color value, and determining the first color based on the calculated distance.

[0016] One embodiment of the present disclosure further includes the steps of: entering a multi-user photo-taking mode; displaying a plurality of background themes on the display of a first user terminal; and receiving input from the first user to select one of the plurality of background themes from the third user.

[0017] In one embodiment of the present disclosure, after entering a multi-user photo-taking mode and before capturing an image captured by the image sensor of the first user terminal, the present embodiment further includes the steps of displaying a first image sequence captured by the image sensor of the first user terminal in a first area of ​​a selected background theme on the display, and displaying a second image sequence received from the second user terminal in a second area of ​​a selected background theme on the display, wherein the first and second image sequences are generated by removing the color of the background area from a chroma key image sequence.

[0018] In one embodiment of the present disclosure, after entering a multi-user photo-taking mode, the process further includes: receiving a fourth user input from a first user selecting a multi-user photo-taking button; receiving a second chroma key image including the second user from a second user terminal; and generating a composite image using the first chroma key image and the second chroma key image, wherein the first chroma key image is generated in response to receiving the fourth user input.

[0019] In one embodiment of the present disclosure, the step of generating a composite image includes: displaying a plurality of background themes on the display of a first user terminal; receiving a fifth user input from the first user to select one of the plurality of background themes; generating a first person image by removing an area displayed in a first color from a first chroma key image; generating a second person image by removing an area displayed in the color of the background area from a second chroma key image; and overlaying the first person image and the second person image onto the selected background theme.

[0020] In an embodiment of the present disclosure, the step of overlaying on the selected background theme includes the step of overlaying the first person image on the first region of the selected background theme, and the step of overlaying the second person image on the second region of the selected background theme.

[0021] In an embodiment of the present disclosure, the step of generating a composite image includes the step of displaying a plurality of background themes on the display of the first user terminal, the step of receiving a sixth user input for selecting one of the plurality of background themes from the first user, and the step of compositing a part of the selected background theme into the background region of the first chroma-key image and the background region of the second chroma-key image.

[0022] In an embodiment of the present disclosure, the method further includes the step of transmitting the composite image to the second user terminal through a chat room including the first user and the second user.

[0023] In an embodiment of the present disclosure, the method further includes the step of generating metadata of a VoIP (Voice over Internet Protocol) call packet including data corresponding to the composite image, and the step of transmitting the metadata to the second user participating in the video call, wherein the metadata is metadata within the protocol of a VoIP call channel connected for the first user and the second user participating in the video call.

[0024] In an embodiment of the present disclosure, the step of entering the multi-user photography mode includes the step of entering a specific subgroup among a plurality of subgroups related to multi-user photography, and the second user is included in the same specific subgroup as the first user.

[0025] In an embodiment of the present disclosure, the maximum number of users that can participate in multi-user photography is less than the maximum number of users that can participate in a video call.

[0026] Provided is a computer-readable non-transitory recording medium having recorded thereon instructions for causing a computer to execute the above-described method according to an embodiment of the present disclosure.

[0027] A first user terminal according to an embodiment of the present disclosure includes a communication module, a display, an image sensor, a memory, and at least one processor coupled to the memory and configured to execute at least one computer-readable program contained in the memory, wherein the at least one program comprises instructions for initiating a video call between the first user terminal associated with a first user and a second user terminal associated with a second user that are included in a chat room of an instant messaging application, entering a multi-user photography mode while the video call is in progress, generating a first chroma-key image based on an image captured by the image sensor of the first user terminal in the multi-user photography mode, and transmitting the first chroma-key image to the second user terminal, wherein the first chroma-key image is an image in which a background region excluding a person region from the captured image is changed to a first color. [Advantageous Effects of Invention]

[0028] In various embodiments of the present disclosure, a user can check whether the user's chroma-key image is normally generated before generating a composite image using the chroma-key image.

[0029] In various embodiments of the present disclosure, on a multi-user photography preview screen, image sequences captured by the image sensors of the respective user terminals are composited to display a composite image sequence, and a user can prepare in advance the position, pose, motion, etc. of the user to be displayed in the composite image through the composite image sequence.

[0030] In various embodiments of the present disclosure, it is possible to reduce the bandwidth required for image transmission and reception for composite image generation or the time / resources required for communication.

[0031] In various embodiments of this disclosure, high-resolution composite images can be generated by capturing / transmitting images used to generate composite images at high resolution.

[0032] In the various embodiments of this disclosure, images to which various photographic themes are applied can be provided.

[0033] In the various embodiments of this disclosure, even if multiple users take images in different environments, images can be generated that appear as if they were taken in a single space.

[0034] In various embodiments of this disclosure, the host can check the number of feedback items that guests have provided to a photo theme through the host terminal, and can reflect the guests' opinions when selecting a photo theme to apply to a photo booth image.

[0035] In the various embodiments of this disclosure, users assume poses according to pose guides displayed on the display, so even if there are slight differences in the timing of the users' photo shoots, the users' poses may not be misaligned.

[0036] The effects of this disclosure are not limited to those mentioned above, and other effects not mentioned can be clearly understood by a person with ordinary skill in the art to which this disclosure pertains (hereinafter referred to as "ordinary art") from the description of the claims. [Brief explanation of the drawing]

[0037] Embodiments of the present disclosure will be described with reference to the accompanying drawings described below, where similar reference numbers indicate similar elements, but are not limited thereto. [Figure 1] This is a diagram illustrating an example of a photo booth image containing multiple users, as described in one embodiment of the present disclosure, being displayed on the user terminal's screen. [Figure 2]This is a schematic diagram showing a configuration in which an information processing system is connected to multiple user terminals in a communicative manner in order to provide an image capture service during a video call according to one embodiment of the present disclosure. [Figure 3] This is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure. [Figure 4] This is a block diagram showing the internal configuration of a user terminal processor according to one embodiment of the present disclosure. [Figure 5] This diagram illustrates an example of how a chroma key image for generating a composite image is generated from an image captured by an image sensor of a user terminal according to one embodiment of the present disclosure. [Figure 6] This drawing illustrates an example of a chroma key image being generated in response to user input according to one embodiment of the present disclosure. [Figure 7] This diagram shows the operation of entering a multi-user photo-taking mode on the host terminal side according to one embodiment of the present disclosure. [Figure 8] This diagram shows the operation of entering a multi-user photo-taking mode on the guest terminal side according to one embodiment of the present disclosure. [Figure 9] This is a drawing illustrating an example of a guest providing feedback on a photo theme according to one embodiment of the present disclosure. [Figure 10] This is a drawing illustrating an example of a photo booth image being displayed according to one embodiment of the present disclosure. [Figure 11] This diagram shows a video call group related to video calls and a subgroup related to a multi-user photo shooting mode according to one embodiment of the present disclosure. [Figure 12] This is a flowchart showing the process of generating a composite image according to one embodiment of the present disclosure. [Figure 13] This flowchart shows a method for capturing images during a video call according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0038] The specific details for implementing this disclosure will be described below with reference to the attached drawings. However, in the following description, specific explanations of well-known functions and configurations will be omitted if they may unnecessarily obscure the gist of this disclosure.

[0039] In the attached drawings, identical or corresponding components are assigned the same reference numeral. Furthermore, in the following descriptions of embodiments, the description of identical or corresponding components may be omitted. However, the omission of a description of a component does not mean that such a component is not included in the embodiment.

[0040] The advantages and features of the disclosed embodiments, and the methods for achieving those advantages and features, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, this disclosure is not limited to the embodiments disclosed below and can be realized in a variety of different forms, and these embodiments are provided merely to complete the disclosure and to fully inform the ordinary art of the scope of the invention.

[0041] This specification provides a brief explanation of the terms used and a more detailed explanation of the disclosed embodiments. The terms used in this specification are selected from currently widely used general terms, taking into account the functions described herein. However, the selected general terms may change based on the intent of articulators in the relevant field, case law, the emergence of new technologies, etc. In some cases, the applicant has arbitrarily selected terms, and in such cases, the meaning of the terms will be described in detail in the description of the relevant invention. Therefore, the terms used in this disclosure should not be merely names, but should be defined based on their meaning and the overall content of this disclosure.

[0042] In this specification, singular expressions include plural expressions unless the context clearly identifies them as singular. Conversely, plural expressions include singular expressions unless the context clearly identifies them as plural. When a part of the specification is said to contain a component, containing a component means, unless otherwise stated to the contrary, that it may contain other components rather than excluding them.

[0043] Furthermore, the terms “module” or “part” as used in this specification mean either a software component or a hardware component, and a “module” or “part” performs a certain role. However, the meaning of “module” or “part” is not limited to software or hardware. A “module” or “part” may be configured to reside on an addressable storage medium, or to be configured to run on one or more processors. Thus, as an example, a “module” or “part” may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within a “module” or “part” may be combined into a smaller number of components and a “module” or “part,” or further separated into additional components and a “module” or “part.”

[0044] According to one embodiment of the present disclosure, a “module” or “part” can be realized by a processor and memory. “Processor” should be broadly interpreted to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In some environments, “processor” may also refer to ordered semiconductors (ASICs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), etc. “Processor” may also refer to combinations of processing devices, such as a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors coupled with a DSP core, or any other combination of processor configurations. “Memory” should also be broadly interpreted to include any electronic component capable of storing electronic information. The term "memory" can also refer to various types of processor-readable media, such as arbitrary access memory (RAM), read-only memory (ROM), non-volatile arbitrary access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage devices, and registers. If a processor can read information from and / or record information into memory, then the memory is said to be in electronic communication with the processor. Memory integrated into a processor is in electronic communication with the processor.

[0045] In this disclosure, “User” may refer to one user of an instant messaging application who is included in a chat room of the instant messaging application, or to a user who is in the middle of a video call with another user in the chat room. Alternatively, “User” may refer to an image or a portion of an image in which the user is displayed.

[0046] In this disclosure, “user account” may refer to an account or data associated with an account that a user creates and uses within an instant messaging application. Furthermore, a user account within an instant messaging application may refer to a user who uses the instant messaging application. Similarly, a user who uses instant messaging or a chat room capable of instant messaging may refer to a user account within an instant messaging application.

[0047] In this disclosure, "chat room" can refer to a virtual space or group that can be created in an instant messaging application or the like installed on a computing device, and in which one or more users (or user accounts) can participate. For example, one or more user accounts can participate in or be included in a chat room and exchange messages, files, etc., in various forms with each other. In addition, the chat room is provided with VoIP (Voice over Internet Protocol) voice call functionality, VoIP video call functionality, live broadcast functionality (VoIP real-time video transmission functionality), and multimedia content generation functionality, and voice calls, video calls, video streaming, and multimedia content transmission between user accounts can be performed.

[0048] In this disclosure, "host" may refer to a user who has requested a multi-user photo shoot while a video call is in progress, or a user who has been granted host privileges from an existing host, and "host terminal" may refer to a user terminal associated with the host.

[0049] In this disclosure, "guest" may refer to a user who accepts a host's request for a multi-user photo shoot and enters multi-user photo shoot mode, and "guest terminal" may refer to a user terminal associated with the guest.

[0050] In this disclosure, unless otherwise specified, “User” may refer to the host and / or guest, and “User Terminal” may refer to the host terminal and / or guest terminal.

[0051] In this disclosure, “Photo Booth Image” may refer to the final result image of a multi-user photo-taking mode, which includes one or more composite images.

[0052] Figure 1 is a diagram illustrating an example of a photo booth image (152) containing multiple users being displayed on the display of a user terminal according to one embodiment of the present disclosure. A first operation (110) may show a video call screen (114) that proceeds between multiple user terminals relating to multiple users. On each of the multiple user terminals, a screen containing each user can be displayed at any position and size on the display, and the video call screen (114) may not be displayed identically on all user terminals. In one embodiment, a video call may be initiated and proceeded between each of the multiple user terminals relating to multiple users included in a chat room of an instant messaging application.

[0053] In Figure 1, four users are shown conducting a video call, but it is not limited to four users. Any number of users (e.g., six) who are at least some of the users (e.g., eight) included in the chat room of the instant messaging application can conduct a video call simultaneously.

[0054] In one embodiment, during a video call, a sequence of images captured by the image sensor of a user terminal participating in the video call can be transmitted to other user terminals participating in the video call. The video call screen (114) can be a screen that displays the sequence of images captured by the image sensor of the user terminal and the sequence of images received from other user terminals.

[0055] In the first action (110), one of the users in a video call can enter the multi-user photo shooting mode by touching the multi-user photo shooting mode button (112) or by other means. Subsequently, a request for multi-user photo shooting can be sent to the other users in the video call. The other users in the video call can enter the multi-user photo shooting mode by agreeing to the request. The specific process for entering the multi-user photo shooting mode will be described in detail in Figures 7 and 8.

[0056] The second operation (120) illustrates an example in which a chroma key image (122) related to the user is output on the display before multi-user photography. The chroma key image (122) can be generated based on an image captured by the image sensor of the user terminal. Through the chroma key image (122), the user can check whether their chroma key image (122) is generated correctly before generating a composite image using the chroma key image. After determining that the chroma key image (122) is generated and output correctly, the user can enter multi-user photography mode by selecting the Enter button (128). Alternatively, the user can enter multi-user photography mode after a predetermined time has elapsed without selecting the Enter button (128).

[0057] A chroma key image (122) can be generated by changing the background region (124), which is the captured image with the person region (126) excluded, to a specific color. For example, the person region (126) can be recognized using a person recognition model that has been pre-trained to distinguish between people and background in an image, and the background region (124), with the person region (126) excluded, can be changed to a specific color. The specific process by which the color of the background region (124) is determined and changed will be described in detail in Figures 5 and 6.

[0058] Contrary to what is shown in Figure 1, the screen for the second operation (120) may not be displayed on the user terminal. That is, in response to selecting the multi-user photo shooting mode button (112) in the first operation (110), the chroma key image (122) may not be output on the display, and the screen for the third operation (130) may be immediately output on the display.

[0059] The third operation (130) shows an example in which, after multiple user terminals have entered multi-user photo shooting mode, a composite image (142) formed by combining an image containing the first user (displayed as User A) and an image containing the second user (displayed as User B), and a multi-user photo shooting preview screen (144) are displayed on the display. The multi-user photo shooting preview screen (144) can display the chroma key images of each user. Additionally or alternatively, the multi-user photo shooting preview screen (144) can be displayed with each user's chroma key image and a background theme combined.

[0060] In one embodiment, in response to a user (host) who selected the multi-user photo shooting mode button (112) in the first operation (110) selecting the multi-user photo shooting button (132), a composite image (142) can be generated and displayed on the display. After the composite image (142) is generated, a multi-user photo shooting preview screen (144) for generating the next composite image can be displayed on the display. In the multi-user photo shooting preview screen (144), image sequences (or chroma key image sequences) captured by the image sensors of each user terminal are combined to display a composite image sequence (or composite chroma key image sequence), and the user can prepare in advance the position, pose, actions, etc. of the user to be displayed in the composite image through the composite image sequence. For example, the first image sequence captured by the first user's image sensor is displayed in the first area of ​​the background theme selected by the user, and the second image sequence received from the second user terminal is displayed in the second area of ​​the background theme, thereby enabling the display of the composite image sequence. At this time, the first image sequence and the second image sequence can be generated by removing the color of the background area from the chroma key image sequence.

[0061] Thereafter, a composite image can be generated by the user selecting the multi-user photo shooting button (132) again. At this time, as shown in the figure, a timer ("3", "2", "1") is displayed on the multi-user photo shooting preview screen (144) in response to the user selecting the multi-user photo shooting button (132), and a composite image can be generated when the timer has elapsed.

[0062] In another example, after a user selects a multi-user photo-taking button (132) and a composite image (142) is generated, other composite images may be generated sequentially at predetermined intervals (e.g., 5 seconds). This process may be repeated until a predetermined number of composite images are generated, or until the user stops taking photos.

[0063] The composite image (142) can be generated on the host terminal. For example, the host terminal can, in response to receiving user input to select a multi-user photo shooting button (132), capture a first image including the host using an image sensor, and generate a first chroma key image by changing the background region of the first image to a first color. After receiving a second chroma key image including the guest from the guest terminal, the host terminal can generate a composite image (142) using the first chroma key image and the second chroma key image.

[0064] In one embodiment, the host terminal can generate a first person image related to the host by removing the area displayed in a first color from a first chroma key image. Similarly, the host terminal can generate a second person image related to the guest by removing the area displayed in the background color from a second chroma key image. Thereafter, the host terminal can generate a composite image (142) by overlaying the generated first and second person images onto a selected background theme (for example, a background theme selected by the host from among multiple background themes). At this time, the first person image can be overlaid onto a first area of ​​the background theme, and the second person image can be overlaid onto a second area of ​​the background theme.

[0065] In other embodiments, the host terminal can composite a portion of the background theme onto the background region of the first chroma key image and the background region of the second chroma key image.

[0066] Alternatively, the host terminal can generate a cut-off image by removing the background region from the captured image, without generating a first chroma key image, and then generate a composite image (142) using the generated cut-off image and the second chroma key image.

[0067] The composite image sequence displayed on the multi-user photo shooting preview screen (144) can be generated at each user terminal that has entered multi-user photo shooting mode. For example, a user terminal can send an image sequence (or chroma key image sequence) captured by an image sensor to other user terminals, and each user terminal can generate a composite image sequence (or composite chroma key image sequence) using the received image sequence (or chroma key image sequence) and the image sequence (or chroma key image sequence) captured by the image sensor.

[0068] The multi-user photo shooting button (132) is activated only on the user terminal of the user (host) who selected the multi-user photo shooting mode entry button (112) in the first action (110), or may only be displayed on that user terminal. Additionally, the multi-user photo shooting button (132) may be deactivated if another user turns off their camera by selecting the camera off button (138).

[0069] In one embodiment, the user can select an effect button (134) to apply various visual effects to the composite image (142) and / or the preview screen (144). Specifically, when the user selects an effect button (134), a list of effects applicable to the composite image (142) and / or the preview screen (144) is displayed on the screen, and by selecting one or more effects from the effect list, the selected effects can be applied to the composite image (142) and / or the preview screen (144). For example, it can be confirmed that an effect displaying user information (such as the ID, dialogue name, nickname, etc. of each user in an instant messaging application, like User A, User B, etc.) has been applied to each of the composite image (142) and the preview screen (144) in Figure 1.

[0070] In this case, the effects that can be applied to the composite image (142) and / or the preview screen (144) may include, but are not limited to, decorative elements (e.g., flower images, fireworks effects, etc.), informational elements (e.g., individual user information included in the composite image, date and time of shooting, text, etc.), and image attribute elements (e.g., brightness, saturation of the composite image, etc.).

[0071] After one or more effects are selected, the user can move the applied effects to their desired location using drag and drop, or apply them to their desired size using gestures such as spread or pinch.

[0072] In one embodiment, the user can select a gallery button (136) associated with the captured image. When the user selects the gallery button (136), the user can view images stored on the user's device (e.g., previously taken composite images, photo booth images, etc.). The user can also select an image stored on the user's device and add it to the photo booth images.

[0073] In one embodiment, the user can select a theme button (140) related to a theme to be applied to the photo booth image (152). For example, when the user selects a theme button (140), a list of themes applicable to the photo booth image (152) is displayed on the screen, and by the user selecting one of the themes from the theme list, the photo booth image with the selected theme applied can be displayed on the screen. Alternatively, a separate theme can be selected and applied to each of the composite images included in the photo booth image. Additionally or alternatively, the user can select a theme in advance before selecting the multi-user photo shooting button (132).

[0074] In one embodiment, the user can delete a composite image (142) by selecting the delete button (145) corresponding to the composite image (142). After the composite image (142) is deleted, the user can generate a new composite image and replace the deleted composite image with it.

[0075] In one embodiment, the delete button (145) may be activated only on the user terminal of the user (host) who selected the multi-user photo shooting mode entry button (112) in the first operation (110), or it may be displayed only on that user terminal. Alternatively, the delete button (145) may be activated for all users who have entered the multi-user photo shooting mode, but the composite image may only be deleted if a majority of all users select the delete button for the same composite image.

[0076] The fourth action (150) can illustrate an example in which, in response to the completion of image capture in the third action (130), a photo booth image (152) containing multiple composite images, each containing multiple users, is displayed on the user terminal's screen.

[0077] In Figure 1, the photo booth image (152) is shown as containing two composite images, but it is not limited to two composite images; any number of composite images can be included in the photo booth image (152) at any position and size. For example, the user can set the number of composite images to be taken, the position and size of each composite image, etc., and the composite image can be generated by taking each user's image a number of times corresponding to the set number of composite images.

[0078] Users can retake the photo booth image (152) by selecting the retake button (154). In one embodiment, the multi-user photo shoot preview screen can be output to the display again in response to the majority of users included in the photo booth image (152) selecting the retake button (154). In another embodiment, the retake button (154) is activated only by the host, and the multi-user photo shoot preview screen can be output to the display again in response to the host selecting the retake button (154).

[0079] Users can select the download button (156) to save the photo booth image (152) to their device.

[0080] The images, button sizes, and positions shown and described in Figure 1 are illustrative and not limited to those shown. For example, some buttons may be added or omitted, or configured to be of a different size and position than those shown.

[0081] Figure 2 is a schematic diagram showing a configuration in which an information processing system (230) is communicably connected to multiple user terminals (210_1, 210_2, 210_3) in order to provide an image capture service during a video call according to one embodiment of the present disclosure. The information processing system (230) may include a system(s) capable of providing instant messaging services, a system(s) capable of providing social networking services, and / or a system(s) capable of providing image capture services during video calls. In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer executable programs (e.g., downloadable applications) and data related to instant messaging services, social networking services, and / or image capture services during video calls, or one or more distributed computing devices and / or distributed databases of a cloud computing service infrastructure. For example, the information processing system (230) may include a separate system (e.g., a server) for instant messaging services, social networking services, and / or image capture services during video calls.

[0082] The instant messaging service, video call image capture service, etc., provided by the information processing system (230) can be provided to users through instant messaging applications etc. installed on each of the multiple user terminals (210_1, 210_2, 210_3). For example, the instant messaging service may include text messaging services, voice messaging services, video call services, voice call services, video streaming services, social networking services, video call image capture services etc., between users of the instant messaging application.

[0083] Multiple user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) via a network (220). The network (220) can be configured to enable communication between the multiple user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) can consist of a wired network such as Ethernet (registered trademark), PowerLine Communication, telephone line communication equipment and RS-serial communication, a wireless network such as a mobile communication network, WLAN (Wireless LAN), Wi-Fi (registered trademark), Bluetooth (registered trademark), and ZigBee (registered trademark), or a combination of a wired network and a wireless network. The communication method is not restricted and can include not only communication methods that utilize communication networks that the network (220) can include (for example, mobile communication networks, wired internet, wireless internet, broadcasting networks, satellite networks, etc.), but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0084] In Figure 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are shown as examples of user terminals. However, user terminals (210_1, 210_2, 210_3) are not limited to mobile phones, tablet terminals, and PC terminals. User terminals can be any computing device capable of wired and / or wireless communication and on which instant messaging applications, etc., can be installed and run. For example, user terminals can include smartphones, mobile phones, navigation systems, computers, laptop computers, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (virtual reality) devices, AR (augmented reality) devices, etc. Furthermore, although Figure 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with the information processing system (230) via the network (220), the system is not limited to three user terminals communicating; it can also be configured to have a different number of user terminals communicating with the information processing system (230) via the network (220).

[0085] In one embodiment, each of the user terminals (210_1, 210_2, 210_3) can receive or transmit information or data from other user terminals via the network (220). For example, each of the user terminals (210_1, 210_2, 210_3) can send and receive user input such as selecting a multi-user photo shooting button, user input such as selecting an exit button for multi-user photo shooting mode, an image (and / or chroma key image) containing the user, user input and / or image sequence (and / or chroma key image sequence) selecting one of several photo themes, etc.

[0086] Figure 3 is a block diagram showing the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure. The user terminal (210) can refer to any computing device capable of running instant messaging applications and the like, and capable of wired / wireless communication, and can include, for example, the mobile phone terminal (210_1), tablet terminal (210_2), PC terminal (210_3) shown in Figure 2. As illustrated, the user terminal (210) can include memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) can include memory (332), a processor (334), a communication module (336), and an input / output interface (338). As shown in Figure 3, the user terminal (210) and the information processing system (230) can be configured to communicate information and / or data over a network (220) using their respective communication modules (316, 336). Furthermore, the input / output device (320) can be configured to input information and / or data to a user terminal (210) or output information and / or data generated from the user terminal (210) via an input / output interface (318).

[0087] The memory (312, 332) may include any non-temporary computer-readable recording medium. In one embodiment, the memory (312, 332) may include a permanent mass storage device such as a ROM (read-only memory), disk drive, SSD (solid-state drive), or flash memory. In another example, a permanent mass storage device such as a ROM, SSD, flash memory, or disk drive may be included in a user terminal (210) or information processing system (230) as a separate permanent storage device distinct from the memory. The memory (312, 332) may also store an operating system and at least one program code (for example, code for an application related to an instant messaging service, a social networking service, or a video call image capture service).

[0088] Software components can be loaded from a computer-readable recording medium separate from memory (312, 332). This separate computer-readable recording medium may include recording media directly connected to a user terminal (210) and an information processing system (230), and may include computer-readable recording media such as floppy drives, disks, tapes, DVD / CD-ROM drives, and memory cards. As another example, software components may also be loaded into memory (312, 332) via a communication module (316, 336) rather than a computer-readable recording medium. For example, at least one program may be loaded into memory (312, 332) based on a computer program (e.g., an application related to an instant messaging service, a social networking service, or a video call image capture service) installed by a file provided via a network (220) by developers or a file distribution system distributing application installation files.

[0089] The processors (314, 334) can be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. Instructions can be provided to the processors (314, 334) by memory (312, 332) or communication modules (316, 336). For example, the processors (314, 334) can be configured to execute instructions received according to program code stored in a recording device such as memory (312, 332).

[0090] The communication modules (316, 336) can provide configurations or functions for a user terminal (210) and an information processing system (230) to communicate with each other via a network (220), and can provide configurations or functions for a user terminal (210) and / or an information processing system (230) to communicate with other user terminals or other systems (for example, a separate cloud system). For example, a request or data (e.g., a request for taking photos between multiple users) generated by the processor (314) of the user terminal (210) according to program code stored in a recording device such as memory (312) can be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, control signals or commands provided under the control of the processor (334) of the information processing system (230) can be received by the user terminal (210) via the communication module (336) and the network (220) through the communication module (316) of the user terminal (210). For example, a user terminal (210) can receive images (and / or image sequences) containing other users from an information processing system (230).

[0091] The input / output interface (318) can be a means for interface with an input / output device (320). For example, the input device may include a camera with an audio sensor and / or image sensor, a keyboard, a microphone, a mouse, etc., and the output device may include a display, a speaker, a haptic feedback device, etc. As another example, the input / output interface (318) can be a means for interface with a device that integrates a configuration or function for performing input and output into one, such as a touchscreen. In Figure 3, the input / output device (320) is shown not to be included in the user terminal (210), but the input / output device (320) is not limited to not being included in the user terminal (210), and can be configured as a single device with the user terminal (210). Furthermore, the input / output interface (338) of the information processing system (230) can be a means for interface with an input or output device (not shown) that is connected to or may be included in the information processing system (230). In Figure 3, the input / output interfaces (318, 338) are shown as elements configured separately from the processors (314, 334). However, they are not limited to being configured separately from the processors; the input / output interfaces (318, 338) can be configured to be included in the processors (314, 334).

[0092] The user terminal (210) and the information processing system (230) may include more components than those shown in Figure 3. However, it is not necessary to clearly illustrate most of the conventional components. In one embodiment, the user terminal (210) can be implemented to include at least some of the above-described input / output devices (320). The user terminal (210) may also further include other components such as a transceiver, a GPS (Global Positioning system) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that are generally included in a smartphone, and the user terminal (210) can be implemented to further include a variety of components such as an accelerometer, a gyroscope, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration.

[0093] In one embodiment, the processor (314) of a user terminal (210) can be configured to run an instant messaging application or a web browser application that provides an instant messaging service including an image capture service during a video call. At this time, program code related to the application can be loaded into the memory (312) of the user terminal (210). While the application is running, the processor (314) of the user terminal (210) can receive information and / or data provided by an input / output device (320) through an input / output interface (318), or receive information and / or data from an information processing system (230) through a communication module (316), process the received information and / or data, and store it in memory (312). Information and / or data can also be provided to the information processing system (230) through the communication module (316).

[0094] While the instant messaging application is running, the processor (314) can receive audio data, text, images, video, etc., input or selected through input devices such as a touchscreen, keyboard, camera including an audio sensor and / or image sensor, and microphone, which are connected to the input / output interface (318). The processor can store the received audio data, text, images, and / or video, etc., in memory (312) or provide them to an information processing system (230) via a communication module (316) and a network (220). In one embodiment, the processor (314) can receive user input to select a graphic object input through an input device and displayed on a display, and provide data / requests corresponding to the received user input to an information processing system (230) via a network (220) and a communication module (316).

[0095] The processor (314) of the user terminal (210) can transmit and output information and / or data to an input / output device (320) via an input / output interface (318). For example, the processor (314) of the user terminal (210) can output processed information and / or data via an output device (320) such as a display output device (e.g., touchscreen, display, etc.) or an audio output device (e.g., speaker). In one embodiment, the processor (314) can display a composite image on the display of the user terminal (210) that is synthesized using chroma key images, each containing a different user.

[0096] The processor (334) of the information processing system (230) can be configured to manage, process, and / or store information and / or data received from multiple user terminals (210) and / or multiple external systems. The information and / or data processed by the processor (334) can be provided to the user terminals (210) via a communication module (336) and a network (220).

[0097] Figure 4 is a block diagram showing the internal configuration of a user terminal processor (314) according to one embodiment of the present disclosure. As shown, the processor (314) may include an image acquisition unit (410), an image preprocessing unit (420), an image synthesis unit (430), an image postprocessing unit (440), and the like.

[0098] The image capture unit (410) can receive images and / or image sequences that include the user, captured using the image sensor of the user terminal. Images and / or image sequences can be received / transmitted using VoIP (Voice over Internet Protocol). For example, the image capture unit (410) can receive an image sequence that includes the user during a video call or after entering a multi-user photo-taking mode. The image sequence or chroma key image sequence that includes the user can be transmitted to other user terminals.

[0099] In another example, the image capture unit (410) can receive an image containing the user (host) from the image sensor in response to the user (host) selecting the multi-user photo capture button. The image capture unit (410) can also receive an image (or chroma key image) containing other users (guests) from other user terminals. Subsequently, a composite image can be generated based on the image (or chroma key image) containing the user (host) and the image (or chroma key image) containing other users (guests).

[0100] In yet another example, the image capture unit (410) can receive an image containing the user (guest) captured using the image sensor in response to another user (host) selecting the multi-user photo capture button. At this time, the image containing the user (guest) (or the chroma key image generated by the image preprocessing unit (420)) can be transmitted to other users (hosts) and used for generating composite images.

[0101] The image preprocessing unit (420) can preprocess images captured by the image capture unit (410) and / or images received from other user terminals. In one embodiment, the image preprocessing unit (420) can use a person recognition model to recognize the person region within the user image captured by the image capture unit (410), and generate a chroma key image by changing the background region, which is the region outside the person region, to a specific color. The specific process by which the background region is changed to a specific color will be described in detail in Figures 5 and 6.

[0102] The image preprocessing unit (420) can generate a person image or a person image sequence by removing the color from the background region of a chroma key image or chroma key image sequence. By generating a composite image based on the image from which the background elements have been removed, it is possible to generate an image that appears as if multiple users took photographs in a single space, and it is also possible to reduce the bandwidth required for image transmission and reception for composite image generation.

[0103] In other embodiments, the image preprocessing unit (420) can normalize the hue, color intensity, brightness, and saturation (e.g., lightness and saturation) of images captured by the image capture unit (410) and / or images of other users received from other user terminals. Through this configuration of normalizing color attributes, even if multiple users capture images in different environments, images can be generated that appear as if they were taken in a single space.

[0104] In further embodiments, the image preprocessing unit (420) can resize the images containing each of the multiple users so that the sizes of the multiple users appear similar in the composite image, or rotate the images so that the multiple users appear in the same / similar orientation.

[0105] The image compositing unit (430) can generate a composite image (or composite image sequence) using chroma key images (or chroma key image sequences) that each of multiple users contains. For example, the image compositing unit (430) can generate a composite image by overlaying a first person image (or first person image sequence), generated by removing the area displayed with the background color from a first chroma key image (or first chroma key image sequence) containing a user (host), onto a first area of ​​the background theme, and then overlaying a second person image (or second person image sequence), generated by removing the area displayed with the background color from a second chroma key image (or second chroma key image sequence) containing other users (guests), onto a second area of ​​the background theme. In another example, the image compositing unit (430) can generate a composite image by compositing a part of the background theme onto the background area of ​​the first chroma key image (or first chroma key image sequence) and the background area of ​​the second chroma key image (or second chroma key image sequence). In one embodiment, a chroma key image (or chroma key image sequence) that includes another user (guest) can be assumed to have been received from the other user's terminal.

[0106] The image compositing unit (430) can generate a composite image containing multiple layers such that the image (e.g., a person's image) taken by the user terminal that entered the multi-user photo-taking mode first among the multiple user terminals is placed on the top layer. For example, since the host terminal enters the multi-user photo-taking mode along with the multi-user photo-taking request, the image containing the host can be placed on the top layer. In other examples, the image compositing unit (430) can change the arrangement of layers in accordance with the host's request to change the position of the layers, or arrange layers according to the layer setting rules set in advance by the host.

[0107] The image post-processing unit (440) can apply a user-selected theme and / or effect to the composite image and / or photo booth image. Alternatively, the configuration for applying a user-selected theme and / or effect to the composite image can be performed in the image compositing unit (430). For example, the operation of applying a background or theme to the composite image can be performed by placing the background image on the lowest layer. In yet another example, the step of displaying an effect on the composite image can be performed by placing the effect image on the top layer.

[0108] The internal configuration of the processor (314) shown in Figure 4 is merely illustrative, and in some embodiments, other configurations may be included in addition to the illustrated internal configuration, and some configurations may be omitted. For example, if some of the internal configuration is omitted, the processor (334) of the information processing system and / or other user terminals may be configured to perform the functions of the omitted internal configuration. For example, the processor (334) of the information processing system and / or other user terminals may perform image preprocessing work. Furthermore, although the internal configuration of the processor (314) is described by dividing it into functional units in Figure 4, this does not necessarily mean that it must be physically divided. The image acquisition unit (410), image preprocessing unit (420), image synthesis unit (430), and image postprocessing unit (440) have been described separately above, but this is to aid in understanding the invention and is not limited to these units.

[0109] Figure 5 is a diagram illustrating an example in which a chroma key image (520) for generating a composite image is generated from an image (510) captured by an image sensor of a user terminal according to one embodiment of the present disclosure. The captured image (510) may include a background region (512) and a person region (514). The chroma key image (520) can be generated by recognizing the person region (514) in the captured image (510), and then changing the background region (512), which is the area outside the person region (514), to a specific color. In one embodiment, the person region (514) can be recognized using a pre-trained person recognition model.

[0110] In one embodiment, the color of the background area (522) obtained by excluding the person area (524) from the chroma key image (520) can be determined to be a color whose similarity to the colors contained within the person area (524) is less than a predetermined threshold. For example, the similarity to each of the multiple colors contained within the person area (524) can be calculated, and the color whose calculated similarity is all less than the threshold can be determined as the color of the background area (522).

[0111] In one embodiment, the color of the background area (522) excluding the person area (524) of the chroma key image (520) can be determined as a predetermined first color. For example, it can be determined first whether or not a color with a similarity to the first color above a predetermined threshold is included in the person area (524). If no color with a similarity to the first color above a predetermined threshold is included in the person area (524), then the first color can be determined as the color of the background area (522). On the other hand, if a color with a similarity to the first color above a predetermined threshold is included in the person area (524), then a second color different from the first color can be determined as the color of the background area (522). The second color can be a predetermined color or a color determined based on the colors included in the person area (524).

[0112] In one embodiment, the color of the background area (522) can be determined based on the average color values ​​associated with the person area (524). For example, the RGB values ​​of multiple colors within the person area (524) are calculated, and after the average value of each RGB value is calculated, the color of the background area (522) can be determined based on the calculated average RGB value. For example, the average RGB value can be determined as the color of the background area (522), or an RGB value that is complementary to the average RGB value can be determined as the color of the background area (522).

[0113] In one embodiment, the color of the background area (522) can be determined based on the distance between each of a plurality of color values ​​related to the person area (524) and an arbitrary color value. For example, after calculating statistical values ​​such as the mean, variance, or standard deviation of the distances between each of the plurality of color values ​​and an arbitrary color value, the arbitrary color value with the smallest calculated statistical value can be determined as the color of the background area (522).

[0114] In one embodiment, a predetermined color can be used as the color of the background area (522).

[0115] Figure 6 is a diagram illustrating an example in which a chroma key image (642) is generated in response to user input according to one embodiment of the present disclosure. The first operation (610) illustrates an example in which the user's chroma key image (612) is output on the user terminal's display. The user can check the chroma key image (612) before or after entering the multi-user photo shooting mode to confirm whether the chroma key image (612) is generated / output correctly.

[0116] The chroma key image (612) may include a background area (614) and a person area (616) that are displayed in a first color. The user can check / change the color of the background area (614) by selecting the background color button (618).

[0117] The second action (620) shows an example in which a pop-up window (622) containing a change request message is output on the display in response to the determination that the similarity between the first color of the background area (614) and the colors contained within the person area (616) is above a predetermined threshold. That is, if the similarity between the first color and the colors contained within the person area (616) is above a predetermined threshold, when the colors of the background area are removed, there is a risk that a part of the person area (616) may be removed together or that noise may be generated in the person area (616), so the user can be encouraged to change the colors of the background area (614).

[0118] The user can change the color of the background area (614) by selecting the first button (624). For example, in response to the user selecting the first button (624), the screen for the third action (630) can be displayed on the display. The user can choose not to change the color of the background area (614) by selecting the second button (626).

[0119] In contrast, when it is necessary to change the color of the background area (614), such as when the similarity between the first color and the color contained within the person area (616) is above a predetermined threshold, the pop-up window (622) is not displayed, and the color of the background area (614) is automatically changed to any other color, or a recommended color is displayed, or the color of the background area (614) is automatically changed to any other color in response to selecting the first button (624). When a change is made, the color of the background area (614) can be automatically recommended or changed according to the embodiment described in Figure 5.

[0120] The third operation (630) provides an example of a screen in which the user can select the color of the background area (614) in response to selecting the background color button (618) in the first operation (610), or in response to selecting the first button (624) in the second operation (620). The user can select the new color of the background area (614) by selecting the second color (634) in the color chart (632) and then selecting the third button (636). If the user selects the fourth button (638), the screen of the first operation (610) can be output to the display again.

[0121] The method for selecting the color of the background area (614) is not limited to the methods described above. For example, users can select colors using a wide variety of methods, such as directly inputting RGB values, HSL (Hue, Saturation, and Lightness) values, or selecting specific values ​​in the CIELAB color space.

[0122] The fourth operation (640) provides an example in which, in response to the selection of the third button (636) in the third operation (630), a chroma key image (642) is output on the display in which the background area (644) excluding the person area (646) is displayed in the second color (634). Subsequently, in response to the user selecting the fifth button (648), the screen of the third operation (630) is output on the display again, and the color of the background area (644) can be changed again. Additionally, in response to the determination that the similarity between the second color (634) and the color contained within the person area is above a predetermined threshold, the pop-up window (622) of the second operation (620) can be output again.

[0123] By changing the background region of a chroma key image from a first color to a second color, a chroma key image (642) with a second-color background region can be transmitted to the receiving terminal. In addition, not only the user's chroma key image, but also the background region of other users' chroma key images can be changed from a first color to a second color. Alternatively, only the chroma key image of the user who changed the background region from a first color to a second color can have the background region color changed from a first color to a second color.

[0124] Figure 7 is a diagram illustrating the operation of entering the multi-user photo-taking mode on the host terminal side according to one embodiment of the present disclosure. The first operation (710) shows an example in which, while a video call is in progress, the user selects the multi-user photo-taking mode button on the host terminal, and an area (712) where the user can select the attributes of the photo booth image to be captured and a button (714) for entering the multi-user photo-taking mode are displayed on the host terminal's display. In Figure 7, the area (712) is shown as selecting the number of composite images to be included in the photo booth image, but it is not limited to selection. For example, the host can pre-set the position, size, theme, etc. of each composite image in the area (712).

[0125] The second operation (720) illustrates the entry into multi-user photo-taking mode upon receiving user input by selecting a button (714) on the host terminal. As shown in the diagram, the first area (722) of the display can display the first image sequence captured by the host terminal's image sensor. The second area (724) of the display can display the second image sequence received from the guest terminal. Here, the guest terminal can be the user terminal of another user who is conducting a video call with the host and has entered the multi-user photo-taking mode generated by the host. Additionally, the host can adjust the position, size, etc., of the areas where the first and second image sequences are displayed in the multi-user photo-taking mode.

[0126] In one embodiment, the host can set / change the number of images to be included in the composite image in region (730). For example, the host can select the number "2" in region (730) to set it to take two composite images in succession. Alternatively, the host can select the number "4" in region (730) to set it to take four composite images in succession. In region (730), the host is shown as being able to select one to four images, but is not limited to this, and the host can select any number.

[0127] In one embodiment, when the host selects the multi-user photo shooting button (726), a shooting preparation notification may be displayed. Subsequently, while the shooting notification is displayed, multi-user photo shooting is performed and a composite image (or photo booth image) can be generated. The composite image can be generated on the host terminal using a first image (or first chroma key image) captured using the host terminal's image sensor and a second image (or second chroma key image) received by the host terminal from the guest terminal. The generated composite image can be displayed on the displays of the host terminal and the guest terminal for confirmation between the host and the guest.

[0128] In response to this, when the host selects the multi-user photo shooting button (726), video recording can be initiated and a composite video (or photo booth video) can be generated. Here, the composite video (or photo booth video) may include multiple composite images (or multiple photo booth images). At this time, the composite video can be recorded for a predetermined period of time or for a period of time according to user input (for example, the time the user holds down the multi-user photo shooting button (726)). The generated composite video (or photo booth video) can be saved or played back at a predetermined speed (for example, 2x speed or 3x speed).

[0129] In one embodiment, the host can select the exit button (728) before or after the completion of taking the photo booth image. If the host selects the exit button (728) before the completion of taking the photo booth image, the multi-user photo shooting mode can be terminated on the host terminal. In this case, host privileges can be granted to one of the guests included in the multi-user photo shooting mode (for example, the guest who entered the multi-user photo shooting mode in the order following the host). At this time, after the multi-user photo shooting mode is terminated on the host terminal, the delegation of host privileges can be made after a predetermined time (for example, 5 seconds) has elapsed, and if the host reconnects to the multi-user photo shooting mode before the predetermined time has elapsed, the host privileges can be maintained.

[0130] In contrast, if the host selects the exit button (728) after completing the photo booth image capture, the photo booth image can be saved on the host terminal and sent to the guest terminal via the chat room where the video call took place or a 1:1 chat room with the guest, or transmitted to the guest terminal in the form of metadata in a VoIP (Voice over Internet Protocol) call packet containing data corresponding to the photo booth image.

[0131] Figure 8 is a diagram illustrating the operation of entering a multi-user photo-taking mode on the guest terminal side according to one embodiment of the present disclosure. The first operation (810) shows an example in which, when the guest terminal receives a multi-user photo-taking request from the host terminal during a video call, an area (812) for determining whether to accept or reject the request is displayed on the guest terminal's display.

[0132] In one embodiment, the area (812) can display attribute information of the photo booth image being taken. For example, in Figure 8, information related to the theme of the photo booth image is shown, but it is not limited to information related to the theme; the number, position, size, etc. of the composite image can also be displayed. Additionally, information such as the host of the multi-user photo shooting mode and a list of participating guests can be displayed. Guests can enter the multi-user photo shooting mode by selecting the first button (814) in the area (812), or reject the multi-user photo shooting by selecting the second button (816).

[0133] The second action (820) illustrates an example where the user on the guest terminal selects the first button (814) to enter multi-user photo shooting mode. Unlike Figure 7, instead of displaying the multi-user photo shooting button on the guest terminal's display, a third button (826) can be displayed, which allows the guest to check the theme currently applied to the photo booth image and / or applicable theme information. After selecting the third button (826), the guest can select the theme information they wish to apply to the photo booth image. The selection of theme information will be described in detail later in Figure 9.

[0134] As shown in the diagram, the first area (822) of the display can show the first image sequence (or person image sequence) received from the host terminal. The second area (824) of the display can show the second image sequence (or person image sequence) captured by the guest terminal's image sensor.

[0135] In one embodiment, a guest can exit the multi-user photo-taking mode by selecting the exit button (828) before or after the photo booth image capture is complete. Upon exiting, the multi-user photo-taking mode ends on the guest's terminal, and the video call screen from before entering the multi-user photo-taking mode may be displayed.

[0136] Figure 9 is a diagram illustrating an example of a guest providing feedback on a photo theme according to one embodiment of the present disclosure. The first operation (910) shows an example of a screen displayed on the guest terminal's display when the guest terminal enters a multi-user photo shooting mode.

[0137] The second action (920) provides an example in which, after the guest terminal enters multi-user photo shooting mode and before the shooting notification is displayed, the guest selects a button (912) to display multiple photo themes that can be applied to the photo booth image / composite image. Thereafter, the guest can provide feedback (e.g., positive or negative feedback) to a particular photo theme by selecting at least one of the visual objects (922, 924, 926) displayed on the preview (e.g., thumbnail) of the multiple photo themes.

[0138] The third action (930) provides an example in which, in response to a guest selecting a visual object (924) and providing feedback (or positive feedback) to a particular photo theme, a visual object (934) indicating the feedback is displayed near the selected photo theme. The visual object (934) displayed to the selected photo theme as a result of the guest providing feedback differs from other visual objects (932, 936) to which feedback has not been provided, allowing the guest to easily identify the photo themes to which they have provided feedback.

[0139] Additionally, the host can see the number of feedback ratings given by guests to a photo theme through their device. Through this configuration, the host can incorporate guest feedback when selecting a photo theme to apply to a photo booth image / composite image. On the other hand, the photo theme with the most positive feedback ratings can be automatically applied to the photo booth image / composite image, or the host may be provided with a message suggesting they change to the photo theme with the most positive feedback ratings.

[0140] After the host selects a photo theme to apply to the photo booth image / composite image, the first image sequence captured by the host terminal's image sensor (or a sequence of people with the background removed from the chroma key image sequence) can be displayed in the first area of ​​the selected photo theme on the display. Additionally, the second image sequence captured by the guest terminal's image sensor (or a sequence of people with the background removed from the chroma key image sequence) can be displayed in the second area of ​​the selected photo theme on the display.

[0141] In one embodiment, the photo theme can include a pose guide. Here, the pose guide can be a visual guide displayed on the screen to allow users to easily imitate specific poses. For example, in the first area of ​​the selected photo theme, a first pose guide can be overlaid on the first image sequence, and in the second area, a second pose guide can be overlaid on the second image sequence. Through this configuration in which the pose guides are overlaid, users can be guided to take photos in a variety of poses, and users can take composite images with high-quality compositions without having to plan poses themselves.

[0142] Figure 10 is a diagram illustrating an example of a photo booth image being displayed according to one embodiment of the present disclosure. In the first to fifth examples (1010, 1020, 1030, 1040, 1050) of Figure 10, the multiple users included in each grid of the photo booth image can correspond to users who have entered the multi-user photo shooting mode. In Figure 10, each grid is shown to contain one user, but it is not limited to one user. The first to fifth examples (1010, 1020, 1030, 1040, 1050) are examples where there are two to six users each who have entered the multi-user photo shooting mode.

[0143] Photo booth images can be generated by combining images containing each of multiple users, and the relative position and size of the grid can change each time a user is added or removed from the multi-user photo-taking mode. For example, in the photo booth image of the second example (1020), it can be seen that the top grid is moved to the left from its existing position and its size is halved after a user is added in the third example (1030). The position of the grid or users within the photo booth image can be determined based on predetermined placement rules and the number of users in the multi-user photo-taking mode, or it can be changed based on user input from the host to change the position of the grid or each user.

[0144] In contrast, the photo booth image / composite image can contain multiple layers, with the image / image sequence of the first user to enter the multi-person photo shoot mode placed on the top layer. In this case, the composite image can be generated by combining images containing each of the multiple users, and the relative position or size (layout) of the users within the composite image can change each time a user is added or removed from the multi-person photo shoot mode.

[0145] The position or size of users within a composite image can be determined based on predetermined user placement rules and the number of users in multi-user photo-taking mode, or it can be changed based on user input from the host to modify the position or size of each user.

[0146] Figure 11 is a diagram showing a video call group (1100) related to a video call and subgroups (1110, 1120, 1130) related to a multi-user photo shooting mode according to one embodiment of the present disclosure. When a video call is initiated within a chat room of an instant messaging application, users participating in the video call can join the video call group (1100). Some or all of the users in the chat room can join the video call group (1100), and user-to-user image sequences can be sent and received within the video call group (1100).

[0147] Thereafter, if a user enters multi-user photo shooting mode while a video call is in progress, the user can enter one of the subgroups (1110, 1120, 1130) related to multi-user photo shooting within the video call group (1100). That is, while a single video call is in progress, multiple subgroups (1110, 1120, 1130) for generating composite images can be created, and a composite image including the users who have entered each subgroup (1110, 1120, 1130) can be generated.

[0148] The maximum number of users who can participate in each subgroup (1110, 1120, 1130) (or multi-user photo shoot) can be less than the maximum number of users who can participate in the video call group (1100) (or video call). In one embodiment, the maximum number of users who can participate in each subgroup can be set to a predetermined number of people (for example, 4 people).

[0149] In one embodiment, a user can choose whether or not to join subgroups (1110, 1120, 1130). For example, a user can receive a request for a photo shoot with multiple users from a host terminal that created a specific subgroup, and by accepting the request, can join that specific subgroup.

[0150] In one embodiment, a user can select which subgroup to join from among several subgroups (1110, 1120, 1130). For example, during a video call, while a multi-user photo-taking mode is in progress in each of the subgroups (1110, 1120, 1130), the user can select which of the subgroups (1110, 1120, 1130) to join. That is, instead of the user performing the host role by creating a new subgroup, they can select and join one of the subgroups that other users are already participating in. For example, the user's terminal may output information related to the subgroups that other users are participating in (such as the time the multi-user photo-taking mode in the subgroup was in progress, and the number of participants), and the user can select one of the subgroups after reviewing the information related to the subgroups.

[0151] If a user attempts to join a specific subgroup but the number of users already in that subgroup exceeds a predetermined limit (e.g., 4 people), the user can enter a photo viewing mode instead of photo taking mode. In other words, a user in photo viewing mode can observe other users taking photos with multiple users, but cannot participate in taking photos with multiple users themselves.

[0152] In contrast, if the number of users participating in a particular subgroup is below a predetermined number, a newly joined user can enter multi-user photo-taking mode. At this time, the first area of ​​the display can show the first image sequence captured by the image sensors of the user terminals already participating, and the second image sequence received from the newly joined user terminal can be displayed in the second area of ​​the display.

[0153] Figure 12 is a flowchart showing the process of generating a composite image according to one embodiment of the present disclosure. After a video call is initiated between a host terminal (1210) associated with a first user and a guest terminal (1220) associated with a second user, both included in a chat room of an instant messaging application, the host terminal (1210) can enter multi-user photo shooting mode (1212). For example, the host terminal can enter multi-user photo shooting mode by receiving user input, such as touch input, to select a button to enter multi-user photo shooting mode.

[0154] When the host terminal (1210) enters multi-user photo shooting mode (1212), the host terminal (1210) can send a multi-user photo shooting request (1214) to the guest terminal (1220). Subsequently, if the guest terminal (1220) receives user input agreeing to the multi-user photo shooting request, it can enter multi-user photo shooting mode (1222). Before entering multi-user photo shooting mode, the host terminal (1210) and / or the guest terminal (1220) may display the user's chroma key image.

[0155] In multi-user photo shooting mode, the guest chroma key image sequence can be transmitted to the host terminal (1210) (1224), and the host chroma key image sequence can be transmitted to the guest terminal (1220) (1216). Using the guest chroma key image sequence and the host chroma key image sequence, a preview screen displaying the composite image sequence can be output to the displays of the host terminal (1210) and the guest terminal (1220).

[0156] Subsequently, the host terminal (1210) can receive user input from the user, indicating that the user has clicked the capture button (1218). Upon receiving user input indicating that the host terminal (1210) has clicked the capture button, the host terminal (1210) can send a capture signal to the guest terminal (1220) (1230).

[0157] In response to the host terminal (1210) receiving user input such as clicking the capture button, the host terminal (1210) can capture an image including the host using the image sensor (1232). Similarly, upon receiving a capture signal from the host terminal (1210), the guest terminal (1220) can capture an image including the guest using the image sensor (1226). The host terminal (1210) and the guest terminal (1220) can generate a chroma key image from the captured images.

[0158] Thereafter, the host terminal (1210) can receive a chroma key image containing the guest from the guest terminal (1220) (1228).

[0159] Thereafter, the host terminal (1210) can generate a composite image using a chroma key image containing the host and a chroma key image containing the guest (1234). At this time, a background theme can be used to create the composite image.

[0160] Thereafter, the generated composite image can be transmitted to the guest terminal (1220) (1236). In one embodiment, the composite image can be transmitted to the guest terminal (1220) through a chat room (e.g., a group chat room or a 1:1 chat room) that includes the host and the guest. Additionally or alternatively, the composite image can be transmitted to the guest terminal (1220) using VoIP (Voice over Internet Protocol). For example, after metadata for a VoIP call packet containing data corresponding to the composite image has been generated, the generated metadata can be transmitted to the guest terminal (1220). In this case, the generated metadata can be metadata within the protocol of a VoIP call channel linked between the host and the guest.

[0161] Figure 13 is a flowchart of a method (1300) for capturing images during a video call according to one embodiment of the present disclosure. The method (1300) can be executed by at least one processor of a first user terminal.

[0162] The processor can initiate a video call between a first user terminal associated with a first user and a second user terminal associated with a second user, both included in a chat room of an instant messaging application (S1310).

[0163] Subsequently, the processor can enter multi-user photo shooting mode while the video call is in progress (S1320). For example, the processor can enter a specific subgroup among several subgroups related to multi-user photo shooting. If it does, the second user can be a user included in the same specific subgroup as the first user. The maximum number of users that can participate in multi-user photo shooting can be less than the maximum number of users that can participate in the video call.

[0164] In one embodiment, after entering a multi-user photo-taking mode, the processor can display multiple background themes on the display of the first user terminal and receive input from the first user to select one of the multiple background themes from the third user.

[0165] In one embodiment, after entering the multi-user photo-taking mode and before the image captured by the image sensor of the first user terminal is captured, the processor can display the first image sequence captured by the image sensor of the first user terminal in the first area of ​​the selected background theme on the display, and display the second image sequence received from the second user terminal in the second area of ​​the selected background theme on the display. At this time, the first and second image sequences can be generated by removing the color from the background area of ​​a chroma key image sequence.

[0166] Subsequently, in the multi-user photo-taking mode, the processor can generate a first chroma-key image based on the image captured by the image sensor of the first user terminal (S1330). At this time, the first chroma-key image can be an image in which the background area, excluding the person area from the captured image, has been changed to a first color.

[0167] The processor uses a person recognition model to recognize the area containing people in the captured image, and generates a first chroma key image by changing the background area, which is the area outside the recognized area containing people, to a first color.

[0168] In one embodiment, the processor can change the background area to the first color in response to receiving a first user input for selecting a first color.

[0169] In one embodiment, the processor determines that the similarity between a predetermined second color and a color contained within the person area is greater than or equal to a predetermined threshold, and determines a first color whose similarity to the color contained within the person area is less than the predetermined threshold, thereby changing the background area to the first color. At this time, the first color can be different from the second color.

[0170] In one embodiment, the processor, in response to determining that the similarity between a predetermined second color and the color contained within the person area is greater than or equal to a predetermined threshold, outputs a message requesting a change of the second color on the display of the first user terminal, and can change the background area to the first color by receiving a second user input to change the second color to the first color.

[0171] In one embodiment, the processor calculates the average color value associated with the person area and determines a first color based on the average color value, thereby changing the background area to the first color.

[0172] In one embodiment, the processor calculates the distance between each of several color values ​​related to the person area and an arbitrary color value, and determines a first color based on the calculated distance, thereby changing the background area to the first color.

[0173] Subsequently, the processor can transmit the first chroma key image to the second user terminal (S1340).

[0174] Subsequently, the processor can output a composite image, created using a first chroma key image containing the first user and a second chroma key image containing the second user, to the display of the first user's terminal.

[0175] After entering multi-user photo-taking mode, the processor can receive a fourth user input from the first user selecting the multi-user photo-taking button, receive a second chroma key image including the second user from the second user's terminal, and generate a composite image using the first and second chroma key images. At this time, the first chroma key image can be generated in response to receiving the fourth user input.

[0176] In one embodiment, the processor displays multiple background themes on the display of a first user terminal, receives a fifth user input from the first user to select one of the multiple background themes, generates a first person image by removing the area displayed in a first color from a first chroma key image, generates a second person image by removing the area displayed in the background area's color from a second chroma key image, and generates a composite image by overlaying the first and second person images onto the selected background theme. For example, the processor can overlay the first person image onto the first area of ​​the selected background theme and overlay the second person image onto the second area of ​​the selected background theme.

[0177] In another embodiment, the processor can display multiple background themes on the display of the first user terminal, receive a sixth user input from the first user to select one of the multiple background themes, and generate a composite image by compositing a portion of the selected background theme into the background area of ​​the first chroma key image and the background area of ​​the second chroma key image.

[0178] In one embodiment, the processor can transmit a composite image to the second user's terminal through a chat room containing both the first and second users. In another embodiment, the processor can generate metadata for a VoIP (Voice over Internet Protocol) call packet containing data corresponding to the composite image and transmit the metadata to the second user participating in the video call. In this case, the metadata can be metadata within the protocol of a VoIP call channel linked to the first and second users participating in the video call.

[0179] The flowcharts illustrated in Figures 12 and 13 and the above-described explanation are merely illustrative examples, and other embodiments may be implemented differently. For example, the order of some steps may be changed, some steps may be omitted, other configurations may be added, and some steps described as being executed sequentially may be executed simultaneously.

[0180] The methods described above can be provided as computer programs stored on computer-readable recording media for execution on a computer. The media may continuously store computer-executable programs or temporarily store them for execution or download. Furthermore, the media can be a variety of recording or storage means, often consisting of a single or multiple hardware components, and is not limited to media directly connected to a computer system; it may also be distributed across a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media including ROM, RAM, and flash memory configured to store program instructions. Other examples of media include recording or storage media managed by application stores and other sites, servers, etc., that supply or distribute various software.

[0181] The methods, operations, or techniques described herein can be implemented by a variety of means. For example, techniques can be implemented in hardware, firmware, software, or a combination of hardware, firmware, and software. A person of ordinary skill will understand that the various exemplary logical blocks, modules, circuits, and algorithmic steps described in conjunction with the disclosure may also be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been generally described above in terms of their functional aspects. Whether a function is implemented as hardware or software depends on the design requirements imposed on the particular application and the overall system. A person of ordinary skill may implement the functions described in a variety of ways for each specific application, but such implementation should not be construed as excluding the scope of this disclosure.

[0182] In hardware implementation, the processing unit used to perform the technique may be implemented in one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, computers, or combinations thereof.

[0183] Accordingly, the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure may be implemented or run by general-purpose processors, DSPs, ASICs, FPGAs and other programmable logic devices, discrete gates and transistor logic, discrete hardware components, or any combination of those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, a processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented by a combination of computing devices, such as a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other combination of configurations.

[0184] In firmware and / or software implementations, the technique may be implemented using instruction words stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic data storage device, or optical data storage device. The instruction words may be executable by one or more processors, causing the processors to perform specific aspects of the functions described herein.

[0185] Although the embodiments described above are described as utilizing aspects of the subject matter of this disclosure in one or more standalone computer systems, this disclosure is not limited to utilizing aspects of the subject matter of this disclosure in one or more standalone computer systems, but may also be implemented in conjunction with any computing environment, such as a network or a distributed computing environment. Furthermore, aspects of the subject matter in this disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Devices may include PCs, network servers, and portable devices.

[0186] While this disclosure has been described in relation to some embodiments, various modifications and alterations can be made, provided that they do not exceed the scope of this disclosure as understandable to a person with ordinary skill in the art to which the inventions of this disclosure pertain. Furthermore, such modifications and alterations should be considered to fall within the scope of the claims appended to this specification.

Claims

1. In a method for capturing images during a video call, which is performed by at least one processor of a first user terminal, The steps include initiating a video call between a first user terminal associated with a first user and a second user terminal associated with a second user, both of which are included in a chat room of an instant messaging application, During the aforementioned video call, the step of entering multi-user photo shooting mode, In the aforementioned multi-user photo shooting mode, the steps include generating a first chroma key image based on an image captured by the image sensor of the first user terminal, The step includes transmitting the first chroma key image to the second user terminal, The first chroma key image is an image obtained by changing the background area, which is the same as the captured image but with the person area removed, to a first color, in a method for capturing an image during a video call.

2. The method for capturing an image during a video call according to claim 1, further comprising the step of outputting a composite image, which is synthesized using the first chroma key image containing the first user and the second chroma key image containing the second user, onto the display of the first user terminal.

3. The step of generating the first chroma key image is: A step of recognizing the area of ​​the person in the captured image using a person recognition model, A method for capturing an image during a video call according to claim 1, comprising the step of changing the background region, which is a region outside the recognized person region in the image, to a first color.

4. The step of changing the background area to the first color is: The method for capturing an image during a video call according to claim 3, comprising the step of changing the background area to the first color in response to receiving a first user input for selecting the first color.

5. The step of changing to the first color is: The steps include determining whether the degree of similarity between a predetermined second color and the color contained within the person area is greater than or equal to a predetermined threshold, The step includes determining a first color whose similarity to the colors included within the person area is less than the predetermined threshold, The method for capturing an image during a video call according to claim 3, wherein the first color is different from the second color.

6. The step of changing to the first color is: In response to determining that the similarity between a predetermined second color and the color contained within the person area is greater than or equal to a predetermined threshold, the steps include outputting a message requesting a change of the second color on the display of the first user terminal, A method for capturing an image during a video call according to claim 3, comprising the step of receiving a second user input for changing the second color to the first color.

7. The step of changing to the first color is: The steps include calculating the average color value related to the aforementioned human area, A method for capturing an image during a video call according to claim 3, comprising the step of determining the first color based on the average color value.

8. The step of changing to the first color is: The steps include calculating the distance between each of the multiple color values ​​related to the aforementioned human region and an arbitrary color value, A method for capturing an image during a video call according to claim 3, comprising the step of determining the first color based on the calculated distance.

9. After entering the aforementioned multi-user photo shooting mode, The steps include displaying multiple background themes on the display of the first user terminal, The method for capturing an image during a video call according to claim 1, further comprising the step of receiving a third user input from the first user to select one of the plurality of background themes.

10. After entering the multi-user photo-taking mode, and before the image captured by the image sensor of the first user terminal is captured, the step of displaying a first image sequence captured by the image sensor of the first user terminal in the first area of ​​the selected background theme on the display, The method further includes the step of displaying a second image sequence received from the second user terminal in a second area of ​​the selected background theme on the display, The method for capturing images during a video call according to claim 9, wherein the first image sequence and the second image sequence are generated by removing the color from the background region from a chroma key image sequence.

11. After entering the multi-user photo shooting mode, The steps include receiving input from the first user indicating the selection of a multi-user photo shooting button, The steps include receiving a second chroma key image containing the second user from the second user terminal, The method further includes the step of generating a composite image using the first chroma key image and the second chroma key image, The method for capturing an image during a video call according to claim 1, wherein the first chroma key image is generated in response to receiving the fourth user input.

12. The step of generating the aforementioned composite image is: The steps include displaying multiple background themes on the display of the first user terminal, The steps include receiving a fifth user input from the first user to select one of the multiple background themes, A step of generating a first person image by removing the area displayed in the first color from the first chroma key image, The steps include generating a second person image by removing the area displayed with the background color from the second chroma key image, A method for capturing images during a video call according to claim 11, comprising the step of overlaying the first person image and the second person image onto the selected background theme.

13. The step of overlaying onto the selected background theme is: The steps include overlaying the first person image onto the first region of the selected background theme, A method for capturing an image during a video call according to claim 12, comprising the step of overlaying the second person image onto a second area of ​​the selected background theme.

14. The step of generating the aforementioned composite image is: The steps include displaying multiple background themes on the display of the first user terminal, The steps include receiving a sixth user input from the first user to select one of the multiple background themes, A method for capturing an image during a video call according to claim 11, comprising the step of compositing a portion of the selected background theme onto the background region of the first chroma key image and the background region of the second chroma key image.

15. The method for capturing an image during a video call according to claim 11, further comprising the step of transmitting the composite image to the second user's terminal through a chat room containing the first user and the second user.

16. The steps include generating metadata for a VoIP (Voice over Internet Protocol) call packet that includes data corresponding to the composite image, The further step includes transmitting the metadata to the second user who participated in the video call, The method for capturing an image during a video call according to claim 11, wherein the metadata is metadata within the protocol of a VoIP call channel linked for the first user and the second user who participated in the video call.

17. The step of entering the multi-user photo shooting mode is: This includes the step of entering a specific subgroup among several subgroups related to multi-user photo shooting, The method for capturing images during a video call according to claim 1, wherein the second user is included in the same specific subgroup as the first user.

18. The method for capturing images during a video call according to claim 1, wherein the maximum number of users who can participate in taking photos with multiple users is less than the maximum number of users who can participate in the video call.

19. A computer-readable non-temporary recording medium that stores instruction words for executing the method described in claim 1 on a computer.

20. On the first user terminal, Communication module and The display and Image sensor and Memory and The system includes at least one processor connected to the memory and configured to execute at least one computer-readable program contained in the memory, The at least one program is A video call is initiated between the first user terminal associated with the first user and the second user terminal associated with the second user, both included in the chat room of an instant messaging application. During the aforementioned video call, enter multi-user photo shooting mode. In the multi-user photo-taking mode, a first chroma key image is generated based on the image captured by the image sensor of the first user terminal. Includes a command for transmitting the first chroma key image to the second user terminal, The first chroma key image is a first user terminal in which the background area, with the person area excluded from the captured image, has been changed to a first color.

Images