Method and system for image capturing during video call

Abstract

The present disclosure relates to a method for image capturing during a video call, the method being performed by at least one processor of a first user terminal. The method for image capturing during a video call comprises the steps of: initiating a video call between a first user terminal associated with a first user and a second user terminal associated with a second user, the first and second users being included in a chat room of an instant messaging application; during the video call, entering a multi-party photographing mode; in the multi-party photographing mode, generating a first chroma-key image on the basis of an image captured by an image sensor of the first user terminal; and transmitting the first chroma-key image to the second user terminal, wherein the first chroma-key image is an image in which a background area obtained by excluding a person area from the captured image is changed to a first color.

Description

Method and system for capturing images during a video call

[0001] The present disclosure relates to a method and system for capturing an image during a video call, and more particularly, to a method and device for generating a chroma key image based on an image captured by an image sensor in a multi-party photo shooting mode, and outputting a generated composite image by transmitting the generated chroma key image to another user terminal.

[0002] The recent proliferation of mobile devices like smartphones and the advancement of the internet have led to the widespread use of instant messaging applications that enable voice and video calls between multiple users. Meanwhile, demand for so-called "photo booths," which provide offline printed photos of multiple people together, is steadily increasing.

[0003] However, the image capture function offered by existing video call services only captures specific moments during a call, making it difficult to provide users with images with diverse formats, concepts, and photo themes. Therefore, compared to photo booths, existing image capture services struggle to provide users with satisfactory images.

[0004] Furthermore, even if the original images captured by each user terminal are transmitted to a specific user terminal and a composite image is generated there to provide high-quality images, transmitting and receiving the original captured images not only requires high bandwidth, but also consumes significant resources and time for image processing. Therefore, there is a need for a method that can achieve high-quality images while reducing the consumption of communication and computing resources.

[0005] The present disclosure provides a method for capturing an image during a video call, a computer-readable non-transitory recording medium recording commands, and a system (device) for solving the above-mentioned problem.

[0006] The present disclosure can be implemented in various ways, including a computer-readable, non-transitory recording medium having recorded thereon a method, system (device), or instructions.

[0007] A method for capturing an image during a video call, performed by at least one processor of a first user terminal according to one embodiment of the present disclosure, comprises the steps of: initiating a video call between a first user terminal associated with a first user included in a chat room of an instant messaging application and a second user terminal associated with a second user; entering a multi-party photo capturing mode during the video call; generating a first chroma-key image based on an image captured by an image sensor of the first user terminal in the multi-party photo capturing mode; and transmitting the first chroma-key image to the second user terminal, wherein the first chroma-key image is an image in which a background area excluding a person area in the captured image is changed to a first color.

[0008] In one embodiment of the present disclosure, the method further includes a step of outputting a composite image synthesized using a first chroma key image including a first user and a second chroma key image including a second user on a display of a first user terminal.

[0009] In one embodiment of the present disclosure, the step of generating a first chroma key image includes the step of recognizing a person area in a photographed image using a person recognition model, and the step of changing a background area, which is an area other than the recognized person area in the image, to a first color.

[0010] In one embodiment of the present disclosure, the step of changing the background area to a first color includes the step of changing the background area to the first color in response to receiving a first user input selecting the first color.

[0011] In one embodiment of the present disclosure, the step of changing to a first color includes a step of determining that a similarity between a predetermined second color and a color included in a person area is greater than or equal to a predetermined threshold, and a step of determining a first color that has a similarity between the predetermined second color and a color included in the person area less than the predetermined threshold, wherein the first color is different from the second color.

[0012] In one embodiment of the present disclosure, the step of changing to a first color includes the step of outputting a request message for changing the second color on a display of a first user terminal in response to determining that a similarity between a predetermined second color and a color included in a person area is greater than a predetermined threshold, and the step of receiving a second user input for changing the second color to the first color.

[0013] In one embodiment of the present disclosure, the step of changing to a first color includes the step of calculating an average color value associated with a person area and the step of determining the first color based on the average color value.

[0014] In one embodiment of the present disclosure, the step of changing to a first color includes the step of calculating a distance between each of a plurality of color values ​​associated with a person area and an arbitrary color value, and the step of determining the first color based on the calculated distance.

[0015] In one embodiment of the present disclosure, after entering a multi-person photo shooting mode, the method further includes the step of displaying a plurality of background themes on a display of a first user terminal and receiving a third user input for selecting one of the plurality of background themes from the first user.

[0016] In one embodiment of the present disclosure, after entering a multi-person photo shooting mode and before capturing an image captured by an image sensor of a first user terminal, the method further includes the step of displaying a first image sequence captured by an image sensor of a first user terminal in a first area of ​​a selected background theme on a display and the step of displaying a second image sequence received from a second user terminal in a second area of ​​the selected background theme on the display, wherein the first image sequence and the second image sequence are generated by removing a color of a background area from a chroma key image sequence.

[0017] In one embodiment of the present disclosure, after entering a multi-person photo shooting mode, the method further includes the steps of: receiving a fourth user input of selecting a multi-person photo shooting button from a first user; receiving a second chroma key image including a second user from a second user terminal; and generating a composite image using the first chroma key image and the second chroma key image, wherein the first chroma key image is generated in response to receiving the fourth user input.

[0018] In one embodiment of the present disclosure, the step of generating a composite image includes the steps of displaying a plurality of background themes on a display of a first user terminal, receiving a fifth user input for selecting one of the plurality of background themes from the first user, generating a first person image by removing an area displayed in a first color from a first chroma key image, generating a second person image by removing an area displayed in a color of a background area from a second chroma key image, and overlaying the first person image and the second person image on the selected background theme.

[0019] In one embodiment of the present disclosure, the step of overlaying on a selected background theme includes the step of overlaying a first person image on a first area of ​​the selected background theme and the step of overlaying a second person image on a second area of ​​the selected background theme.

[0020] In one embodiment of the present disclosure, the step of generating a composite image includes the steps of displaying a plurality of background themes on a display of a first user terminal, receiving a sixth user input for selecting one of the plurality of background themes from the first user, and synthesizing a part of the selected background theme into a background area of ​​a first chroma key image and a background area of ​​a second chroma key image.

[0021] In one embodiment of the present disclosure, the method further includes a step of transmitting a composite image to a second user terminal through a chat room including a first user and a second user.

[0022] In one embodiment of the present disclosure, the method further includes a step of generating metadata of a Voice over Internet Protocol (VoIP) call packet including data corresponding to a composite image and a step of transmitting the metadata to a second user who participated in the video call, wherein the metadata is metadata within a protocol of a VoIP call channel connected between the first user and the second user who participated in the video call.

[0023] In one embodiment of the present disclosure, the step of entering a multi-person photo shooting mode includes the step of entering a specific sub-group among a plurality of sub-groups associated with multi-person photo shooting, wherein the second user is included in the same specific sub-group as the first user.

[0024] In one embodiment of the present disclosure, the maximum number of users that can participate in multi-party photo shooting is less than the maximum number of users that can participate in a video call.

[0025] A computer-readable non-transitory recording medium having recorded thereon instructions for executing the above-described method according to one embodiment of the present disclosure is provided.

[0026] In one embodiment of the present disclosure, a first user terminal comprises a communication module, a display, an image sensor, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program comprises commands for initiating a video call between a first user terminal associated with a first user included in a chat room of an instant messaging application and a second user terminal associated with a second user, entering a multi-party photo capturing mode during the video call, generating a first chroma key image based on an image captured by an image sensor of the first user terminal in the multi-party photo capturing mode, and transmitting the first chroma key image to the second user terminal, wherein the first chroma key image is an image in which a background area excluding a person area in the captured image is changed to a first color.

[0027] In various embodiments of the present disclosure, a user can check whether the user's chroma key image is generated normally before generating a composite image using the chroma key image.

[0028] In various embodiments of the present disclosure, a multi-person photo shooting preview screen displays a composite image sequence by synthesizing image sequences captured by image sensors of each user terminal, thereby allowing the user to prepare in advance the user's position, pose, movement, etc. to be displayed in the composite image.

[0029] In various embodiments of the present disclosure, it is possible to reduce the bandwidth required for image transmission and reception for generating a synthetic image or the time / resources required for communication.

[0030] In various embodiments of the present disclosure, a high resolution composite image can be generated by capturing / transmitting an image used to generate a composite image at a high resolution.

[0031] In various embodiments of the present disclosure, images with various photographic themes applied may be provided.

[0032] In various embodiments of the present disclosure, images can be generated as if they were taken in a single space even if multiple users take pictures in different environments.

[0033] In various embodiments of the present disclosure, the host can check the number of feedbacks given by the guest(s) to the photo theme through the host terminal, so that the guest's opinion can be reflected when selecting a photo theme to apply to the photo booth image.

[0034] In various embodiments of the present disclosure, since users assume poses according to pose guides displayed on the display, even if the timing of taking photos of the users is somewhat different, the users' poses may not be out of alignment.

[0035] The effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present disclosure belongs (hereinafter referred to as “ordinary skilled person”) from the description of the claims.

[0036] Embodiments of the present disclosure will be described below with reference to the accompanying drawings, wherein like reference numerals represent similar elements, but are not limited thereto.

[0037] FIG. 1 is a drawing showing an example of a photo booth image including multiple users being displayed on a display of a user terminal according to one embodiment of the present disclosure.

[0038] FIG. 2 is a schematic diagram showing a configuration in which an information processing system is connected to enable communication with multiple user terminals in order to provide an image capturing service during a video call according to one embodiment of the present disclosure.

[0039] FIG. 3 is a block diagram showing the internal configuration of a user terminal and an information processing system according to one embodiment of the present disclosure.

[0040] FIG. 4 is a block diagram showing the internal configuration of a processor of a user terminal according to one embodiment of the present disclosure.

[0041] FIG. 5 is a diagram illustrating an example of generating a chroma key image for generating a composite image from an image captured by an image sensor of a user terminal according to one embodiment of the present disclosure.

[0042] FIG. 6 is a diagram illustrating an example of generating a chroma key image in response to a user input according to one embodiment of the present disclosure.

[0043] FIG. 7 is a diagram illustrating an operation of entering a multi-party photo shooting mode on the host terminal side according to one embodiment of the present disclosure.

[0044] FIG. 8 is a diagram illustrating an operation of entering a multi-party photo shooting mode on a guest terminal side according to one embodiment of the present disclosure.

[0045] FIG. 9 is a diagram illustrating an example of a guest providing feedback on a photo theme according to one embodiment of the present disclosure.

[0046] FIG. 10 is a drawing showing an example of displaying a photo booth image according to one embodiment of the present disclosure.

[0047] FIG. 11 is a diagram illustrating a video call group associated with a video call and a subgroup associated with a multi-party photo shooting mode according to one embodiment of the present disclosure.

[0048] FIG. 12 is a flowchart showing a process for generating a composite image according to one embodiment of the present disclosure.

[0049] FIG. 13 is a flowchart illustrating a method for capturing an image during a video call according to one embodiment of the present disclosure.

[0050] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the attached drawings. However, in the following description, specific descriptions of widely known functions or configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0051] In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0052] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below, along with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure the completeness of the disclosure and to fully inform those skilled in the art of the scope of the invention.

[0053] The terms used in this specification will be briefly explained, followed by a detailed description of the disclosed embodiments. The terms used in this specification have been selected from widely used, current terms, taking into account the functions of the present disclosure. However, these terms may vary depending on the intentions of engineers working in the relevant field, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on their meanings and the overall content of the present disclosure.

[0054] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0055] Also, the term 'module' or 'part' used in the specification means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. The functionality provided within the components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0056] According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0057] In this disclosure, "user" may refer to a user of an instant messaging application within a chat room of the instant messaging application, or a user engaged in a video call with another user within the chat room. Alternatively, "user" may refer to an image or a portion of an image displaying such a user.

[0058] In this disclosure, a "user account" may refer to an account or data related to an account created and used by a user in an instant messaging application. Furthermore, a user account in an instant messaging application may refer to a user using the instant messaging application. Similarly, a user using instant messaging or a chat room that supports instant messaging may refer to a user account in an instant messaging application.

[0059] In the present disclosure, a "chat room" may refer to a virtual space or group in which one or more users (or user accounts) can participate, such as may be created in an instant messaging application installed on a computing device. For example, one or more user accounts may participate in or be included in a chat room and exchange various types of messages, files, etc. In addition, the chat room may provide a VoIP (Voice over Internet Protocol) voice call function, a VoIP video call function, a live broadcast function (VoIP real-time video transmission function), and a multimedia content creation function, so that voice calls, video calls, video streaming, and multimedia content transmission between user accounts may be performed.

[0060] In the present disclosure, a 'host' may refer to a user who requests multi-party photo taking among users in a video call or a user who has been given host authority from an existing host, and a 'host terminal' may refer to a user terminal associated with a host.

[0061] In the present disclosure, a 'guest' may refer to a user who has entered a multi-person photo shooting mode by accepting a host's request for multi-person photo shooting, and a 'guest terminal' may refer to a user terminal associated with a guest.

[0062] In the present disclosure, unless otherwise specified, a 'user' may refer to a host and / or a guest, and a 'user terminal' may refer to a host terminal and / or a guest terminal.

[0063] In the present disclosure, a 'photobooth image' may refer to a final result image of a multi-photo shooting mode that includes one or more composite images.

[0064] FIG. 1 is a diagram illustrating an example of a photo booth image (152) including multiple users displayed on a display of a user terminal according to an embodiment of the present disclosure. A first operation (110) may indicate a video call screen (114) being performed between multiple user terminals associated with multiple users. In each of the multiple user terminals, a screen including each user may be displayed at any location and size on the display, and the video call screen (114) may not be displayed identically on all user terminals. In one embodiment, a video call may be initiated and performed between each of multiple user terminals associated with multiple users included in a chat room of an instant messaging application.

[0065] Although FIG. 1 depicts four users making a video call, the present invention is not limited thereto, and any number of users (e.g., six) who are at least a part of the users included in the chat room of the instant messaging application (e.g., eight) may make a video call simultaneously.

[0066] In one embodiment, during a video call, an image sequence captured by an image sensor of a user terminal participating in the video call may be transmitted to another user terminal participating in the video call. The video call screen (114) may be a screen that displays an image sequence captured by the image sensor of the user terminal and an image sequence received from another user terminal.

[0067] In the first operation (110), one of the multiple users in a video call may enter the multi-person photo shooting mode by selecting the multi-person photo shooting mode entry button (112) by touch input or the like. Thereafter, a multi-person photo shooting request may be sent to other users in the video call. The other users in the video call may enter the multi-person photo shooting mode by agreeing to the multi-person photo shooting request. The specific process of entering the multi-person photo shooting mode will be described in detail later with reference to FIGS. 7 and 8 .

[0068] The second operation (120) illustrates an example in which a chroma key image (122) associated with a user is output on the display before taking a multi-person photo. The chroma key image (122) may be generated based on an image captured by an image sensor of the user terminal. Through this, the user can check whether the user's chroma key image (122) is normally generated before creating a composite image using the chroma key image. After determining that the chroma key image (122) is normally generated and output, the user can enter the multi-person photo taking mode by selecting the entry button (128). Alternatively, the user may enter the multi-person photo taking mode after a predetermined period of time has elapsed without selecting the entry button (128).

[0069] A chroma key image (122) can be generated by changing the background area (124) of a captured image, excluding the person area (126), to a specific color. For example, the person area (126) can be recognized using a pre-trained person recognition model that distinguishes between the person and the background in the image, and the background area (124) excluding the person area (126) can be changed to a specific color. The specific process of determining and changing the color of the background area (124) will be described in detail later with reference to FIGS. 5 and 6.

[0070] Unlike that illustrated in FIG. 1, the screen of the second operation (120) may not be displayed on the user terminal. That is, in response to selecting the multi-person photo shooting mode entry button (112) in the first operation (110), the chroma key image (122) may not be output on the display, and the screen of the third operation (130) may be output directly on the display.

[0071] The third operation (130) illustrates an example in which, after multiple user terminals enter a multi-person photo shooting mode, a composite image (142) comprising an image including a first user (indicated as user A) and an image including a second user (indicated as user B) is displayed on the display, and a multi-person photo shooting preview screen (144) is displayed. The multi-person photo shooting preview screen (144) may display a chroma key image of each user. Additionally or alternatively, the multi-person photo shooting preview screen (144) may display a composite chroma key image and a background theme of each user.

[0072] In one embodiment, a composite image (142) may be generated and displayed on a display in response to a user (host) selecting a multi-person photo shooting mode entry button (112) in a first operation (110) and selecting a multi-person photo shooting button (132). After the composite image (142) is generated, a multi-person photo shooting preview screen (144) for generating the next composite image may be displayed on the display. In the multi-person photo shooting preview screen (144), image sequences (or chroma key image sequences) captured by the image sensors of each user terminal are synthesized to display a composite image sequence (or composite chroma key image sequence), and the user may prepare in advance the position, pose, movement, etc. of the user to be displayed in the composite image through this. For example, a first image sequence captured by the image sensor of the first user may be displayed in a first area of ​​a background theme selected by the user, and a second image sequence received from the second user terminal may be displayed in a second area of ​​the background theme, thereby displaying a composite image sequence. At this time, the first image sequence and the second image sequence can be generated by removing the color of the background area from the chroma key image sequence.

[0073] Thereafter, a composite image can be generated by the user selecting the multi-photo shooting button (132) again. At this time, as illustrated, in response to the user selecting the multi-photo shooting button (132), a timer ('3', '2', '1') is displayed on the multi-photo shooting preview screen (144), and a composite image can be generated when the time of the timer elapses.

[0074] In another example, a user may select a multi-photo button (132) to generate a composite image (142), and then sequentially generate another composite image after a predetermined amount of time (e.g., 5 seconds) has elapsed. This process may be repeated until a predetermined number of composite images have been generated, or until the user stops taking pictures.

[0075] The composite image (142) can be generated at the host terminal. For example, in response to receiving a user input selecting a multi-person photo capture button (132), the host terminal can capture a first image including the host using an image sensor, and generate a first chroma key image by changing the background area of ​​the first image to a first color. The host terminal can receive a second chroma key image including the guest from the guest terminal, and then generate a composite image (142) using the first chroma key image and the second chroma key image.

[0076] In one embodiment, the host terminal can generate a first person image associated with the host by removing an area indicated by a first color from a first chroma key image. Similarly, the host terminal can generate a second person image associated with the guest by removing an area indicated by a color from a second chroma key image. Thereafter, the host terminal can generate a composite image (142) by overlaying the generated first person image and the second person image on a selected background theme (e.g., a background theme selected by the host from among a plurality of background themes). At this time, the first person image can be overlaid on a first area of ​​the background theme, and the second person image can be overlaid on a second area of ​​the background theme.

[0077] In another embodiment, the host terminal may composite a portion of the background theme into the background area of ​​the first chroma key image and the background area of ​​the second chroma key image.

[0078] Alternatively, the host terminal may generate a cut-off image by removing a background area from a captured image without generating a first chroma key image, and generate a composite image (142) using the generated cut-off image and the second chroma key image.

[0079] The composite image sequence displayed on the multi-person photo shooting preview screen (144) can be generated from each user terminal that has entered the multi-person photo shooting mode. For example, a user terminal transmits an image sequence (or chroma key image sequence) captured by an image sensor to another user terminal, and each user terminal can generate a composite image sequence (or composite chroma key image sequence) using the transmitted image sequence (or chroma key image sequence) and the image sequence (or chroma key image sequence) captured by the image sensor.

[0080] The multi-person photo capture button (132) may be activated only on the user terminal of the user (host) who selected the multi-person photo capture mode entry button (112) in the first operation (110), or may be displayed only on the user terminal. Additionally, the multi-person photo capture button (132) may be deactivated if another user turns off the camera by selecting the camera turn off button (138).

[0081] In one embodiment, a user can apply various visual effects to the composite image (142) and / or the preview screen (144) by selecting the effect button (134). Specifically, when the user selects the effect button (134), a list of effects applicable to the composite image (142) and / or the preview screen (144) is displayed on the display, and when the user selects one or more effects from the effect list, the selected effects can be applied to the composite image (142) and / or the preview screen (144). For example, it can be confirmed that an effect that displays user information (such as the ID, screen name, nickname, etc. of each user in an instant messaging application, such as User A, User B, etc.) is applied to each of the composite image (142) and the preview screen (144) of FIG. 1 .

[0082] At this time, effects applicable to the composite image (142) and / or the preview screen (144) may include, but are not limited to, decorative elements (e.g., flower pictures, firecracker effects, etc.), information elements (e.g., information of each user included in the composite image, shooting date and time, text, etc.), image property elements (e.g., brightness, saturation, etc. of the composite image), etc.

[0083] After one or more effects are selected, the user can move the applied effects to the desired location by using drag and drop, or apply them to the desired size by using gestures such as spread or pinch.

[0084] In one embodiment, the user can select a gallery button (136) associated with a captured image. When the user selects the gallery button (136), the user can view images stored on the user terminal (e.g., previously captured composite images, photo booth images, etc.). Additionally, the user can select images stored on the user terminal and add them to the photo booth images.

[0085] In one embodiment, a user may select a theme button (140) associated with a theme to be applied to the photobooth image (152). For example, when the user selects the theme button (140), a list of themes applicable to the photobooth image (152) may be displayed on the display, and when the user selects a theme from the list of themes, a photobooth image with the selected theme applied may be displayed on the display. Alternatively, a separate theme may be selected and applied to each composite image included in the photobooth image. Additionally or alternatively, the user may select a theme in advance before selecting the multi-photo capture button (132).

[0086] In one embodiment, a user can delete a composite image (142) by selecting a delete button (145) corresponding to the composite image (142). After the composite image (142) is deleted, the user can create a new composite image to replace the deleted composite image.

[0087] In one embodiment, the delete button (145) may be activated only for the user terminal of the user (host) who selected the multi-person photo shooting mode entry button (112) in the first operation (110), or may be displayed only for that user terminal. Alternatively, the delete button (145) may be activated for all users who have entered the multi-person photo shooting mode, but the composite image may be deleted only when a majority of all users select the delete button for the same composite image.

[0088] The fourth action (150) may represent an example in which a photo booth image (152) including multiple composite images each including multiple users is displayed on the display of a user terminal in response to the completion of image capturing in the third action (130).

[0089] In Fig. 1, the photo booth image (152) is illustrated as including two composite images, but this is not limited thereto, and any number of composite images may be included in the photo booth image (152) at any position and in any size. For example, the user may set the number of composite images to be captured, the position and size of each composite image, etc., and the composite image may be generated by capturing each user's image a number of times corresponding to the set number of composite images.

[0090] A user can retake a photo booth image (152) by selecting a retake button (154). In one embodiment, in response to a majority of users included in the photo booth image (152) selecting the retake button (154), the multi-person photo shoot preview screen can be displayed again. In another embodiment, the retake button (154) is activated only for the host, and in response to the host selecting the retake button (154), the multi-person photo shoot preview screen can be displayed again.

[0091] The user can select the download button (156) to save the photo booth image (152) to the user terminal.

[0092] The sizes and locations of the images, buttons, etc. illustrated and described in Figure 1 are exemplary and are not limited thereto. For example, some buttons may be added or omitted, or configured in sizes and locations different from those illustrated.

[0093] FIG. 2 is a schematic diagram illustrating a configuration in which an information processing system (230) is connected to a plurality of user terminals (210_1, 210_2, 210_3) so as to be able to communicate with each other in order to provide an image capturing service during a video call according to one embodiment of the present disclosure. The information processing system (230) may include a system(s) capable of providing an instant messaging service, a system(s) capable of providing a social network service, and / or a system(s) capable of providing an image capturing service during a video call. In one embodiment, the information processing system (230) may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to the instant messaging service, the social network service, and / or the image capturing service during a video call, or one or more distributed computing devices and / or distributed databases based on a cloud computing service. For example, the information processing system (230) may include separate systems (e.g., servers) for the instant messaging service, the social network service, and / or the image capturing service during a video call.

[0094] Instant messaging services, image capture services during video calls, etc. provided by the information processing system (230) may be provided to users through instant messaging applications installed on each of a plurality of user terminals (210_1, 210_2, 210_3). For example, the instant messaging service may include a text messaging service, a voice messaging service, a video call service, a voice call service, a video streaming service, a social network service, an image capture service during video calls, etc. between users of the instant messaging application.

[0095] A plurality of user terminals (210_1, 210_2, 210_3) can communicate with an information processing system (230) via a network (220). The network (220) can be configured to enable communication between the plurality of user terminals (210_1, 210_2, 210_3) and the information processing system (230). Depending on the installation environment, the network (220) can be configured as a wired network such as Ethernet, a wired home network (Power Line Communication), a telephone line communication device, and RS-serial communication, a wireless network such as a mobile communication network, WLAN (Wireless LAN), Wi-Fi, Bluetooth, and ZigBee, or a combination thereof. The communication method is not limited, and may include not only a communication method utilizing a communication network (e.g., a mobile communication network, wired Internet, wireless Internet, broadcasting network, satellite network, etc.) that the network (220) may include, but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0096] In FIG. 2, a mobile phone terminal (210_1), a tablet terminal (210_2), and a PC terminal (210_3) are illustrated as examples of user terminals, but are not limited thereto, and the user terminals (210_1, 210_2, 210_3) may be any computing device capable of wired and / or wireless communication and capable of installing and executing instant messaging applications, etc. For example, the user terminals may include smartphones, mobile phones, navigation devices, computers, laptops, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet PCs, game consoles, wearable devices, IoT (Internet of Things) devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, etc. In addition, although FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with the information processing system (230) via the network (220), this is not limited thereto, and a different number of user terminals may be configured to communicate with the information processing system (230) via the network (220).

[0097] In one embodiment, each of the user terminals (210_1, 210_2, 210_3) may receive information or data from or transmit information or data to other user terminals via the network (220). For example, each of the user terminals (210_1, 210_2, 210_3) may transmit and receive a user input for selecting a multi-person photo shooting button, a user input for selecting a multi-person photo shooting mode exit button, an image (and / or a chroma key image) in which the user is included, a user input for selecting one of a plurality of photo themes, and / or an image sequence (and / or a chroma key image sequence), etc.

[0098] FIG. 3 is a block diagram illustrating the internal configuration of a user terminal (210) and an information processing system (230) according to one embodiment of the present disclosure. The user terminal (210) may refer to any computing device capable of executing an instant messaging application, etc. and capable of wired / wireless communication, and may include, for example, a mobile phone terminal (210_1), a tablet terminal (210_2), a PC terminal (210_3) of FIG. 2. As illustrated, the user terminal (210) may include a memory (312), a processor (314), a communication module (316), and an input / output interface (318). Similarly, the information processing system (230) may include a memory (332), a processor (334), a communication module (336), and an input / output interface (338). As illustrated in FIG. 3, the user terminal (210) and the information processing system (230) may be configured to communicate information and / or data via a network (220) using respective communication modules (316, 336). In addition, the input / output device (320) may be configured to input information and / or data to the user terminal (210) or output information and / or data generated from the user terminal (210) via the input / output interface (318).

[0099] The memory (312, 332) may include any non-transitory computer-readable recording medium. According to one embodiment, the memory (312, 332) may include a non-permanent mass storage device such as a read-only memory (ROM), a disk drive, a solid-state drive (SSD), a flash memory, etc. As another example, a non-permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the user terminal (210) or the information processing system (230) as a separate permanent storage device distinct from the memory. In addition, the memory (312, 332) may store an operating system and at least one program code (e.g., code for an application associated with an instant messaging service, a social network service, or an image capturing service during a video call).

[0100] These software components may be loaded from a computer-readable recording medium separate from the memory (312, 332). This separate computer-readable recording medium may include a recording medium directly connectable to the user terminal (210) and the information processing system (230), and may include, for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (312, 332) through a communication module (316, 336) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (312, 332) based on a computer program (e.g., an application associated with an instant messaging service, a social network service, or an image capture service during a video call) that is installed by files provided by developers or a file distribution system that distributes installation files of applications through a network (220).

[0101] The processor (314, 334) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. Instructions may be provided to the processor (314, 334) by a memory (312, 332) or a communication module (316, 336). For example, the processor (314, 334) may be configured to execute instructions received according to program code stored in a storage device such as the memory (312, 332).

[0102] The communication module (316, 336) may provide a configuration or function for the user terminal (210) and the information processing system (230) to communicate with each other via the network (220), and may provide a configuration or function for the user terminal (210) and / or the information processing system (230) to communicate with another user terminal or another system (e.g., a separate cloud system, etc.). For example, a request or data (e.g., a request for multi-party photo shooting, etc.) generated by the processor (314) of the user terminal (210) according to a program code stored in a recording device such as a memory (312) may be transmitted to the information processing system (230) via the network (220) under the control of the communication module (316). Conversely, a control signal or command provided under the control of the processor (334) of the information processing system (230) may be received by the user terminal (210) through the communication module (316) of the user terminal (210) via the communication module (336) and the network (220). For example, the user terminal (210) may receive an image (and / or image sequence) including another user from the information processing system (230).

[0103] The input / output interface (318) may be a means for interfacing with an input / output device (320). As an example, the input device may include a device such as a camera, a keyboard, a microphone, a mouse, etc., including an audio sensor and / or an image sensor, and the output device may include a device such as a display, a speaker, a haptic feedback device, etc. As another example, the input / output interface (318) may be a means for interfacing with a device that has a configuration or function integrated into one for performing input and output, such as a touch screen. In FIG. 3, the input / output device (320) is illustrated as not being included in the user terminal (210), but is not limited thereto and may be configured as a single device with the user terminal (210). In addition, the input / output interface (338) of the information processing system (230) may be a means for interfacing with a device (not shown) for input or output that is connected to the information processing system (230) or that the information processing system (230) may include. In FIG. 3, the input / output interface (318, 338) is illustrated as an element configured separately from the processor (314, 334), but is not limited thereto, and the input / output interface (318, 338) may be configured to be included in the processor (314, 334).

[0104] The user terminal (210) and the information processing system (230) may include more components than those shown in FIG. 3. However, it is not necessary to explicitly illustrate most of the conventional technical components. In one embodiment, the user terminal (210) may be implemented to include at least some of the input / output devices (320) described above. In addition, the user terminal (210) may further include other components such as a transceiver, a Global Positioning System (GPS) module, a camera, various sensors, a database, etc. For example, if the user terminal (210) is a smartphone, it may include components that a smartphone generally includes, and for example, various components such as an acceleration sensor, a gyro sensor, a microphone module, a camera module, various physical buttons, buttons using a touch panel, input / output ports, and a vibrator for vibration may be implemented to be further included in the user terminal (210).

[0105] According to one embodiment, the processor (314) of the user terminal (210) may be configured to operate an instant messaging application or a web browser application that provides an instant messaging service, including an image capture service during a video call. At this time, program code associated with the application may be loaded into the memory (312) of the user terminal (210). While the application is operating, the processor (314) of the user terminal (210) may receive information and / or data provided from the input / output device (320) through the input / output interface (318) or may receive information and / or data from the information processing system (230) through the communication module (316), and may process the received information and / or data and store it in the memory (312). In addition, such information and / or data may be provided to the information processing system (230) through the communication module (316).

[0106] While the instant messaging application is running, the processor (314) may receive voice data, text, images, videos, etc. input or selected through input devices such as a camera, microphone, including a touch screen, keyboard, audio sensor, and / or image sensor connected to the input / output interface (318), and may store the received voice data, text, images, and / or videos in the memory (312) or provide the received voice data, text, images, and / or videos to the information processing system (230) through the communication module (316) and the network (220). In one embodiment, the processor (314) may receive a user input for selecting a graphic object displayed on a display, which is input through the input device, and provide data / requests corresponding to the received user input to the information processing system (230) through the network (220) and the communication module (316).

[0107] The processor (314) of the user terminal (210) can output information and / or data by transmitting the information and / or data to the input / output device (320) through the input / output interface (318). For example, the processor (314) of the user terminal (210) can output the processed information and / or data through the output device (320), such as a display output capable device (e.g., a touch screen, a display, etc.) or a voice output capable device (e.g., a speaker). In one embodiment, the processor (314) can display a composite image synthesized using chroma key images each of a plurality of users included on the display of the user terminal (210).

[0108] The processor (334) of the information processing system (230) may be configured to manage, process, and / or store information and / or data received from multiple user terminals (210) and / or multiple external systems. Information and / or data processed by the processor (334) may be provided to the user terminal (210) via a communication module (336) and a network (220).

[0109] FIG. 4 is a block diagram illustrating the internal configuration of a processor (314) of a user terminal according to one embodiment of the present disclosure. As illustrated, the processor (314) may include an image capturing unit (410), an image preprocessing unit (420), an image synthesis unit (430), an image postprocessing unit (440), and the like.

[0110] The image capturing unit (410) can receive an image and / or image sequence including a user, captured using an image sensor of a user terminal. The image and / or image sequence can be received / transmitted using Voice over Internet Protocol (VoIP). For example, the image capturing unit (410) can receive an image sequence including a user during a video call or after entering a multi-party photo capturing mode. The image sequence including the user or the chroma key image sequence can be transmitted to another user terminal.

[0111] In another example, the image capturing unit (410) may receive an image including a user (host) from the image sensor in response to the user (host) selecting a multi-person photo capture button. In addition, the image capturing unit (410) may receive an image (or chroma key image) including another user (guest) from another user terminal. Thereafter, a composite image may be generated based on the image (or chroma key image) including the user (host) and the image (or chroma key image) including the other user (guest).

[0112] In another example, the image capture unit (410) may receive an image including a user (guest) captured using an image sensor in response to another user (host) selecting a multi-person photo capture button. At this time, the image including the user (guest) (or the chroma key image generated by the image preprocessing unit (420)) may be transmitted to the other user (host) and used to create a composite image.

[0113] The image preprocessing unit (420) can preprocess images captured by the image capturing unit (410) and / or images received from other user terminals. In one embodiment, the image preprocessing unit (420) can recognize a person area within a user image captured by the image capturing unit (410) using a person recognition model, and generate a chroma key image by changing the background area, which is an area outside the person area, to a specific color. The specific process of changing the background area to a specific color will be described in detail later with reference to FIGS. 5 and 6.

[0114] The image preprocessing unit (420) can generate a portrait image or a portrait image sequence by removing the color of the background area from a chroma key image or a chroma key image sequence. This allows for the generation of a composite image based on an image with the background removed, thereby not only generating an image that appears as if multiple users took photos in a single space, but also reducing the bandwidth required for image transmission and reception for the generation of the composite image.

[0115] In another embodiment, the image preprocessing unit (420) may normalize the color properties (e.g., including brightness and saturation) that indicate the hue, intensity, contrast, and density of an image captured by the image capturing unit (410) and / or an image of another user received from another user terminal. Through this configuration, even if multiple users capture images in different environments, images that appear as if they were captured in a single space may be generated.

[0116] In another embodiment, the image preprocessing unit (420) may resize an image including each of the plurality of users so that the sizes of the plurality of users are displayed similarly in the composite image, or rotate the image so that the plurality of users are displayed in the same / similar direction.

[0117] The image synthesis unit (430) can generate a composite image (or composite image sequence) using chroma key images (or chroma key image sequences) each of which includes a plurality of users. For example, the image synthesis unit (430) can generate a first person image (or first person image sequence) by removing an area indicated by the color of the background area from a first chroma key image (or first chroma key image sequence) including a user (host) and overlay the first person image (or first person image sequence) generated by removing an area indicated by the color of the background area from a second chroma key image (or second chroma key image sequence) including another user (guest) and overlay the second person image (or second person image sequence) generated by removing an area indicated by the color of the background area from a second chroma key image (or second chroma key image sequence) including another user (guest) and overlay the second person image (or second person image sequence) generated by overlaying the second person image (or second chroma key image sequence) on the second area of ​​the background theme to generate a composite image. In another example, the image synthesis unit (430) can generate a composite image by synthesizing a part of the background theme into the background area of ​​the first chroma key image (or first chroma key image sequence) and the background area of ​​the second chroma key image (or second chroma key image). In one embodiment, the chroma key image (or chroma key image sequence) containing another user (guest) may be received from the terminal of the other user.

[0118] The image synthesis unit (430) may generate a composite image including multiple layers such that an image (e.g., a portrait image) captured by a user terminal that first enters the multi-person photo shooting mode among multiple user terminals is placed in the upper layer. For example, since a host terminal enters the multi-person photo shooting mode with a multi-person photo shooting request, an image including the host may be placed in the uppermost layer. In another example, the image synthesis unit (430) may change the arrangement of layers according to a request from a host to change the position of a layer, or may place layers according to layer setting rules preset by the host.

[0119] The image post-processing unit (440) can apply a user-selected theme and / or effect to the composite image and / or photo booth image. Alternatively, the configuration for applying the user-selected theme and / or effect to the composite image can be performed by the image synthesis unit (430). For example, the operation of applying a background or theme to the composite image can be performed by placing the corresponding background image in the lowest layer. In another example, the step of displaying the effect in the composite image can be performed by placing the corresponding effect image in the highest layer.

[0120] The internal configuration of the processor (314) illustrated in FIG. 4 is merely an example, and in some embodiments, other configurations may be additionally included in addition to the illustrated internal configurations, and some configurations may be omitted. For example, if some of the above internal configurations are omitted, the processor (334) of the information processing system and / or other user terminals may be configured to perform the functions of some of the omitted internal configurations. For example, the processor (334) of the information processing system and / or other user terminals may perform image preprocessing tasks. In addition, although the internal configuration of the processor (314) in FIG. 4 is described by dividing it by function, it does not necessarily mean that it is physically separated. The image capturing unit (410), the image preprocessing unit (420), the image synthesis unit (430), and the image postprocessing unit (440) are described separately, but this is to help understanding of the invention and is not limited thereto.

[0121] FIG. 5 is a diagram illustrating an example of generating a chroma key image (520) for generating a composite image from an image (510) captured by an image sensor of a user terminal according to one embodiment of the present disclosure. The captured image (510) may include a background area (512) and a person area (514). The chroma key image (520) may be generated by recognizing the person area (514) in the captured image (510) and then changing the background area (512), which is an area outside the person area (514), to a specific color. In one embodiment, the person area (514) may be recognized using a pre-learned person recognition model.

[0122] In one embodiment, the color of the background area (522) excluding the person area (524) of the chroma key image (520) may be determined as a color whose similarity with a color included in the person area (524) is less than a predetermined threshold. For example, the similarity with each of a plurality of colors included in the person area (524) may be calculated, and a color whose calculated similarity is all less than the threshold may be determined as the color of the background area (522).

[0123] In one embodiment, the color of the background area (522) excluding the person area (524) of the chroma key image (520) may be determined as a predetermined first color. In one example, it may first be determined whether a color having a similarity to the first color greater than or equal to a predetermined threshold is included in the person area (524). At this time, if a color having a similarity to the first color greater than or equal to the predetermined threshold is not included in the person area (524), the first color may be determined as the color of the background area (522). On the other hand, if a color having a similarity to the first color greater than or equal to the predetermined threshold is included in the person area (524), a second color different from the first color may be determined as the color of the background area (522). The second color may be a predetermined color or a color determined based on a color included in the person area (524).

[0124] In one embodiment, the color of the background area (522) may be determined based on an average color value associated with the person area (524). For example, RGB values ​​of multiple colors within the person area (524) may be calculated, an average value of each RGB value may be calculated, and then the color of the background area (522) may be determined based on the calculated average RGB value. For example, the average RGB value may be determined as the color of the background area (522), or an RGB value that is complementary to the average RGB value may be determined as the color of the background area (522).

[0125] In one embodiment, the color of the background area (522) may be determined based on the distance between each of the plurality of color values ​​associated with the person area (524) and an arbitrary color value. For example, a statistical value, such as an average value, variance, or standard deviation of the distance between each of the plurality of color values ​​and an arbitrary color value, may be calculated, and then the arbitrary color value with the smallest calculated statistical value may be determined as the color of the background area (522).

[0126] In one embodiment, a predetermined color may be used as the color of the background area (522).

[0127] FIG. 6 is a diagram illustrating an example of generating a chroma key image (642) in response to a user input according to one embodiment of the present disclosure. The first operation (610) illustrates an example of outputting a user's chroma key image (612) on a display of a user terminal. The user can check the chroma key image (612) before or after entering the multi-person photo shooting mode and confirm whether the chroma key image (612) is generated / output normally.

[0128] A chroma key image (612) may include a background area (614) and a person area (616) displayed in a first color. A user can check / change the color of the background area (614) by selecting a background color button (618).

[0129] The second operation (620) illustrates an example in which a pop-up window (622) including a change request message is output on the display in response to determining that the similarity between the first color of the background area (614) and the color included in the person area (616) is greater than a predetermined threshold. That is, if the similarity between the first color and the color included in the person area (616) is greater than a predetermined threshold, it may be recommended to the user to change the color of the background area (614) because when the color of the background area is removed, a part of the person area (616) may be removed together or noise may be generated in the person area (616).

[0130] A user can change the color of the background area (614) by selecting the first button (624). For example, in response to the user selecting the first button (624), a screen of the third action (630) may be displayed on the display. The user may not change the color of the background area (614) by selecting the second button (626).

[0131] Alternatively, even when the color of the background area (614) needs to be changed, such as when the similarity between the first color and the color included in the character area (616) is greater than a predetermined threshold, the color of the background area (614) may be automatically changed to any other color without displaying the pop-up window (622), or a recommended color may be displayed, or the color of the background area (614) may be automatically changed to any other color in response to selecting the first button (624). In this case, the color of the background area (614) may be automatically recommended or changed according to the embodiment described in FIG. 5.

[0132] The third action (630) illustrates an example of a screen that allows a user to select a color for the background area (614) in response to a user selecting the background color button (618) of the first action (610) or in response to a user selecting the first button (624) of the second action (620). The user can select a second color (634) within the color palette (632) and select a new color for the background area (614) by selecting the third button (636). If the user selects the fourth button (638), the screen of the first action (610) may be displayed again.

[0133] The method for selecting a color for the background area (614) is not limited to the above method. For example, the user can select a color in various ways, such as by directly entering RGB values, HSL (Hue, Saturation, and Lightness) values, or by selecting a specific value from the CIELAB color space.

[0134] The fourth action (640) illustrates an example in which, in response to the selection of the third button (636) in the third action (630), a chroma key image (642) is output on the display in which the background area (644) excluding the person area (646) is displayed in a second color (634). Thereafter, in response to the user selecting the fifth button (648), the screen of the third action (630) may be output on the display again, and the color of the background area (644) may be changed again. Additionally, in response to the determination that the similarity between the second color (634) and the color included in the person area is greater than a predetermined threshold, the pop-up window (622) of the second action (620) may be output again.

[0135] When a user changes the background area of ​​a chroma key image from a first color to a second color, a chroma key image (642) having a background area of ​​the second color can be transmitted to the other party's terminal. In addition, not only the user's chroma key image, but also the background area of ​​another user's chroma key image can be changed from the first color to the second color. Alternatively, only the chroma key image of a user who changed the background area of ​​the image from the first color to the second color can have the color of the background area changed from the first color to the second color.

[0136] FIG. 7 is a diagram illustrating an operation of entering a multi-party photo shooting mode on the host terminal side according to one embodiment of the present disclosure. The first operation (710) illustrates an example in which, during a video call, a user selects a multi-party photo shooting mode entry button on the host terminal, and an area (712) for selecting the properties of a photo booth image to be captured and a button (714) for entering the multi-party photo shooting mode are displayed on the display of the host terminal. In FIG. 7, the number of composite images to be included in the photo booth image is illustrated as being selected in the area (712), but the present invention is not limited thereto. For example, the host can preset the position, size, theme, etc. of each composite image in the area (712).

[0137] The second operation (720) illustrates an example of entering a multi-person photo shooting mode by receiving a user input of selecting a button (714) from the host terminal. As illustrated, a first image sequence captured by an image sensor of the host terminal may be displayed in a first area (722) of the display. In addition, a second image sequence received from a guest terminal may be displayed in a second area (724) of the display. Here, the guest terminal may be a user terminal of another user who entered the multi-person photo shooting mode created by the host while conducting a video call with the host. Additionally, the host may adjust the position, size, etc. of the area where the first image sequence and the second image sequence are to be displayed in the multi-person photo shooting mode.

[0138] In one embodiment, the host can set / change the number of images to be included in the composite image in the area (730). For example, the host can select the number '2' in the area (730) to set continuous shooting of two composite images. Alternatively, the host can select the number '4' in the area (730) to set continuous shooting of four composite images. In the area (730), the host is shown as being able to select one of 1 to 4 images, but this is not limited thereto, and the host can select any number.

[0139] In one embodiment, a shooting preparation notification may be displayed when the host selects the multi-person photo capture button (726). Thereafter, multi-person photo capture may be performed with the shooting notification displayed, thereby generating a composite image (or photo booth image). The composite image may be generated on the host terminal using a first image (or first chroma key image) captured using the image sensor of the host terminal and a second image (or second chroma key image) received by the host terminal from the guest terminal. The generated composite image may be displayed on the displays of the host terminal and the guest terminal for the host and guest's confirmation.

[0140] Alternatively, when the host selects the multi-person photo capture button (726), video capture may begin, thereby generating a composite video (or photo booth video). The composite video (or photo booth video) may include multiple composite images (or multiple photo booth images). The composite video may be captured for a predetermined time period, or for a time period determined by a user (e.g., the time period during which the user presses and holds the multi-person photo capture button (726). The generated composite video (or photo booth video) may be saved or played back at a predetermined speed (e.g., 2x or 3x).

[0141] In one embodiment, the host may select an exit button (728) before / after the photo booth image capture is completed. When the host selects the exit button (728) before the photo booth image capture is completed, the multi-person photo capture mode may be terminated on the host terminal. In this case, one of the guests included in the multi-person photo capture mode (e.g., a guest who entered the multi-person photo capture mode in the following order from the host) may be granted host authority. In this case, after a predetermined time (e.g., 5 seconds) has elapsed after the multi-person photo capture mode is terminated on the host terminal, the delegation of host authority may be performed accordingly, and if the host reconnects to the multi-person photo capture mode before the predetermined time has elapsed, the host authority may be maintained.

[0142] In contrast, when the host selects the exit button (728) after completing the shooting of the photo booth image, the photo booth image is stored in the host terminal and transmitted to the guest terminal through the chat room where the video call was made or the 1:1 chat room with the guest, or transmitted to the guest terminal in the form of metadata of a VoIP (Voice over Internet Protocol) call packet containing data corresponding to the photo booth image.

[0143] FIG. 8 is a diagram illustrating an operation for entering a multi-party photo capture mode on a guest terminal according to one embodiment of the present disclosure. The first operation (810) illustrates an example in which, upon receiving a multi-party photo capture request from a host terminal during a video call, an area (812) for determining whether to accept the request is displayed on the guest terminal's display.

[0144] In one embodiment, the area (812) may display information such as attribute information of the photo booth image to be captured. For example, although FIG. 8 illustrates that information related to the theme of the photo booth image is displayed, the present invention is not limited thereto, and the number, location, size, etc. of composite images may be displayed. Additionally, information such as the host of the multi-person photo shooting mode and a list of participating guests may be displayed. Guests may select the first button (814) in the area (812) to enter the multi-person photo shooting mode, or may select the second button (816) to decline the multi-person photo shooting.

[0145] The second operation (820) illustrates an example in which a user selects the first button (814) on a guest terminal to enter multi-person photo shooting mode. Unlike FIG. 7, instead of displaying the multi-person photo shooting button, the guest terminal may display a third button (826) that allows the user to check the currently applied theme and / or applicable theme information for the photo booth image. After selecting the third button (826), the guest can select the desired theme information to be applied to the photo booth image. This will be described in detail later in FIG. 9.

[0146] As illustrated, a first image sequence (or, a character image sequence) received from the host terminal may be displayed in a first area (822) of the display. Additionally, a second image sequence (or, a character image sequence) captured by an image sensor of the guest terminal may be displayed in a second area (824) of the display.

[0147] In one embodiment, a guest may exit the multi-person photo shooting mode by selecting the exit button (828) before or after the photo booth image capture is completed. In this case, the multi-person photo shooting mode may be terminated on the guest terminal, and the video call screen prior to entering the multi-person photo shooting mode may be displayed.

[0148] FIG. 9 is a diagram illustrating an example of a guest providing feedback on a photo theme according to one embodiment of the present disclosure. The first operation (910) illustrates an example of a screen displayed on the display of a guest terminal when the guest terminal enters multi-person photo shooting mode.

[0149] The second operation (920) illustrates an example in which, after the guest terminal enters the multi-person photo shooting mode and before the shooting notification is displayed, the guest selects a button (912) to display multiple photo themes that can be applied to the photo booth image / composite image. Thereafter, the guest can provide feedback (e.g., positive feedback or negative feedback, etc.) to a specific photo theme by selecting at least one of the visual objects (922, 924, 926) displayed on the preview (e.g., thumbnail) of the multiple photo themes.

[0150] The third action (930) illustrates an example in which, in response to a guest providing feedback (or positive feedback) to a specific photo theme by selecting a visual object (924), a visual object (934) indicating feedback is displayed near the selected photo theme. As the guest provides feedback to a specific photo theme, the visual object (934) displayed for the selected photo theme is different from other visual objects (932, 936) to which feedback was not provided, so that the guest can easily identify the photo theme to which he or she provided feedback.

[0151] Additionally, the host can check the number of feedback guests have given to a photo theme via the host device. This configuration allows the host to consider guests' opinions when selecting a photo theme to apply to the photobooth image / composite image. Alternatively, the photo theme with the highest number of positive feedbacks can be automatically applied to the photobooth image / composite image, or a message can be provided suggesting that the host change to the photo theme with the highest number of positive feedbacks.

[0152] After the host selects a photo theme to be applied to the photo booth image / composite image, a first image sequence captured by the image sensor of the host terminal (or a person image sequence with the background area removed from the chroma key image sequence) may be displayed in a first area of ​​the selected photo theme on the display. In addition, a second image sequence captured by the image sensor of the guest terminal (or a person image sequence with the background area removed from the chroma key image sequence) may be displayed in a second area of ​​the selected photo theme on the display.

[0153] In one embodiment, a photo theme may include a pose guide. The pose guide may be a visual guide displayed on the display to help users easily follow a specific pose. For example, a first pose guide may be overlaid on a first image sequence in a first region of the selected photo theme, and a second pose guide may be overlaid on a second image sequence in a second region. This configuration encourages users to take photos in various poses, allowing them to capture a highly composed composite image without having to devise a pose.

[0154] FIG. 10 is a diagram illustrating examples of displaying photo booth images according to one embodiment of the present disclosure. In the first to fifth examples (1010, 1020, 1030, 1040, and 1050) of FIG. 10, multiple users included in each grid of the photo booth images may correspond to users who have entered a multi-person photo shooting mode. In FIG. 10, each grid is illustrated as including one user, but this is not limited thereto. The first to fifth examples (1010, 1020, 1030, 1040, and 1050) are examples of cases where the number of users who have entered the multi-person photo shooting mode is 2 to 6, respectively.

[0155] A photo booth image can be generated by combining images including each of multiple users, and the relative position and size of the grid can change whenever a user who has entered the multi-user photo shooting mode is added or removed. For example, in the photo booth image of the second example (1020), the upper grid can be seen to have moved to the left from its original position and reduced in size by half after a user is added in the third example (1030). The positions of the grid or users within the photo booth image can be determined based on predetermined layout rules and the number of users who have entered the multi-user photo shooting mode, or can be changed based on user input from the host who changes the position of each grid or user.

[0156] Alternatively, a photo booth image / composite image may include multiple layers, with the image / image sequence of the user who first entered the multi-user photo shooting mode placed on the upper layer. In this case, the composite image may be created by synthesizing images of each of the multiple users, and the relative positions or sizes (layout) of the users within the composite image may change as users entering the multi-user photo shooting mode are added or removed.

[0157] The position or size of the users within the composite image may be determined based on predetermined user placement rules and the number of users entering the multi-photo shooting mode, or may be changed based on user input from the host that changes the position or size of each user.

[0158] FIG. 11 is a diagram illustrating a video call group (1100) associated with a video call and subgroups (1110, 1120, 1130) associated with a multi-party photo capture mode according to one embodiment of the present disclosure. When a video call is initiated within a chat room of an instant messaging application, users participating in the video call can enter the video call group (1100). Some or all users within the chat room can enter the video call group (1100), and image sequences can be transmitted and received between users within the video call group (1100).

[0159] Thereafter, when a user enters a multi-party photo shooting mode during a video call, the user can enter one of the sub-groups (1110, 1120, 1130) associated with multi-party photo shooting within the video call group (1100). That is, while one video call is in progress, multiple sub-groups (1110, 1120, 1130) for generating a composite image can be generated, and a composite image including users who entered each of the sub-groups (1110, 1120, 1130) can be generated.

[0160] The maximum number of users that can participate in each subgroup (1110, 1120, 1130) (or multi-person photo shooting) may be less than the maximum number of users that can participate in the video call group (1100) (or video call). In one embodiment, the maximum number of users that can participate in each subgroup may be set to a predetermined number of people (e.g., 4).

[0161] In one embodiment, a user can choose whether to enter a subgroup (1110, 1120, 1130). For example, a user can receive a multi-person photo shoot request from a host terminal that created a specific subgroup and enter that subgroup by accepting the request.

[0162] In one embodiment, the user can select a subgroup to enter from among multiple subgroups (1110, 1120, 1130). For example, during a video call, while a multi-party photo shooting mode is in progress in each of the multiple subgroups (1110, 1120, 1130), the user can select which of the multiple subgroups (1110, 1120, 1130) to enter. That is, instead of the user acting as a host by creating a new subgroup, the user can select and enter one of the subgroups in which other users are already participating. For example, the user terminal outputs information related to the subgroups in which other users are participating (such as the time when the multi-party photo shooting mode was in progress within the subgroup, information on the number of participants, etc.), and the user can check the information related to the subgroup and then select one of the subgroups.

[0163] If a user attempts to enter a specific subgroup but the number of participants exceeds the preset number (e.g., 4), the user may enter photo-taking viewing mode instead of photo-taking mode. This means that users in photo-taking viewing mode can view other users' multi-person photo-taking sessions, but cannot participate in the multi-person photo-taking themselves.

[0164] Conversely, if the number of users participating in a specific subgroup is less than a predetermined number, a newly entrant user can enter multi-user photo capture mode. In this case, a first image sequence captured by the image sensor of a user terminal already participating may be displayed in the first area of ​​the display, and a second image sequence received from the newly entrant user terminal may be displayed in the second area of ​​the display.

[0165] FIG. 12 is a flowchart illustrating a process for generating a composite image according to one embodiment of the present disclosure. After a video call is initiated between a host terminal (1210) associated with a first user and a guest terminal (1220) associated with a second user in a chat room of an instant messaging application, the host terminal (1210) may enter a multi-party photo shooting mode (1212). For example, the host terminal may enter the multi-party photo shooting mode by receiving a user input selecting a multi-party photo shooting mode entry button, such as a touch input.

[0166] When the host terminal (1210) enters the multi-person photo shooting mode (1212), the host terminal (1210) can transmit a multi-person photo shooting request (1214) to the guest terminal (1220). Thereafter, when the guest terminal (1220) receives a user input agreeing to the multi-person photo shooting request, the guest terminal (1220) can enter the multi-person photo shooting mode (1222). The display of the host terminal (1210) and / or the guest terminal (1220) can output the user's chroma key image before entering the multi-person photo shooting mode.

[0167] In multi-person photo shooting mode, a guest chroma key image sequence can be transmitted to a host terminal (1210) (1224), and a host chroma key image sequence can be transmitted to a guest terminal (1220) (1216). Using the guest chroma key image sequence and the host chroma key image sequence, a preview screen displaying a composite image sequence can be output on the displays of the host terminal (1210) and the guest terminal (1220).

[0168] Thereafter, the host terminal (1210) can receive a user input from the user to click a shooting button (1218). As the host terminal (1210) receives the user input to click the shooting button, the host terminal (1210) can transmit a shooting signal to the guest terminal (1220) (1230).

[0169] In response to the host terminal (1210) receiving a user input of clicking a capture button, the host terminal (1210) can capture an image including the host using an image sensor (1232). Similarly, upon receiving a capture signal from the host terminal (1210), the guest terminal (1220) can capture an image including the guest using an image sensor (1226). The host terminal (1210) and the guest terminal (1220) can generate a chroma key image from the captured images.

[0170] Thereafter, the host terminal (1210) can receive a chroma key image including the guest from the guest terminal (1220) (1228).

[0171] Thereafter, the host terminal (1210) can generate a composite image using a chroma key image including the host and a chroma key image including the guest (1234). At this time, a background theme can be used to create the composite image.

[0172] Thereafter, the generated composite image may be transmitted to the guest terminal (1220) (1236). In one embodiment, the composite image may be transmitted to the guest terminal (1220) through a chat room (e.g., a group chat room or a 1:1 chat room) including the host and the guest. Additionally or alternatively, the composite image may be transmitted to the guest terminal (1220) using Voice over Internet Protocol (VoIP). For example, after metadata of a Voice over Internet Protocol (VoIP) call packet including data corresponding to the composite image is generated, the generated metadata may be transmitted to the guest terminal (1220). At this time, the generated metadata may be metadata within the protocol of a VoIP call channel connected to the host and the guest.

[0173] FIG. 13 is a flowchart illustrating a method (1300) for capturing an image during a video call according to one embodiment of the present disclosure. The method (1300) may be performed by at least one processor of a first user terminal.

[0174] The processor can initiate a video call between a first user terminal associated with a first user included in a chat room of an instant messaging application and a second user terminal associated with a second user (S1310).

[0175] Thereafter, the processor may enter multi-party photo shooting mode during the video call (S1320). For example, the processor may enter a specific subgroup among multiple subgroups associated with the multi-party photo shooting. In this case, the second user may be a user included in the same specific subgroup as the first user. The maximum number of users that can participate in the multi-party photo shooting may be less than the maximum number of users that can participate in the video call.

[0176] In one embodiment, after entering a multi-person photo shooting mode, the processor may display a plurality of background themes on the display of the first user terminal and receive a third user input from the first user to select one of the plurality of background themes.

[0177] In one embodiment, after entering the multi-person photo capture mode and before capturing an image captured by the image sensor of the first user terminal, the processor may display a first image sequence captured by the image sensor of the first user terminal in a first area of ​​a selected background theme on the display, and may display a second image sequence received from the second user terminal in a second area of ​​the selected background theme on the display. At this time, the first image sequence and the second image sequence may be generated by removing a color of a background area from a chroma key image sequence.

[0178] Thereafter, the processor may generate a first chroma-key image based on an image captured by an image sensor of the first user terminal in a multi-person photo capture mode (S1330). At this time, the first chroma-key image may be an image in which the background area excluding the person area of ​​the captured image is changed to a first color.

[0179] The processor can use a person recognition model to recognize a person area within a captured image, and generate a first chroma key image by changing a background area, which is an area other than the recognized person area in the image, to a first color.

[0180] In one embodiment, the processor may change the background area to the first color in response to receiving a first user input selecting a first color.

[0181] In one embodiment, the processor determines that the similarity between a predetermined second color and a color contained within the person area is greater than a predetermined threshold, and determines a first color whose similarity to the color contained within the person area is less than the predetermined threshold, thereby changing the background area to the first color. In this case, the first color may be different from the second color.

[0182] In one embodiment, the processor, in response to determining that the similarity between the predetermined second color and a color included in the person area is greater than a predetermined threshold, outputs a request message for changing the second color on the display of the first user terminal, and receives a second user input for changing the second color to the first color, thereby changing the background area to the first color.

[0183] In one embodiment, the processor can change the background area to the first color by calculating an average color value associated with the character area and determining a first color based on the average color value.

[0184] In one embodiment, the processor can change the background area to the first color by calculating a distance between each of a plurality of color values ​​associated with the character area and an arbitrary color value, and determining the first color based on the calculated distance.

[0185] After that, the processor can transmit the first chroma key image to the second user terminal (S1340).

[0186] Thereafter, the processor can output a composite image synthesized using the first chroma key image including the first user and the second chroma key image including the second user on the display of the first user terminal.

[0187] After entering the multi-person photo shooting mode, the processor may receive a fourth user input selecting a multi-person photo shooting button from the first user, receive a second chroma key image including the second user from the second user terminal, and generate a composite image using the first chroma key image and the second chroma key image. At this time, the first chroma key image may be generated in response to receiving the fourth user input.

[0188] In one embodiment, the processor may display a plurality of background themes on a display of a first user terminal, receive a fifth user input for selecting one of the plurality of background themes from the first user, generate a first person image by removing an area displayed in a first color from a first chroma key image, generate a second person image by removing an area displayed in a color of a background area from a second chroma key image, and generate a composite image by overlaying the first person image and the second person image on the selected background theme. For example, the processor may overlay the first person image on the first area of ​​the selected background theme and overlay the second person image on the second area of ​​the selected background theme.

[0189] In another embodiment, the processor may display a plurality of background themes on a display of a first user terminal, receive a sixth user input from the first user to select one of the plurality of background themes, and generate a composite image by synthesizing a portion of the selected background theme with a background area of ​​the first chroma key image and a background area of ​​the second chroma key image.

[0190] In one embodiment, the processor may transmit a composite image to a second user terminal via a chat room involving the first and second users. In another embodiment, the processor may generate metadata for a Voice over Internet Protocol (VoIP) call packet containing data corresponding to the composite image and transmit the metadata to the second user participating in the video call. In this case, the metadata may be metadata within the protocol of a VoIP call channel connected to the first and second users participating in the video call.

[0191] The flowcharts and descriptions illustrated in FIGS. 12 and 13 are merely examples, and may be implemented differently in other embodiments. For example, the order of some steps may be changed, some steps may be omitted, other components may be added, and some steps described as being performed sequentially may be performed concurrently.

[0192] The above-described method may be provided as a computer program stored on a computer-readable recording medium for execution on a computer. The medium may be one that continuously stores a computer-executable program or one that temporarily stores it for execution or download. In addition, the medium may be various recording or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program instructions, including ROM, RAM, and flash memory. In addition, examples of other media may include recording or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc.

[0193] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various exemplary logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and the design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0194] In a hardware implementation, the processing units used to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, a computer, or a combination thereof.

[0195] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with the present disclosure may be implemented or performed by any combination of a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented by a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0196] In a firmware and / or software implementation, the techniques may be implemented as instructions stored on a computer-readable medium, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, a compact disc (CD), a magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described herein.

[0197] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more standalone computer systems, the present disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the present disclosure may be implemented in multiple processing chips or devices, and storage may be similarly affected across multiple devices. Such devices may include personal computers, network servers, and portable devices.

[0198] While the present disclosure has been described in connection with certain embodiments herein, various modifications and variations may be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

Claims

1. A method for capturing an image during a video call, performed by at least one processor of a first user terminal, A step of initiating a video call between a first user terminal associated with a first user included in a chat room of an instant messaging application and a second user terminal associated with a second user; During the above video call, a step of entering multi-party photo shooting mode; In the above multi-person photo shooting mode, a step of generating a first chroma-key image based on an image captured by an image sensor of the first user terminal; and A step of transmitting the first chroma key image to the second user terminal Including, A method for capturing an image during a video call, wherein the first chroma key image is an image in which the background area excluding the person area of ​​the captured image is changed to the first color.

2. In paragraph 1, A step of outputting a composite image synthesized using the first chroma key image including the first user and the second chroma key image including the second user on the display of the first user terminal. A method for capturing an image during a video call, further comprising:

3. In paragraph 1, The step of generating the first chroma key image is as follows: A step of recognizing the person area within the photographed image using a person recognition model; and A step of changing the background area, which is an area other than the recognized person area in the image, to the first color. A method for capturing an image during a video call, comprising:

4. In paragraph 3, The step of changing the above background area to the first color is: In response to receiving a first user input selecting the first color, a step of changing the background area to the first color. A method for capturing an image during a video call, comprising:

5. In paragraph 3, The step of changing to the above first color is: A step of determining that the similarity between a predetermined second color and a color included in the character area is greater than a predetermined threshold; and A step of determining the first color whose similarity to a color included in the above-described character area is less than the predetermined threshold value. Including, A method for capturing an image during a video call, wherein the first color is different from the second color.

6. In paragraph 3, The step of changing to the above first color is: In response to determining that the similarity between a predetermined second color and a color included in the character area is greater than a predetermined threshold, a step of outputting a request message for changing the second color on the display of the first user terminal; and A step of receiving a second user input for changing the second color to the first color A method for capturing an image during a video call, comprising:

7. In paragraph 3, The step of changing to the above first color is: a step of calculating an average color value associated with the above-mentioned character area; and A step of determining the first color based on the average color value A method for capturing an image during a video call, comprising:

8. In paragraph 3, The step of changing to the above first color is: A step of calculating a distance between each of a plurality of color values ​​associated with the above-mentioned character area and an arbitrary color value; and A step of determining the first color based on the calculated distance A method for capturing an image during a video call, comprising:

9. In paragraph 1, After entering the above multi-person photo shooting mode, A step of displaying a plurality of background themes on the display of the first user terminal; and A step of receiving a third user input for selecting one of the plurality of background themes from the first user. A method for capturing an image during a video call, further comprising:

10. In paragraph 9, After entering the multi-person photo shooting mode and before capturing an image captured by the image sensor of the first user terminal, a step of displaying a first image sequence captured by the image sensor of the first user terminal in a first area of ​​the selected background theme on the display; and A step of displaying a second image sequence received from the second user terminal in a second area of ​​the selected background theme on the display. Including more, A method for capturing images during a video call, wherein the first image sequence and the second image sequence are generated by removing the color of a background area from a chroma key image sequence.

11. In paragraph 1, After entering the above multi-person photo shooting mode, A step of receiving a fourth user input selecting a multi-person photo taking button from the first user; A step of receiving a second chroma key image including the second user from the second user terminal; and A step of generating a composite image using the first chroma key image and the second chroma key image Including more, A method for capturing an image during a video call, wherein the first chroma key image is generated in response to receiving the fourth user input.

12. In paragraph 11, The step of generating the above composite image is: A step of displaying a plurality of background themes on the display of the first user terminal; A step of receiving a fifth user input selecting one of the plurality of background themes from the first user; A step of generating a first person image by removing an area indicated by the first color from the first chroma key image; A step of generating a second person image by removing an area indicated by the color of the background area from the second chroma key image; and A step of overlaying the first person image and the second person image on the selected background theme. A method for capturing an image during a video call, comprising:

13. In paragraph 12, The step of overlaying the selected background theme above is: a step of overlaying the first person image on the first area of ​​the selected background theme; and A step of overlaying the second character image on the second area of ​​the selected background theme. A method for capturing an image during a video call, comprising:

14. In paragraph 11, The step of generating the above composite image is: A step of displaying a plurality of background themes on the display of the first user terminal; receiving a sixth user input selecting one of the plurality of background themes from the first user; and A step of synthesizing a part of the selected background theme into the background area of ​​the first chroma key image and the background area of ​​the second chroma key image. A method for capturing an image during a video call, comprising:

15. In paragraph 11, A step of transmitting the above composite image to the second user terminal through a chat room including the first user and the second user. A method for capturing an image during a video call, further comprising:

16. In paragraph 11, A step of generating metadata of a VoIP (Voice over Internet Protocol) call packet including data corresponding to the above-mentioned synthetic image; and A step of transmitting the metadata to the second user who participated in the video call. Including more, A method for capturing an image during a video call, wherein the above metadata is metadata within the protocol of a VoIP call channel connected to the first user and the second user who participated in the video call.

17. In paragraph 1, The steps to enter the above multi-person photo shooting mode are: Steps to enter a specific subgroup among multiple subgroups involved in multi-person photo shooting Including, A method for capturing an image during a video call, wherein the second user is included in the same specific subgroup as the first user.

18. In paragraph 1, A method of capturing images during a video call, where the maximum number of users who can participate in a multi-party photo shoot is less than the maximum number of users who can participate in a video call.

19. A computer-readable non-transitory recording medium recording commands for executing the method according to paragraph 1 on a computer.

20. In the first user terminal, Communication module; display; image sensor; memory; and At least one processor connected to said memory and configured to execute at least one computer-readable program contained in said memory Including, At least one program above, Initiating a video call between a first user terminal associated with a first user included in a chat room of an instant messaging application and a second user terminal associated with a second user, During the above video call, enter multi-party photo shooting mode, In the above multi-person photo shooting mode, a first chroma key image is generated based on an image captured by an image sensor of the first user terminal, Includes commands for transmitting the first chroma key image to the second user terminal, The first user terminal, wherein the first chroma key image is an image in which the background area excluding the person area of ​​the captured image is changed to the first color.

Images