Method and system for capturing images during a video call

Abstract

The present disclosure relates to a method for capturing images during a video call executed by at least one processor of a first user terminal. [Solution] A method for capturing images during a video call includes the steps of starting a video call between a first user terminal associated with a first user and a second user terminal associated with a second user included in a chat room of an instant messaging application, switching to a multi-party capturing mode during the video call, displaying a capturing notification on the display of the first user terminal for a first period of time, and displaying a composite image on the display that combines a first image including the first user and a second image including the second user.

Description

[Technical Field]

[0001] The present disclosure relates to a method and system for capturing images during a video call, and more particularly to a method and apparatus for capturing an image of a user during a video call using an instant messaging application, and generating and displaying a composite image based on the captured image. [Background technology]

[0002] In recent years, with the spread of mobile devices such as smartphones and the development of Internet technology, instant messaging applications that allow not only voice calls but also video calls between multiple users have become widely used. Meanwhile, there has been a steady increase in demand for so-called "photo booths," which print and provide photos taken by multiple people together offline.

[0003] However, image capture functions provided in conventional video call services have problems such as capturing an image when the user is not ready to take an image, and requiring the user to give voice instructions as to when to take an image. Furthermore, because the function merely captures a screen at a specific moment during a video call, it is difficult to provide users with images that incorporate various formats, concepts, photo themes, etc. Therefore, conventional image capture services have a problem in that they are less likely to provide users with images that are satisfactory to them than images taken in a "photo booth." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Korean Patent Publication No. 10-2020-0145420 Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure provides a method for capturing images during a video call, a computer-readable non-transitory recording medium having instructions recorded thereon, and a system (apparatus) for solving the above-mentioned problems. [Means for solving the problem]

[0006] The present disclosure may be embodied in numerous forms, including as a method, a system (apparatus), or a non-transitory computer-readable storage medium having instructions recorded thereon.

[0007] In one embodiment of the present disclosure, a method for capturing images during a video call, executed by at least one processor of a first user terminal, includes the steps of: initiating a video call between a first user terminal associated with a first user and a second user terminal associated with a second user included in a chat room of an instant messaging application; switching to a multi-party capturing mode during the video call; displaying a capturing notification on a display of the first user terminal for a first period of time; and displaying a composite image on the display, in which a first image including the first user and a second image including the second user are combined.

[0008] In one embodiment of the present disclosure, the method further includes the steps of receiving a user input selecting a multi-party photo taking button from a first user after switching to the multi-party photo taking mode and before displaying a photo taking notification, capturing a first image including the first user using an image sensor of the first user terminal while the photo taking notification is displayed, receiving a second image including the second user from the second user terminal, and combining the first image and the second image to generate a composite image.

[0009] In one embodiment of the present disclosure, the method further includes the steps of receiving a user input for multi-party photography from a first user after switching to the multi-party photography mode and before displaying a photography notification, capturing a first image including the first user using an image sensor of the first user terminal while the photography notification is displayed, obtaining a screenshot of a second image including the second user displayed on the display, and combining the first image and the second image to generate a composite image.

[0010] In one embodiment of the present disclosure, the method further includes deactivating a multi-party photo taking button in response to determining that the second user terminal has selected the camera off button.

[0011] In one embodiment of the present disclosure, after switching to the multi-party shooting mode and before displaying the shooting notification, the method further includes displaying a shooting preparation notification on the display for a second time period, the second time period being longer than the first time period.

[0012] In one embodiment of the present disclosure, the method further includes deactivating a camera off button and a multi-party shooting mode exit button while the shooting preparation notification is displayed.

[0013] In one embodiment of the present disclosure, the method further includes a step of capturing a first image including the first user using an image sensor of the first user terminal for a third time while the capture notification is displayed, the first time being a predetermined time or more longer than the third time.

[0014] In one embodiment of the present disclosure, the time at which the third time for capturing the first image ends may be the same as the time at which the first time for displaying the capture notification ends, or may be before the time at which the first time for displaying the capture notification ends.

[0015] In one embodiment of the present disclosure, the method further includes a step of measuring a communication delay of data between the first user terminal and the second user terminal, and the timing of displaying a shooting preparation notification on the display is determined based on the measured communication delay, and the first image and the second image are captured at the same timing in the first user terminal and the second user terminal based on the measured communication delay.

[0016] In one embodiment of the present disclosure, the method further includes the steps of: displaying a plurality of photo themes on the display after switching to the multi-party photography mode and before displaying the photography notification; receiving a user input from a first user selecting one of the plurality of photo themes; displaying a first image sequence captured by an image sensor of the first user terminal in a first area of ​​the selected photo theme on the display; displaying a second image sequence received from the second user terminal in a second area of ​​the selected photo theme on the display; and receiving a user input from the first user selecting a multi-party photography button.

[0017] In one embodiment of the present disclosure, when multiple photo themes are displayed on a display, a visual object is displayed in the vicinity of the photo theme for which the second user has given feedback.

[0018] In one embodiment of the present disclosure, a first pose guide is displayed overlaid on a first sequence of images in a first region, and a second pose guide is displayed overlaid on a second sequence of images in a second region.

[0019] In one embodiment of the present disclosure, the composite image includes multiple layers, and an image captured by the user terminal that switched to multi-party shooting mode first, out of the first user terminal and the second user terminal, is placed in the upper layer.

[0020] In one embodiment of the present disclosure, after switching to the multi-party shooting mode, in response to determining that the number of users participating in the multi-party shooting mode is equal to or less than a predetermined number, the method further includes the steps of: displaying a first image sequence captured by an image sensor of the first user terminal in a first area of ​​the display; and displaying a second image sequence received from the second user terminal in a second area of ​​the display.

[0021] In one embodiment of the present disclosure, after switching to the multi-party capturing mode, in response to determining that the number of users participating in the multi-party capturing mode exceeds a predetermined number, the method further includes a step of displaying a second image sequence received from the second user terminal in a second area of ​​the display, wherein the first image sequence captured by the image sensor of the first user terminal is not displayed on the display.

[0022] In one embodiment of the present disclosure, the method further includes a step of displaying a first image sequence captured by an image sensor of the first user terminal in a first area of ​​the display in response to a user participating in the multi-party capture mode leaving and determining that the number of users participating in the multi-party capture mode is less than or equal to a predetermined number.

[0023] In one embodiment of the present disclosure, the method further includes transmitting the composite image to a second user terminal through a chat room.

[0024] In one embodiment of the present disclosure, in response to receiving a user input from the first user selecting an exit multi-party capture mode button, the composite image is transmitted to the second user terminal through the chat room.

[0025] A non-transitory computer-readable storage medium having instructions recorded thereon for causing a computer to perform the above method according to one embodiment of the present disclosure is provided.

[0026] A first user terminal according to one embodiment of the present disclosure includes a communication module, a display, a memory, and at least one processor connected to the memory and configured to execute at least one computer-readable program included in the memory, wherein the at least one program includes instructions for initiating a video call between a first user terminal associated with a first user and a second user terminal associated with a second user included in a chat room of an instant messaging application, switching to a multi-party shooting mode during the video call, displaying a shooting notification on the display for a first time period, and displaying a composite image on the display in which a first image including the first user and a second image including the second user are combined. [Effects of the Invention]

[0027] According to various embodiments of the present disclosure, a composite image of users participating in a video call taken at the same time may be provided.

[0028] According to various embodiments of the present disclosure, images to which various photographic themes are applied may be provided.

[0029] According to various embodiments of the present disclosure, by guiding a user to maintain a photo pose for a period of time during which a photo notification is displayed, which is longer than the actual photo taking time, it is possible to prevent a problem in which an unintended appearance or pose of the user is created in a composite image due to a communication delay between user terminals.

[0030] According to various embodiments of the present disclosure, even if multiple users take images in different environments, images can be generated that appear as if they were taken in a single space.

[0031] According to various embodiments of the present disclosure, the host can check the number of feedbacks on the photo theme from the guests through the host terminal, and can reflect the guests' reactions when selecting a photo theme to apply to the photo booth images.

[0032] According to various embodiments of the present disclosure, the user assumes a pose in accordance with the pose guide displayed on the display, so that even if there is a slight difference in the timing of photo taking between users, there is no deviation in the user's pose.

[0033] The effects of the present disclosure are not limited to those mentioned above, and other effects not mentioned will also be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains (hereinafter referred to as "a person skilled in the art") from the description in the claims. [Brief explanation of the drawings]

[0034] Embodiments of the present disclosure will now be described with reference to the following accompanying drawings, in which like reference numerals indicate like elements, but are not limited to the drawings, in which:

[0035] [Figure 1] FIG. 10 is a diagram showing an example in which a photo booth image including multiple users is displayed on a display in one embodiment of the present disclosure. [Figure 2] 1 is a schematic diagram illustrating a configuration in which an information processing system is communicably connected to a plurality of user terminals to provide an image capturing service during a video call in an embodiment of the present disclosure. [Figure 3] 1 is a block diagram illustrating an internal configuration of a user terminal and an information processing system according to an embodiment of the present disclosure. [Figure 4] FIG. 2 is a block diagram illustrating an internal configuration of a processor of a user terminal in an embodiment of the present disclosure. [Figure 5] FIG. 10 is a diagram illustrating an operation of switching to multi-party photography mode on the host terminal side in an embodiment of the present disclosure. [Figure 6] FIG. 10 is a diagram illustrating an operation of switching to a multi-party shooting mode on the guest terminal side in an embodiment of the present disclosure. [Figure 7] 10A and 10B are diagrams illustrating an example in which a shooting preparation notification and a shooting notification are displayed in an embodiment of the present disclosure. [Figure 8]FIG. 10 illustrates an example of a guest providing feedback on a photo theme in accordance with one embodiment of the present disclosure. [Figure 9] FIG. 10 is a diagram illustrating an example in which a composite image is displayed in an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram showing an example in which a photo booth image is displayed in one embodiment of the present disclosure. [Figure 11] FIG. 10 is a diagram showing an example in which a mode is switched to a photograph viewing mode in an embodiment of the present disclosure. [Figure 12] 1 is a flowchart illustrating a process for generating a composite image in one embodiment of the present disclosure. [Figure 13] 1 is a flowchart illustrating a method for capturing images during a video call in one embodiment of the present disclosure. [Figure 14] 10A and 10B are diagrams illustrating examples in which visual objects including stamps or user information are displayed in a composite image in accordance with one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0036] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, specific descriptions of well-known functions and configurations may be omitted if they may obscure the gist of the present disclosure.

[0037] In the accompanying drawings, identical or corresponding components are denoted by the same reference numerals. In addition, in the following description of the embodiments, duplicated descriptions of identical or corresponding components may be omitted. However, omission of a description of a component does not mean that such a component is not included in any of the embodiments.

[0038] The advantages and features of the disclosed embodiments, and methods for achieving them, will become apparent from the following detailed description of the embodiments taken in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, and may be embodied in various different forms. The present embodiments are provided solely so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0039] The terms used in this specification will be briefly explained, and the disclosed embodiments will be described in detail. The terms used in this specification are currently commonly used and generally selected as much as possible while taking into consideration the function of the present disclosure. However, these terms may change depending on the intentions of engineers in the relevant field, legal precedents, the emergence of new technologies, etc. In addition, in certain cases, the applicant may arbitrarily select terms, and in such cases, the meanings thereof will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined based on the meanings of the terms and the overall content of this disclosure, rather than simply by the names of the terms.

[0040] In this specification, the singular expression includes the plural expression unless the context clearly dictates otherwise. Furthermore, the plural expression includes the singular expression unless the context clearly dictates otherwise. Throughout the specification, when a part includes a certain element, this does not exclude other elements, and means that other elements may also be included, unless specifically stated to the contrary.

[0041] Additionally, the terms "module" or "module" as used herein refer to a software or hardware component, and a "module" or "module" performs a certain function. However, "module" or "module" is not limited to software or hardware. A "module" or "module" may be configured to reside on an addressable storage medium or to execute one or more processors. Thus, by way of example, a "module" or "module" may include components such as software components, object-oriented software components, class components, and task components, as well as at least one of processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The components and functionality provided within a "module" or "module" may be combined into fewer components and "modules" or "modules," or further separated into additional components and "modules" or "modules."

[0042] According to one embodiment of the present disclosure, a "module" or "unit" may be embodied with a processor and memory. "Processor" should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, "processor" may also refer to an application-specific semiconductor (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), etc. "Processor" may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such configuration. Also, "memory" should be broadly interpreted to include any electronic component capable of storing electronic information. "Memory" may refer to various types of processor-readable media such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage devices, registers, etc. Memory is said to be in electronic communication with a processor if the processor can read information from and / or write information to the memory. Memory that is integrated into a processor is in electronic communication with the processor.

[0043] In this disclosure, a "user" may refer to a user of an instant messaging application who is included in a chat room of the instant messaging application, or who is engaged in a video call with other users in the chat room. Alternatively, a "user" may refer to an image or a region of an image in which such user is displayed.

[0044] In this disclosure, a "user account" may refer to an account or data associated therewith that a user creates and uses with an instant messaging application. Also, a user account for an instant messaging application may refer to a user who uses the instant messaging application. Similarly, a user who uses instant messaging or an instant messaging-enabled chat room may refer to a user account for the instant messaging application.

[0045] In this disclosure, a "chat room" may refer to a virtual space or group that can be created by an instant messaging application installed on a computing device, and in which one or more users (or user accounts) can participate. For example, one or more user accounts can participate in or be included in a chat room and exchange various types of messages, files, etc. In addition, chat rooms may provide a Voice over Internet Protocol (VoIP) voice call function, a VoIP video call function, a live broadcast function (VoIP real-time video transmission function), and a multimedia content generation function, and voice calls, video calls, video streaming, multimedia content transmission, etc. may be performed between user accounts.

[0046] In this disclosure, a "host" may refer to a user who is making a video call and has requested a multi-party photo shoot or a user who has received host privileges from an existing host.

[0047] In this disclosure, a "guest" may refer to a user who accepts a host's request for multi-party photography and switches to multi-party photography mode.

[0048] FIG. 1 illustrates an example in which a photo booth image (140) including multiple users is displayed on a display in one embodiment of the present disclosure. A first operation (110) may represent a video call screen between multiple user terminals. Each of the multiple user terminals may display a screen including each user at any position and with any size on the display, and the video call screen may not be displayed in the same way on all user terminals. In one embodiment, a video call may be initiated and progressed between multiple user terminals associated with multiple users included in a chat room of an instant messaging application.

[0049] Although Figure 1 illustrates two users making a video call, this is not limited to this, and any number of users (e.g., six) that are at least a portion of the users (e.g., eight) included in a chat room of an instant messaging application may make a video call simultaneously.

[0050] In a first operation (110), one of multiple users in a video call can switch to multi-party shooting mode by selecting a multi-party shooting mode switch button (112) by touch input or the like. A multi-party shooting request can then be sent to the other users in the video call. The other users in the video call can switch to multi-party shooting mode by accepting the multi-party shooting request. The specific process of switching to multi-party shooting mode will be described in detail below with reference to FIGS. 5 and 6.

[0051] The second operation (120) may show an example in which a photo booth image (140) including multiple users is displayed on the display after multiple user terminals switch to multi-party photography mode. In this case, the photo booth image (140) may include composite images (132, 134) generated by combining images including each of the multiple users.

[0052] 1, the photo booth image 140 is illustrated as including two composite images 132 and 134, but is not limited thereto, and any number of composite images may be included in any position and any size in the photo booth image 140. For example, a user may set the number of composite images to be taken, the position and size of each composite image, etc., and a composite image may be generated by taking each user's image a number of times corresponding to the set number of composite images.

[0053] In one embodiment, a user (host) who selected the multiparty photography mode switching button 112 in the first operation 110 may then select the multiparty photo capture button 122 to generate and display composite images 132, 134. For example, after a user selects the multiparty photo capture button 122 to generate a first composite image 132, the user may again select the multiparty photo capture button 122 to generate and display a second composite image 134. In another example, after a user selects the multiparty photo capture button 122 to generate a first composite image 132, a second composite image 134 may be sequentially generated after a predetermined time (e.g., 5 seconds) has elapsed. This process may be repeated until a predetermined number of composite images have been generated or until the user stops taking photos.

[0054] The multiparty photo taking button 122 may be activated or displayed only on the user terminal of the user (host) who selected the multiparty photo taking mode switching button 112 in the first operation 110. Additionally, the multiparty photo taking button 122 may be deactivated if another user turns off the camera by selecting the camera off button 128.

[0055] In one embodiment, a user can select the effect button 124 to apply various visual effects to the composite images 132 and 134. Specifically, when the user selects the effect button 124, a list of effects that can be applied to the composite images 132 and 134 is displayed on the display. The user can select one or more effects from the list of effects, and the selected effects can be applied to the composite images 132 and 134. For example, it can be seen that an effect displaying user information (such as each user's ID, conversation name, nickname, etc. in an instant messaging application, e.g., User A, User B, etc.) has been applied to each of the composite images 132 and 134 in FIG. 1.

[0056] In this case, the effects that can be applied to the composite image (132, 134) may include, but are not limited to, decorative elements (e.g., floral pictures, firework effects, etc.), information elements (e.g., information about each user included in the composite image, the date and time of the photo, text, etc.), and image attribute elements (e.g., brightness, saturation, etc. of the composite image).

[0057] After one or more effects are selected, the user can move the applied effect to the desired position using drag and drop, or apply it to the desired size using gestures such as spread or pinch.

[0058] In one embodiment, the user can select a gallery button (126) associated with the captured image. When the user selects the gallery button (126), the user can review images stored on the user device (e.g., previously captured composite images, photo booth images, etc.). The user can also select images stored on the user device to add to the photo booth images (140).

[0059] In one embodiment, a user may select a theme button 130 associated with a theme to be applied to the photo booth image 140. For example, when a user selects the theme button 130, a list of themes applicable to the photo booth image 140 may be displayed on the display, and when the user selects one theme from the list, the photo booth image 140 with the selected theme applied may be displayed on the display. Alternatively, a separate theme may be selected and applied to each of the composite images 132, 134. Additionally or alternatively, a user may select a theme in advance before selecting the multi-person photo button 122.

[0060] In one embodiment, a user can delete one or all of the composite images (132, 134) from the photo booth image (140) by selecting a delete button (136, 138) corresponding to each of the composite images (132, 134). After at least some of the composite images (132, 134) have been deleted, the user can generate new composite images to replace the deleted composite images.

[0061] In one embodiment, the delete buttons 136, 138 may be activated or displayed only on the user terminal of the user (host) who selected the multi-party photography mode switching button 112 in the first operation 110. Alternatively, the delete buttons 136, 138 may be activated for all users who have switched to multi-party photography mode, but if a majority of all users do not select the delete button for the same composite image, the composite image may be deleted from the photo booth images 140.

[0062] The sizes and positions of the images, buttons, etc. shown and described in Fig. 1 are illustrative and are not limiting. For example, some buttons may be added or omitted, or may be configured in sizes and positions different from those shown.

[0063] 2 is a schematic diagram illustrating a configuration in which an information processing system 230 is communicatively connected to multiple user terminals 210_1, 210_2, and 210_3 to provide a video call image capture service according to one embodiment of the present disclosure. The information processing system 230 may include a system capable of providing an instant messaging service, a system capable of providing a social networking service, and / or a system capable of providing a video call image capture service. In one embodiment, the information processing system 230 may include one or more server devices and / or databases, or one or more cloud computing service-based distributed computing devices and / or distributed databases, capable of storing, providing, and executing computer-executable programs (e.g., downloadable applications) and data related to the instant messaging service, the social networking service, and / or the video call image capture service. For example, the information processing system 230 may include separate systems (e.g., servers) for the instant messaging service, the social networking service, and / or the video call image capture service.

[0064] The instant messaging service, image capture service during a video call, etc. provided by the information processing system 230 may be provided to users through an instant messaging application installed in each of the multiple user terminals 210_1, 210_2, 210_3. For example, the instant messaging service may include a text messaging service, a voice messaging service, a video call service, a voice call service, a video streaming service, a social network service, an image capture service during a video call, etc. between users of the instant messaging application.

[0065] A plurality of user terminals 210_1, 210_2, and 210_3 can communicate with an information processing system 230 through a network 220. The network 220 can be configured to enable communication between the plurality of user terminals 210_1, 210_2, and 210_3 and the information processing system 230. Depending on the installation environment, the network 220 can be configured as a wired network such as Ethernet, a wired home network (Power Line Communication), a telephone line communication device, and RS-serial communication, a mobile communication network, a wireless network such as WLAN (Wireless LAN), Wi-Fi, Bluetooth, and ZigBee, or a combination thereof. The communication method is not limited, and may include not only communication methods that utilize communication networks that the network (220) may include (for example, mobile communication networks, wired Internet, wireless Internet, broadcasting networks, satellite networks, etc.), but also short-range wireless communication between user terminals (210_1, 210_2, 210_3).

[0066] 2, a mobile phone terminal 210_1, a tablet terminal 210_2, and a PC terminal 210_3 are illustrated as examples of user terminals, but are not limited thereto, and the user terminals 210_1, 210_2, and 210_3 may be any computing device capable of wired and / or wireless communication and capable of installing and running an instant messaging application, etc. For example, the user terminal may include a smartphone, a mobile phone, a navigation system, a computer, a notebook, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a tablet PC, a game console, a wearable device, an internet of things (IoT) device, a virtual reality (VR) device, an augmented reality (AR) device, etc. Also, although FIG. 2 illustrates three user terminals (210_1, 210_2, 210_3) communicating with information processing system (230) through network (220), this is not limited thereto, and a different number of user terminals may be configured to communicate with information processing system (230) through network (220).

[0067] In one embodiment, each of the user terminals 210_1, 210_2, and 210_3 can receive information or data from or transmit information or data to other user terminals via the network 220. For example, each of the user terminals 210_1, 210_2, and 210_3 can transmit or receive a user input selecting a multi-party photo capture button, a user input selecting a multi-party photo capture mode exit button, an image containing a user, a user input selecting one of a plurality of photo themes, and / or an image sequence, etc.

[0068] FIG. 3 is a block diagram illustrating the internal configuration of a user terminal 210 and an information processing system 230 according to one embodiment of the present disclosure. The user terminal 210 may refer to any computing device capable of executing an instant messaging application or the like and capable of wired / wireless communication, and may include, for example, the mobile phone terminal 210_1, tablet terminal 210_2, or PC terminal 210_3 of FIG. 2. As illustrated, the user terminal 210 may include a memory 312, a processor 314, a communication module 316, and an input / output interface 318. Similarly, the information processing system 230 may include a memory 332, a processor 334, a communication module 336, and an input / output interface 338. As illustrated in FIG. 3, the user terminal 210 and the information processing system 230 may be configured to communicate information and / or data over the network 220 using their respective communication modules 316 and 336. Additionally, the input / output device 320 may be configured to input information and / or data to the user terminal 210 via the input / output interface 318, or to output information and / or data generated by the user terminal 210.

[0069] The memory (312, 332) may include any non-transitory computer-readable storage medium. In one embodiment, the memory (312, 332) includes a read only memory (ROM). memory), disk drives, SSDs (solid state permanent mass storage devices such as flash memory, As another example, a non-perishable mass storage device such as a ROM, SSD, flash memory, or disk drive may be included in the user terminal 210 or the information processing system 230 as a separate permanent storage device distinct from the memory. The memory 312, 332 may also store an operating system and at least one program code (e.g., code for an application associated with an instant messaging service, a social networking service, or a video call image capture service).

[0070] These software components may be loaded from a computer-readable recording medium separate from the memory 312, 332. These separate computer-readable recording media may include recording media directly connectable to the user terminal 210 and the information processing system 230, such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, or a memory card. As another example, the software components may be loaded into the memory 312, 332 via the communication module 316, 336 rather than from a computer-readable recording medium. For example, at least one program may be loaded into the memory 312, 332 based on a computer program (e.g., an application associated with an instant messaging service, a social networking service, or a video call image capture service) that is installed by a file provided over the network 220 by a developer or a file distribution system that distributes application installation files.

[0071] The processors 314, 334 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processors 314, 334 by the memory 312, 332 or the communication modules 316, 336. For example, the processors 314, 334 may be configured to execute instructions received according to program code stored in a storage device, such as the memory 312, 332.

[0072] The communication modules 316, 336 may provide a configuration or function for the user terminal 210 and the information processing system 230 to communicate with each other via the network 220, and may provide a configuration or function for the user terminal 210 and / or the information processing system 230 to communicate with other user terminals or other systems (e.g., another cloud system). For example, a request or data (e.g., a request for multi-party photography) generated by the processor 314 of the user terminal 210 in accordance with program code stored in a storage device such as the memory 312 may be transmitted to the information processing system 230 via the network 220 under the control of the communication module 316. Conversely, a control signal or command provided under the control of the processor 334 of the information processing system 230 may be received by the user terminal 210 via the communication module 316 of the user terminal 210 via the communication module 336 and the network 220. For example, the user terminal (210) can receive an image containing other users from the information processing system (230).

[0073] The input / output interface 318 may be a means for interfacing with the input / output device 320. For example, the input device may be a device such as a camera including an audio sensor and / or an image sensor, a keyboard, a microphone, or a mouse, and the output device may be a device such as a display, a speaker, or a haptic feedback device. As another example, the input / output interface 318 may be a means for interfacing with a device that integrates components or functions for performing input and output, such as a touchscreen. While FIG. 3 illustrates the input / output device 320 as not being included in the user terminal 210, this is not limiting and the input / output device 320 may be configured as a single device together with the user terminal 210. Furthermore, the input / output interface 338 of the information processing system 230 may be a means for interfacing with an input or output device (not shown) that is connected to the information processing system 230 or that may be included in the information processing system 230. Although the input / output interfaces (318, 338) are illustrated in FIG. 3 as elements configured separately from the processors (314, 334), this is not limiting, and the input / output interfaces (318, 338) may be configured to be included in the processors (314, 334).

[0074] The user terminal 210 and the information processing system 230 may include more components than those shown in FIG. 3 . However, it is not necessary to clearly illustrate most of the conventional components. In one embodiment, the user terminal 210 may be embodied to include at least a portion of the input / output device 320 described above. The user terminal 210 may also include other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, a database, etc. For example, if the user terminal 210 is a smartphone, it may include components typically included in a smartphone, such as an acceleration sensor, a gyro sensor, a microphone module, a camera module, various physical buttons, buttons using a touch panel, an input / output port, a vibrator for vibration, etc.

[0075] According to one embodiment, the processor 314 of the user terminal 210 may be configured to run an instant messaging application or a web browser application that provides an instant messaging service, including a service for taking pictures during a video call. In this case, program code associated with the application may be loaded into the memory 312 of the user terminal 210. While the application is running, the processor 314 of the user terminal 210 may receive information and / or data provided by the input / output device 320 through the input / output interface 318 or from the information processing system 230 through the communication module 316, process the received information and / or data, and store the processed information and / or data in the memory 312. The information and / or data may also be provided to the information processing system 230 through the communication module 316.

[0076] During operation of the instant messaging application, the processor 314 may receive voice data, text, images, video, etc. input or selected through an input device, such as a touch screen, keyboard, camera including an audio sensor and / or image sensor, microphone, etc., connected to the input / output interface 318, and may store the received voice data, text, images, and / or video in the memory 312 or provide the received voice data, text, images, and / or video to the information processing system 230 via the communication module 316 and the network 220. In one embodiment, the processor 314 may receive user input, input through the input device, to select a graphical object displayed on the display, and provide data / requests corresponding to the received user input to the information processing system 230 via the network 220 and the communication module 316.

[0077] The processor 314 of the user terminal 210 may transmit information and / or data to the input / output device 320 via the input / output interface 318 for output. For example, the processor 314 of the user terminal 210 may output the processed information and / or data via the output device 320, such as a display-capable device (e.g., a touchscreen, display, etc.) or an audio-capable device (e.g., a speaker). In one embodiment, the processor 314 may display a composite image on the display of the user terminal 210, in which images including each of the multiple users are composited.

[0078] The processor 334 of the information processing system 230 may be configured to manage, process, and / or store information and / or data received from the user terminals 210 and / or the external systems. The information and / or data processed by the processor 334 may be provided to the user terminals 210 through the communication module 336 and the network 220.

[0079] 4 is a block diagram showing the internal configuration of the processor 314 of the user terminal according to one embodiment of the present disclosure. As shown, the processor 314 may include an image capture unit 410, an image pre-processing unit 420, an image synthesis unit 430, an image post-processing unit 440, and the like.

[0080] The image capturing unit 410 may receive an image including the user captured using an image sensor of the user terminal, or may capture / generate a screenshot of an image including the user displayed on a display. For example, the image capturing unit 410 may receive an image including the user (host) from an image sensor in response to the user (host) selecting a multi-party photo capture button. The image capturing unit 410 may also receive an image including another user (guest) from another user terminal. In another example, the image capturing unit 410 may capture / generate a screenshot of an image including another user displayed on a display. In yet another example, the image capturing unit 410 may receive an image including the user (guest) captured using an image sensor in response to a capture signal received from another user terminal when the other user (host) selects a multi-party photo capture button.

[0081] The image capture unit 410 may output a capture notification to the user terminal for a first time period. For example, the image capture unit 410 may output a message such as "Please do not move while taking a photo" on the display of the user terminal or output an audio signal (e.g., a beep) indicating that a photo is being taken. Additionally, the image capture unit 410 may output a capture preparation notification to the user terminal for a second time period before displaying the capture notification. For example, the image capture unit 410 may display a timer on the display or output an audio signal that times out after the second time period has elapsed. In one embodiment, the timing for displaying the capture preparation notification may be determined based on a data communication delay measured between the user terminal and another user terminal. That is, the image capture unit 410 may capture an image including a user at the same time as an image including another user is captured at another user terminal based on a data communication delay measured between the user terminal and another user terminal.

[0082] In one embodiment, while the photo notification is displayed on the display of the user terminal for a first time, the image capturing unit (410) may capture images including the user using the image sensor of the user terminal for a third time and receive the captured images from the image sensor. In this case, the first time may be longer than the third time by a predetermined time (e.g., 1 second) or more, and the end of the third time may be the same as the end of the first time or may occur before the end of the first time.

[0083] Through this configuration, the user is guided to maintain a shooting pose while the shooting notification is displayed, which is longer than the actual shooting time, thereby preventing the problem of a composite image being created with the user in an unintended pose or appearance due to communication delays between user terminals.

[0084] The image preprocessing unit 420 may preprocess images captured by the image capturing unit 410, images acquired through screenshots, and / or images received from other user terminals. In one embodiment, the image preprocessing unit 420 may determine a user's outline included in a user image captured by the image capturing unit 410 and separate a background image from the user based on the determined outline. Through this configuration, a composite image may be generated based on images from which background elements have been removed, thereby generating images that appear as if multiple users had taken photos in a single space.

[0085] In another embodiment, the image pre-processing unit 420 may normalize color attributes (e.g., including brightness and saturation) representing the hue or color intensity, brightness, and shading of an image captured by the image capturing unit 410, an image acquired through a screenshot, and / or an image of another user received from another user terminal. Through this configuration, even if multiple users capture images in different environments, images can be generated as if they were taken in the same space.

[0086] In yet another embodiment, the image preprocessing unit (420) can resize images containing multiple users so that the multiple users appear similar in size in the composite image, or rotate the images so that the multiple users appear in the same / similar orientation.

[0087] The image synthesis unit 430 can generate a composite image by synthesizing images including each of a plurality of users. For example, the image synthesis unit 430 can generate a composite image by synthesizing an image including the user (host) and an image including other users (guests). In one embodiment, the image including other users (guests) may be received from the terminal of the other users. Alternatively or additionally, the image including other users (guests) may correspond to a screenshot captured on the terminal of the user (host).

[0088] The image composition unit 430 may generate a composite image including multiple layers such that an image captured by a user terminal that first switched to multi-party photography mode among multiple user terminals is placed in the top layer. For example, since the host terminal switches to multi-party photography mode upon receiving a request for multi-party photography, an image including the host may be placed in the top layer. In another example, the image composition unit 430 may change the arrangement of layers in response to a request from the host to change the position of the layers, or the host may arrange the layers according to predetermined layer setting rules.

[0089] The image post-processing unit 440 can apply a user-selected theme and / or effect to the composite image. Alternatively, applying a user-selected theme and / or effect to the composite image can be performed by the image compositing unit 430. For example, applying a background or theme to the composite image can be performed by placing the background image in the bottom layer. In yet another example, displaying an effect on the composite image can be performed by placing the effect image in the top layer.

[0090] The internal configuration of the processor (314) illustrated in FIG. 4 is merely exemplary, and in some embodiments, other components may be included in addition to the illustrated internal components, or some components may be omitted. For example, if some of the internal components are omitted, the processor (334) of the information processing system and / or another user terminal may be configured to perform the functions of the omitted internal components. For example, the processor (334) of the information processing system and / or another user terminal may perform image pre-processing. Also, although the internal configuration of the processor (314) is described in FIG. 4 as being divided by function, this does not necessarily mean that the components are physically divided. Although the image capture unit (410), image pre-processing unit (420), image synthesis unit (430), and image post-processing unit (440) are described separately above, this is for ease of understanding and is not limiting.

[0091] FIG. 5 illustrates an operation for switching to multi-party photography mode on the host terminal according to an embodiment of the present disclosure. A first operation (510) illustrates an example in which a user selects the multi-party photography mode switch button on the host terminal during a video call, and an area (512) for selecting the attributes of the photo booth image to be captured and a button (514) for switching to multi-party photography mode are displayed on the host terminal's display. While FIG. 5 illustrates the area (512) as a selection area for the number of composite images to be included in the photo booth image, this is not a limitation. For example, the host can pre-set the position, size, theme, etc. of each composite image in area (512).

[0092] A second operation (520) illustrates an example in which the host terminal switches to multi-party capture mode in response to receiving a user input selecting button (514) on the host terminal. As illustrated, a first image sequence captured by an image sensor of the host terminal may be displayed in a first area (522) of the display. A second image sequence received from a guest terminal may be displayed in a second area (524) of the display. Here, the guest terminal may be a user terminal of another user who is performing a video call with the host and has switched to the multi-party capture mode generated by the host. Additionally, the host may adjust the position, size, etc. of the areas in which the first and second image sequences are displayed in the multi-party capture mode.

[0093] In one embodiment, the host can set / change the number of images included in the composite image in area 530. For example, the host can select the number "2" in area 530 to set two composite images to be taken consecutively. Alternatively, the host can select the number "4" in area 530 to set four composite images to be taken consecutively. While area 530 is illustrated as allowing the host to select one of 1 to 4, this is not limiting and the host can select any number.

[0094] In one embodiment, when the host selects the multi-person photo button 526, a photo preparation notification may be displayed. Then, the multi-person photo is taken while the photo notification is displayed, and a composite image (or photo booth image) may be generated. The generated composite image may be displayed on the displays of the host terminal and guest terminal for the host and guests to review.

[0095] Alternatively, when the host selects the multi-party photo taking button (526), ​​video recording can begin and a composite video (or photo booth video) can be generated. Here, the composite video (or photo booth video) can include multiple composite images (or multiple photo booth images). In this case, the composite video can be recorded for a predetermined time or for a time input by the user (e.g., the time the user holds down the multi-party photo taking button (526)). The generated composite video (or photo booth video) can be saved or played at a predetermined speed (e.g., 2x or 3x).

[0096] In one embodiment, the host may select the Exit button (528) before or after completing the photo booth image capture. When the host selects the Exit button (528) before completing the photo booth image capture, the multi-party photo capture mode may be terminated on the host terminal. In this case, host authority may be granted to one of the guests included in the multi-party photo capture mode (e.g., the guest who enters the multi-party photo capture mode after the host). In this case, the host authority may be delegated after a predetermined time (e.g., 5 seconds) has elapsed after the multi-party photo capture mode is terminated on the host terminal, and if the host re-connects to the multi-party photo capture mode before the predetermined time has elapsed, the host authority may be maintained.

[0097] Alternatively, if the host selects the exit button (528) after completing the photo booth image capture, the photo booth image may be saved in the host terminal and sent to the guest terminal through the chat room where the video call was made or the 1:1 chat room with the guest.

[0098] 6 illustrates an operation of switching to multi-party photography mode on the guest terminal side in one embodiment of the present disclosure. In the first operation (610), when the guest terminal receives a request to take a multi-party photo from the host terminal during a video call, an area (612) for deciding whether to accept the request is displayed on the display of the guest terminal.

[0099] In one embodiment, area 612 may display attribute information of the photo booth image being captured. For example, while FIG. 6 illustrates information related to the theme of the photo booth image, the number, position, size, etc. of composite images may be displayed, without limitation. Additionally, information such as the host of the multi-party photo mode and a list of participating guests may be displayed. Guests can select a first button 614 in area 612 to switch to multi-party photo mode, or select a second button 616 to decline multi-party photo capture.

[0100] The second operation (620) illustrates an example in which a user selects the first button (614) on the guest terminal to switch to multi-party photo mode. Unlike FIG. 5, instead of displaying the multi-party photo button on the guest terminal display, a third button (626) may be displayed, allowing the guest to check the theme currently applied to the photo booth image and / or applicable theme information. After selecting the third button (626), the guest can select the theme information they wish to apply to the photo booth image. This will be described in detail below with reference to FIG. 8.

[0101] As shown, a first region of the display (622) may display a first sequence of images received from the host terminal, and a second region of the display (624) may display a second sequence of images captured by an image sensor in the guest terminal.

[0102] In one embodiment, a guest can select the Exit button (628) to exit the multi-party photography mode before or after completing the photo booth image capture. In this case, the multi-party photography mode ends on the guest terminal, and the video call screen before switching to the multi-party photography mode may be displayed.

[0103] 7 is a diagram illustrating an example of displaying a photo-taking preparation notification and a photo-taking notification according to an embodiment of the present disclosure. First through third actions (710, 720, and 730) illustrate an example in which a photo-taking preparation notification is output on the display of a user terminal for a second time (t2) in response to the host selecting a multi-party photo-taking button. For example, as illustrated, a timer that times out as the second time (t2) elapses may be displayed on the display of the user terminal. In this case, the time at which the photo-taking preparation notification begins to be displayed on each of the host terminal and the guest terminal may be determined based on the data communication delay measured between the host terminal and the guest terminal.

[0104] A fourth operation (740) illustrates an example in which a photographing notification is displayed on the display of the user terminal for a first time (t1) after the photographing preparation notification is output. For example, a message such as "Photographing in progress" may be output on the display of the user terminal. Additionally or alternatively, a guide sound may be output to indicate that a photograph is being taken.

[0105] In one embodiment, while a capture notification is displayed on the display of the user terminal for a first time (t1), an image including the user may be captured using an image sensor of the user terminal for a third time (t3). In this case, the first time (t1) may be longer than the third time (t3) by a predetermined time (e.g., 1 second) or more, and the end of the third time (t3) may be the same as the end of the first time (t1). Alternatively, the end of the third time (t3) may be before the end of the first time (t1). The third time (t3) at which the image is captured may vary depending on the capture environment, the specifications of the image sensor, etc. For example, an image may be captured one second after the capture notification is displayed, and the first time (t1) and the third time (t3) may end when the image capture is completed.

[0106] In one embodiment, the camera off button and the exit multi-party capture mode button may be deactivated while the capture preparation notification or capture notification is displayed. Additionally, the host may select a button to abort capture while the capture preparation notification or capture notification is displayed. If the host selects the button to abort capture, the display of the capture preparation notification or capture notification is aborted, and the camera off button and the exit multi-party capture mode button may be reactivated.

[0107] 8 is a diagram illustrating an example of a guest providing feedback on a photo theme in an embodiment of the present disclosure. A first operation (810) illustrates an example of a screen displayed on the display of the guest terminal as the guest terminal switches to multi-party shooting mode.

[0108] A second operation 820 illustrates an example in which, after the guest terminal switches to the multi-party shooting mode and before the shooting notification is displayed, the guest selects button 812 to display multiple photo themes that can be applied to the photo booth image / composite image. The guest can then provide feedback (e.g., positive feedback or negative feedback) on a particular photo theme by selecting at least one of visual objects 822, 824, 826 displayed on the previews (e.g., thumbnails) of the multiple photo themes.

[0109] A third operation (830) illustrates an example in which, in response to a guest selecting a visual object (824) to provide feedback (or positive feedback) for a particular photo theme, a visual object (834) indicating the feedback is displayed adjacent to the selected photo theme. As a guest provides feedback for a particular photo theme, the visual object (834) displayed for the selected photo theme is distinct from the other visual objects (832, 836) for which the guest has not provided feedback, allowing the guest to easily identify the photo theme for which the guest has provided feedback.

[0110] Additionally, the host can check the number of feedbacks given by guests to the photo theme through the host terminal. Through this configuration, the host can reflect the guests' opinions when selecting a photo theme to apply to the photo booth image. Alternatively, the photo theme with the most positive feedbacks may be automatically applied to the photo booth image, or a message may be provided to the host suggesting that the photo theme be changed to the photo theme with the most positive feedbacks.

[0111] After the host selects a photo theme to apply to the photo booth images, the display may display a first sequence of images captured by the image sensor of the host terminal in a first area of ​​the selected photo theme, and a second sequence of images captured by the image sensor of the guest terminal in a second area of ​​the selected photo theme.

[0112] In one embodiment, a photo theme may include a pose guide. Here, the pose guide may be a visual guide displayed on a display to help a user easily imitate a particular pose. For example, in a first region of a selected photo theme, a first pose guide may be displayed as an overlay on a first image sequence, and in a second region, a second pose guide may be displayed as an overlay on a second image sequence. This configuration guides a user to take photos in various poses, eliminating the need for the user to invent poses and allowing for the capture of a highly composed composite image. Furthermore, because the user poses according to the pose guide displayed on the display, the user's pose may not deviate even if there is a slight difference in the user's timing when taking photos.

[0113] 9 is a diagram showing an example of a composite image displayed according to an embodiment of the present disclosure. The multiple users included in each of the composite images displayed in the first through fifth examples (910, 920, 930, 940, and 950) of FIG. 9 may correspond to users who have switched to multi-party capture mode. The first through fifth examples (910, 920, 930, 940, and 950) are examples of cases where 2 to 6 users have switched to multi-party capture mode.

[0114] In one embodiment, the composite image may include multiple layers, with the image / image sequence of a user who switched to multi-party capture mode first being placed in a higher layer. For example, in the third example (930), the user placed in the lower layer may be the user who switched to multi-party capture mode earlier than the user placed in the upper layer.

[0115] The composite image may be generated by combining images including each of multiple users, and the relative positions or sizes (layout) of users within the composite image may change each time a user who has switched to multi-party capture mode is added or removed. For example, a user positioned at the top of the second example (920) may be moved to the left of their existing position in the third example (930), to which a user who has switched to multi-party capture mode has been added. Furthermore, the size of the image including each user in the third example (930) may be larger than the size of the image including each user in the fourth example (940), to which a user who has switched to multi-party capture mode has been added.

[0116] The position or size of the users within the composite image may be determined based on predetermined user placement rules and the number of users who have switched to multi-party capture mode, or may be changed based on user input from the host that changes the position or size of each user.

[0117] FIG. 10 is a diagram illustrating an example of a photo booth image displayed according to an embodiment of the present disclosure. In the first to fifth examples (1010, 1020, 1030, 1040, and 1050) of FIG. 10, multiple users included in each grid of the photo booth image may correspond to users who have switched to multi-party photography mode. While FIG. 10 illustrates each grid as including one user, this is not limited to this. The first to fifth examples (1010, 1020, 1030, 1040, and 1050) are examples in which 2 to 6 users have switched to multi-party photography mode, respectively.

[0118] A photo booth image may be generated by combining images containing multiple users, and the relative position and size of the grid may change each time a user who switches to multi-party photography mode is added or removed. For example, in the photo booth image of the second example (1020), it can be seen that the upper grid is moved to the left from its existing position and reduced in size by half after a user is added in the third example (1030). The positions of the grid or users in the photo booth image may be determined based on predetermined placement rules and the number of users who switch to multi-party photography mode, or may be changed based on user input from the host that changes the positions of the grid or users.

[0119] FIG. 11 illustrates an example of switching to a photo-taking viewing mode in an embodiment of the present disclosure. A first operation (1110) illustrates an example in which a user attempts to switch to multi-party photo-taking mode, but the number of users participating in the multi-party photo-taking mode exceeds a predetermined number (e.g., six), causing the user to switch to photo-taking viewing mode instead of photo-taking mode. For example, area (1112) displays image sequences received from user terminals already participating, while image sequences captured by the image sensor of the user terminal that has switched to photo-taking viewing mode may not be displayed in area (1112). In other words, a user who has switched to photo-taking viewing mode can view the multi-party photo-taking process of other users who were in a video call, but cannot participate in the multi-party photo-taking themselves.

[0120] Alternatively, if a user switches to multi-party shooting mode and the number of users participating in the multi-party shooting mode is less than a predetermined number, the newly switched user may switch to multi-party shooting mode instead of photo-taking and viewing mode. In this case, a first image sequence captured by the image sensor of a user device already participating may be displayed in a first area of ​​the display, and a second image sequence received from the newly switched user device may be displayed in a second area of ​​the display. If there are three or more users participating, a third image sequence captured by the image sensor of another user device may be displayed in a similar manner in a third area of ​​the display.

[0121] In one embodiment, when a user switches to the photo viewing mode, the user can select the photo theme button (1114) to view multiple photo themes and provide feedback (e.g., positive or negative feedback) for some of the photo themes based on their preferences.

[0122] A second operation (1120) illustrates an example in which some of the users participating in the multi-party capture mode leave, reducing the number of participating users below a predetermined number. In this case, a join button (1124) for joining the multi-party capture mode may be displayed on the display of a user terminal viewing in photo capture viewing mode. A viewing user can select the join button (1124) to leave the photo capture viewing mode and switch to the multi-party capture mode. In response to a user selecting the join button (1124), a sequence of images captured by an image sensor of the user terminal may be displayed in a portion of the display area (1122).

[0123] 12 is a flowchart illustrating a process for generating a composite image in an embodiment of the present disclosure. After a video call is initiated between a first user terminal associated with a first user and a second user terminal associated with a second user included in a chat room of an instant messaging application, the host terminal (1210) may switch to multi-party photo capture mode (1212). For example, the host terminal may switch to multi-party photo capture mode by receiving a user input such as a touch input to select a multi-party photo capture mode switch button. When the host terminal (1210) switches to multi-party photo capture mode (1212), the host terminal (1210) may send a multi-party photo capture request to the guest terminal (1220) (1214).

[0124] Thereafter, the guest terminal (1220) may switch to multi-party photography mode (1222) if it receives user input agreeing to the multi-party photography request.

[0125] The host terminal 1210 can then receive user input from the user clicking a capture button 1216. As the host terminal 1210 receives user input from the user clicking a capture button, the host terminal 1210 can transmit a capture signal to the guest terminal 1220 1218.

[0126] In response to the host terminal 1210 receiving user input clicking a capture button, the host terminal 1210 can capture 1230 an image including the host using an image sensor. Similarly, upon receiving a capture signal from the host terminal 1210, the guest terminal 1220 can capture 1224 an image including the guest using an image sensor. In one embodiment, the data communication delay between the host terminal 1210 and the guest terminal 1220 can be measured. In this case, the time at which the image is captured at the host terminal 1210 and the time at which the image is captured at the guest terminal 1220 can be synchronized based on the measured data communication delay.

[0127] Thereafter, the host terminal (1210) can receive an image including the guest from the guest terminal (1220) (1226). Alternatively, the host terminal (1210) can capture / generate a screenshot of the image including the guest.

[0128] Thereafter, the host terminal (1210) can generate a composite image by combining the image including the host and the image including the guest (1232).

[0129] 13 is a flowchart illustrating a method 1300 for capturing an image during a video call according to one embodiment of the present disclosure. The method 1300 may be executed by at least one processor of a first user terminal. The processor may initiate a video call (S1310) between a first user terminal associated with a first user and a second user terminal associated with a second user included in a chat room of an instant messaging application.

[0130] The processor may then switch to a multi-party photo taking mode during the video call (S1320). In one embodiment, the processor may deactivate a multi-party photo taking button in response to determining that the second user terminal has selected the camera off button.

[0131] The processor may then display a capture notification on the display of the first user terminal for a first time period (S1330).

[0132] In one embodiment, after switching to the multi-party shooting mode and before displaying the shooting notification, the processor may receive a user input from a first user selecting a multi-party photo taking button, and while the shooting notification is displayed, may capture a first image including the first user using an image sensor of the first user terminal. After receiving a second image including the second user from the second user terminal, the processor may combine the first image and the second image to generate a composite image. Instead of receiving the second image from the second user terminal, the processor may also capture a screenshot of the second image including the second user displayed on the display, and combine the first image and the second image to generate a composite image.

[0133] In one embodiment, the processor may display a capture ready notification on the display for a second time period after switching to the multi-party capture mode and before displaying the capture notification, where the second time period may be longer than the first time period. The processor may also deactivate the camera off button and the exit multi-party capture mode button while displaying the capture ready notification.

[0134] In one embodiment, the processor may measure a data communication delay between the first user terminal and the second user terminal, and the timing for displaying the image capture preparation notification on the display may be determined based on the measured communication delay, and the first and second images may be captured at the same time in the first and second user terminals based on the measured communication delay.

[0135] In one embodiment, the processor may capture a first image including the first user using an image sensor of the first user terminal for a third time period while the photo-taking notification is displayed. In this case, the first time period may be longer than the third time period by a predetermined time or more. Furthermore, the end of the third time period for capturing the first image may be the same as the end of the first time period during which the photo-taking notification is displayed, or may occur before the end of the first time period during which the photo-taking notification is displayed.

[0136] In one embodiment, after entering the multi-party photography mode and before displaying a photography notification, the processor may display multiple photo themes on the display, receive user input from a first user selecting one of the multiple photo themes, display a first image sequence captured by an image sensor of the first user terminal in a first area of ​​the selected photo theme on the display, display a second image sequence received from a second user terminal in a second area of ​​the selected photo theme on the display, and receive user input from the first user selecting a multi-party photography button. When the processor displays the multiple photo themes on the display, a visual object may be displayed near the photo theme for which the second user has provided feedback. Also, a first pose guide may be displayed as an overlay on the first image sequence in the first area, and a second pose guide may be displayed as an overlay on the second image sequence in the second area.

[0137] In one embodiment, after switching to the multi-party capture mode, the processor may, in response to determining that the number of users participating in the multi-party capture mode is less than or equal to a predetermined number, display a first image sequence captured by an image sensor of the first user terminal in a first area of ​​the display and display a second image sequence received from the second user terminal in a second area of ​​the display.

[0138] Alternatively, after switching to the multi-party capture mode, the processor may display a second image sequence received from the second user terminal in the second area of ​​the display in response to determining that the number of users participating in the multi-party capture mode exceeds a predetermined number. At this time, the first image sequence captured by the image sensor of the first user terminal may not be displayed on the display. Thereafter, the processor may display the first image sequence captured by the image sensor of the first user terminal in the first area of ​​the display in response to determining that a user participating in the multi-party capture mode has left the multi-party capture mode and the number of users participating in the multi-party capture mode is less than the predetermined number.

[0139] The processor may then display on the display a composite image obtained by combining the first image including the first user and the second image including the second user (S1340). In one embodiment, the composite image may include multiple layers, and an image captured by the user terminal that switched to the multi-party shooting mode first, of the first and second user terminals, may be placed in a higher layer.

[0140] In one embodiment, the processor may transmit the composite image to the second user terminal through a chat room in response to receiving user input from the first user selecting an exit button for multi-party capture mode. For example, the composite image may be uploaded to a chat room including the first user and the second user, and the uploaded image may be downloaded to the second user terminal.

[0141] 12 and 13 and the above description are merely examples and may be implemented differently in other embodiments. For example, the order of some steps may be changed, some steps may be omitted, other configurations may be added, and some steps described as being performed sequentially may be performed simultaneously.

[0142] 14 illustrates an example in which a stamp (1414) or visual object (1424) containing user information is displayed on a composite image according to an embodiment of the present disclosure. A first operation (1410) illustrates an example in which a selected stamp (1414) is displayed on the composite image in response to a user input selecting one of a plurality of stamp templates in an area (1412). In one embodiment, a user (or host) can add the stamp (1414) to the composite image by selecting at least one of the plurality of stamp templates, and can change the position of the added stamp (1414), rotate the displayed stamp, or remove the stamp.

[0143] The stamp templates may include D-day, anniversary phrases, greetings, etc., and may be presented in a variety of styles (fonts, decorative effects, etc.). Additionally, users can modify the content of the stamp templates or add new stamp templates.

[0144] In one embodiment, the template indicating D-day may correspond to one associated with a user in the chat room or a user in a video call. For example, a template indicating D-day may be generated based on the current date and anniversary information associated with the guest and host stored in the instant messaging application. In yet another example, if the current date corresponds to the birthday of at least some of the users in the chat room, a "Happy Birthday" stamp template may be displayed in area 1412.

[0145] Additionally, information related to the current time (or the duration of the current call) may be displayed in the composite image. For example, but not limited to, the current time may be displayed in the upper left area (1416) of the composite image. The upper left area (1416) may also display the duration of the video / voice call currently taking place in the chat room (e.g., 18 minutes and 17 seconds).

[0146] The user (or host) can remove all stamps, user information, etc. displayed on the composite image by selecting button (1418).

[0147] A second operation (1420) illustrates an example in which a visual object (1424) containing user information is displayed on the composite image in response to a user input selecting one of a plurality of user information templates in area (1422). In one embodiment, the user (or host) can select at least one of the plurality of user information templates to display a visual object (1424) indicating user information (e.g., a dialogue name, a name, status information, etc.) on the composite image, and can change the position of the displayed visual object (1424), rotate the visual object (1424), or remove the visual object (1424). The user can change the user information displayed in the visual object (1424).

[0148] Multiple user information templates can be presented in a variety of styles (fonts, decorative effects, etc.) Additionally, users can modify the design of stamp templates or add new stamp templates.

[0149] The above-described method may be provided as a program stored on a computer-readable recording medium for execution by a computer. The medium may continuously store a computer-executable program or temporarily store it for execution or download. The medium may also be various recording or storage means in the form of a single piece of hardware or multiple pieces of hardware combined together. The medium is not limited to media directly connected to a computer system but may also be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Other examples of media include recording or storage media managed by app stores that distribute applications and other sites or servers that provide or distribute various software.

[0150] The methods, operations, or techniques of the present disclosure may be embodied by various means. For example, these techniques may be embodied in hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the present disclosure may be embodied in electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is embodied as hardware or software depends on the particular application and design requirements imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation should not be interpreted as causing a departure from the scope of the present disclosure.

[0151] In a hardware implementation, the processing units utilized to perform the techniques may be embodied within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or combinations thereof.

[0152] Accordingly, the various illustrative logic blocks, modules, and circuits described in connection with this disclosure may be embodied in or performed by a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be embodied in a combination of computing devices, e.g., a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

[0153] In a firmware and / or software implementation, the techniques may be embodied with instructions stored on a computer-readable medium such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processors to perform certain aspects of the functions described in this disclosure.

[0154] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more stand-alone computer systems, the present disclosure is not limited thereto and may be implemented in connection with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the subject matter in this disclosure may be implemented in multiple processing chips or devices, and storage may be affected across multiple devices as well. These devices may include PCs, network servers, and handheld devices.

[0155] Although the present disclosure has been described herein with respect to some embodiments, various modifications and changes that would be understood by those skilled in the art to which the present disclosure pertains can be made without departing from the scope of the present disclosure, and such modifications and changes should be considered to fall within the scope of the claims appended hereto.

Claims

1. 1. A method for capturing images during a video call, executed by at least one processor of a first user terminal, comprising: Initiating a video call between the first user terminal associated with a first user and a second user terminal associated with a second user included in a chat room of an instant messaging application; switching to a multi-party shooting mode during the video call; displaying a photographing notification on a display of the first user terminal for a first time period; and displaying on the display a composite image obtained by combining a first image including the first user and a second image including the second user.

2. receiving a user input from the first user selecting a multi-party photo capture button after switching to the multi-party photo capture mode and before displaying the photo capture notification; taking a first image including the first user using an image sensor of the first user terminal while the notification of taking a picture is displayed; receiving a second image including the second user from the second user terminal; The method of claim 1 , further comprising: combining the first image and the second image to generate the combined image.

3. receiving a user input for multi-party photography from the first user after switching to the multi-party photography mode and before displaying the photography notification; taking a first image including the first user using an image sensor of the first user terminal while the notification of taking a picture is displayed; capturing a screenshot of a second image displayed on the display, the second image including the second user; The method of claim 1 , further comprising: combining the first image and the second image to generate the combined image.

4. The method of claim 1 , further comprising: deactivating a multi-party photo-taking button in response to determining that the second user terminal has selected a camera-off button.

5. the step of: after switching to the multi-party shooting mode and before displaying the shooting notification, displaying a shooting preparation notification on the display for a second time period; The method of claim 1 , wherein the second time period is longer than the first time period.

6. The method of claim 5 , further comprising the step of deactivating a camera-off button and a multi-party shooting mode exit button while the shooting preparation notification is displayed.

7. The method further includes taking a first image including the first user using an image sensor of the first user terminal for a third time while the notification of taking a picture is displayed; The method of claim 1 , wherein the first time period is longer than the third time period by at least a predetermined time period.

8. 8. The image capturing method during a video call according to claim 7, wherein the end of the third time period for capturing the first image is the same as the end of the first time period for displaying the image capturing notification, or is before the end of the first time period for displaying the image capturing notification.

9. further comprising measuring a communication delay of data between the first user terminal and the second user terminal; a timing for displaying a shooting preparation notification on the display is determined based on the measured communication delay; The image capturing method during a video call according to claim 5 , wherein the first image and the second image are captured at the same timing in the first user terminal and the second user terminal based on the measured communication delay.

10. displaying a plurality of photo themes on the display after switching to the multi-party photography mode and before displaying the photography notification; receiving a user input from the first user selecting one of the plurality of photo themes; displaying a first sequence of images captured by an image sensor of the first user terminal in a first area of ​​the selected photographic theme on the display; displaying a second image sequence received from the second user terminal in a second area of ​​the selected photographic theme on the display; The method of claim 1 , further comprising: receiving a user input from the first user selecting a multi-party photo taking button.

11. The method for taking pictures during a video call according to claim 10, wherein when a plurality of photo themes are displayed on the display, a visual object is displayed near the photo theme for which the second user has given feedback.

12. a first pose guide is displayed as an overlay on the first sequence of images in the first region; The method of claim 10 , wherein a second pose guide is displayed as an overlay on the second sequence of images in the second region.

13. the composite image includes multiple layers; 2. The image capturing method during a video call according to claim 1, wherein an image captured by a user terminal that switches to the multi-party capturing mode first, of the first user terminal and the second user terminal, is placed on a higher layer.

14. After switching to the multi-party shooting mode, displaying a first image sequence captured by an image sensor of the first user terminal in a first area of ​​the display in response to determining that the number of users participating in the multi-party capture mode is equal to or less than a predetermined number; The method of claim 1 , further comprising: displaying a second sequence of images received from the second user terminal in a second area of ​​the display.

15. and further including, in response to determining that the number of users participating in the multi-party photography mode exceeds a predetermined number after switching to the multi-party photography mode, displaying a second image sequence received from the second user terminal in a second area of ​​the display; The method of claim 1 , wherein the first image sequence captured by the image sensor of the first user terminal is not displayed on the display.

16. 16. The method for capturing images during a video call according to claim 15, further comprising: displaying a first image sequence captured by an image sensor of the first user terminal in a first area of ​​the display in response to a determination that a user participating in the multi-party capturing mode has left the multi-party capturing mode and the number of users participating in the multi-party capturing mode is less than a predetermined number.

17. The method of claim 1 , further comprising: transmitting the composite image to the second user terminal through the chat room.

18. 18. The method of claim 17, wherein the composite image is transmitted to the second user terminal through the chat room in response to receiving a user input from the first user selecting an exit multi-party capture mode button.

19. A non-transitory computer-readable recording medium having recorded thereon instructions for causing a computer to execute the image capturing method according to any one of claims 1 to 18.

20. a first user terminal, a communication module; The display and Memory and at least one processor coupled to the memory and configured to execute at least one computer-readable program contained in the memory; The at least one program Initiating a video call between a first user terminal associated with a first user and a second user terminal associated with a second user included in a chat room of an instant messaging application; During the video call, switch to multi-party shooting mode, displaying a capture notification on the display for a first time period; The first user terminal includes instructions for displaying on the display a composite image obtained by combining a first image including the first user and a second image including the second user.

Images