Method and device for providing 360-degree panoramic background for use during video call

Abstract

To provide a method and device for providing a 360-degree panoramic background used while making a video call.SOLUTION: A background provision method performed by an electronic device, includes the steps of separating an object video and a background video from video data input to a video call screen of a terminal during the video call, replacing the background image separated from the image data with a panoramic image, displaying new video data generated by synthesizing the background video replaced with the panoramic video image with the object video on the video call screen, and sharing the panoramic video image such that the background video of the new video data displayed on the video call screen is displayed on the video call screen of the other terminal.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The following description relates to a technique for providing a usable background while making a video call between electronic devices. [Background technology]

[0002] In general video calls, information about an individual's private life, such as the user's calling location, position, surrounding environment, etc., is transmitted to the other user, which can lead to an element of invasion of privacy. As a result, research is being conducted to change the background image to protect the privacy of users who use video calls. For example, a video deletion method has been disclosed in which the video portion is removed from the video call and only the audio portion is transmitted, and a method of changing the background image to a specific image using an image conversion server on the Internet or individually configured.

[0003] When a camera application or a video call application provides background images that allow a variety of backgrounds to be selected, the background can be changed by selecting a preferred background image. However, conventional background images have problems in that they only have monotonous, single-color backgrounds, or that the background is not completely removed during the process of cutting out a person (i.e., the process of separating the background from the subject (e.g., a person)), leaving some of it behind. To solve these problems, there is a demand for technology that can completely hide the background of a user who does not want his or her location to be seen, while providing a sense of reality and topicality beyond the bounds of monotonous pattern background images. Summary of the Invention [Problem to be solved by the invention]

[0004] To provide a method and device for replacing a background image of video data inputted to a video call screen during a video call with a 360-degree panoramic video image, and displaying video data generated by synthesizing the background image replaced with the panoramic video image with an object image on the video call screen. [Means for solving the problem]

[0005] A program recorded on a computer-readable recording medium for executing a background providing method executed by an electronic device, the background providing method may include a step of separating an object image and a background image from video data input to a video call screen of a terminal during a video call, a step of replacing the background image separated from the video data with a panoramic video image, a step of displaying new video data on the video call screen by combining the background image replaced with the panoramic video image with the object image, and a step of sharing the panoramic video image so that the background image of the new video data displayed on the video call screen is displayed on the video call screen of the other terminal.

[0006] The background providing method performed by the electronic device may include a step of separating an object image and a background image from video data input to a video call screen of a terminal during a video call, a step of replacing the background image separated from the video data with a panoramic video image, a step of displaying new video data on the video call screen by combining the background image replaced with the panoramic video image with the object image, and a step of sharing the panoramic video image so that the background image of the new video data displayed on the video call screen is displayed on the video call screen of the other terminal.

[0007] The electronic device may include a video separation unit that separates an object video and a background video from video data input to a video call screen of a terminal during a video call, a background video replacement unit that replaces the background video separated from the video data with a panoramic video image, a video data display unit that displays new video data on the video call screen, generated by combining the background video replaced with the panoramic video image with the object video, and a background video sharing unit that shares the panoramic video image so that the background video of the new video data displayed on the video call screen is displayed on the video call screen of the other terminal. Effect of the Invention

[0008] A 360-degree panoramic video image can be displayed as the background video of the video call screen, creating the feeling that the user and the other user are together in the same space.

[0009] Furthermore, since the user's surrounding information displayed on the video call screen during a video call can be hidden, the user's privacy can be protected.

[0010] Additionally, users can meet new people without revealing their space. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 illustrates an example of a network environment in one embodiment. [Diagram 2] FIG. 2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an embodiment. [Diagram 3] FIG. 2 is a block diagram illustrating components that may be included in a processor of an electronic device in one embodiment. [Figure 4] 1 is a flowchart illustrating a background providing method that can be executed by an electronic device in one embodiment. [Diagram 5] 4 is an exemplary diagram illustrating separation of video data input during a video call in an electronic device into an object video and a background video according to an embodiment; FIG. [Figure 6] 10 is a flowchart illustrating a method for displaying a background image of a video call screen whose position moves based on motion information in an electronic device according to an embodiment. [Figure 7] 11 is an exemplary diagram illustrating a background image replaced with a panoramic video image displayed on a video call screen of an electronic device according to an embodiment; [Figure 8] 11 is an exemplary diagram illustrating a background image replaced with a panoramic video image displayed on a video call screen of an electronic device according to an embodiment; [Figure 9] 10 is a flowchart illustrating a method for controlling a background image of a counterpart terminal based on time information set in an electronic device in an embodiment. [Figure 10] 11 is an exemplary diagram illustrating how a background image of a remote terminal is maintained based on time information set in an electronic device according to an embodiment. FIG. [Figure 11] 10 is a flowchart illustrating a method for setting position information at which an object video is displayed on an electronic device according to an embodiment. [Figure 12] 11 is an exemplary diagram illustrating how an object image is positioned based on position information set in an electronic device according to an embodiment. FIG. [Figure 13] 11 is an exemplary diagram illustrating a background image of an electronic device being displayed on a video call screen of a remote terminal according to an embodiment; FIG. [Figure 14] 10 is a flowchart illustrating a method for generating an event in a background video based on interaction information in an electronic device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <Summary of the Invention> A program recorded on a computer-readable recording medium for executing a background providing method executed by an electronic device, the background providing method may include a step of separating an object image and a background image from video data input to a video call screen of a terminal during a video call, a step of replacing the background image separated from the video data with a panoramic video image, a step of displaying new video data on the video call screen by combining the background image replaced with the panoramic video image with the object image, and a step of sharing the panoramic video image so that the background image of the new video data displayed on the video call screen is displayed on the video call screen of the other terminal.

[0013] The displaying step may include displaying, on the video call screen, a panoramic video image whose position moves in response to input user movement information of new video data displayed on the video call screen.

[0014] The sharing step may include transmitting a panoramic video image included in a background video of the new video data displayed on the video call screen to a counterpart terminal during the video call.

[0015] The sharing step may include a step of controlling video data including the background image replaced with the panoramic video image to be displayed on a video call screen of the counterpart terminal based on time information set to maintain the background image replaced with the panoramic video image.

[0016] The sharing step may include a step of setting location information for displaying an object image inputted into a video call screen captured by a camera of the counterpart terminal on the panoramic image, and transmitting the panoramic image including the set location information to the counterpart terminal.

[0017] The sharing step may include a step of transmitting a panoramic video image including the replaced time information to the counterpart terminal so that a background image of the video data input to the video call screen of the terminal is displayed on the video call screen of the counterpart terminal based on time difference information with the counterpart terminal based on time information in which the background image of the video data input to the video call screen of the terminal is replaced with the panoramic video image.

[0018] The sharing step may include generating an event in a background object present in the panoramic video image based on the interaction information.

[0019] The sharing step may include a step of placing an item in a portion of the panoramic image, and generating an event by inputting interaction information into the item placed in the portion of the panoramic image that moves according to user movement information.

[0020] The displaying step may include playing background music set for the panoramic video image.

[0021] The replacing step may include replacing a panoramic image stored in the electronic device as the background image.

[0022] The replacing may include generating a panoramic image using at least one image acquired by the electronic device, and replacing the generated panoramic image as the background image.

[0023] The substituting step may include providing a background image list for the background image, and substituting a panoramic image selected from the provided background image list as the background image.

[0024] The background providing method performed by the electronic device may include a step of separating an object image and a background image from video data input to a video call screen of a terminal during a video call, a step of replacing the background image separated from the video data with a panoramic video image, a step of displaying new video data on the video call screen by combining the background image replaced with the panoramic video image with the object image, and a step of sharing the panoramic video image so that the background image of the new video data displayed on the video call screen is displayed on the video call screen of the other terminal.

[0025] The displaying step may include displaying, on the video call screen, a panoramic video image whose position moves in response to input user movement information of new video data displayed on the video call screen.

[0026] The sharing step may include a step of controlling video data including the background image replaced by the panoramic video image to be displayed on a video call screen of the other terminal based on time information set to maintain the background image replaced by the panoramic video image.

[0027] The sharing step may include a step of setting location information for displaying an object image inputted into a video call screen captured by a camera of the counterpart terminal on the panoramic image, and transmitting the panoramic image including the set location information to the counterpart terminal.

[0028] The electronic device may include a video separation unit that separates an object video and a background video from video data input to a video call screen of a terminal during a video call, a background video replacement unit that replaces the background video separated from the video data with a panoramic video image, a video data display unit that displays new video data on the video call screen, generated by combining the background video replaced with the panoramic video image with the object video, and a background video sharing unit that shares the panoramic video image so that the background video of the new video data displayed on the video call screen is displayed on the video call screen of the other terminal.

[0029] In the other terminal, the background screen of the video data displayed on the video call screen of the other terminal may be converted into a panoramic video image based on time difference information with the other terminal based on time information at which the background image of the video data input to the video call screen is replaced with a panoramic video image.

[0030] In the remote terminal, it may be set whether or not to receive the panoramic video image.

[0031] In the remote terminal, the received panoramic video image may be substituted as a background video of the video data input to the camera of the remote terminal, and the background video substituted as the panoramic video image may be displayed on the video call screen of the remote terminal.

[0032] <Details of the Invention> Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings.

[0033] Fig. 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in Fig. 1 shows an example including a plurality of electronic devices 110, 120, 130, 140, a plurality of servers 150, 160, and a network 170. Fig. 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited to those shown in Fig. 1.

[0034] The electronic devices 110, 120, 130, and 140 may be fixed terminals or mobile terminals realized by computer systems. Examples of the electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, personal computers (PCs), notebook PCs, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), tablets, game consoles, wearable devices, internet of things (IoT) devices, virtual reality (VR) devices, and augmented reality (AR) devices. As an example, FIG. 1 shows a smartphone as an example of the electronic device 110, but in the embodiment of the present invention, the electronic device 110 may mean one of various physical computer systems that can communicate with other electronic devices 120, 130, and 140 and / or servers 150 and 160 via a network 170 using a substantially wireless or wired communication method.

[0035] The communication method is not limited, and may include not only a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcast network, a satellite network, etc.) that the network 170 can include, but also short-range wireless communication between devices. For example, the network 170 may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, etc. Furthermore, the network 170 may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc.

[0036] Each of the servers 150 and 160 may be realized by one or more computer devices that communicate with the multiple electronic devices 110, 120, 130, and 140 via the network 170 to provide instructions, codes, files, content, services, and the like. For example, the server 150 may be a system that provides a first service to the multiple electronic devices 110, 120, 130, and 140 connected via the network 170, and the server 160 may be a system that provides a second service to the multiple electronic devices 110, 120, 130, and 140 connected via the network 170. As a more specific example, the server 150 may provide a service (e.g., a messaging service, for example) targeted by an application, which is a computer program installed and executed in the multiple electronic devices 110, 120, 130, and 140, as a first service to the multiple electronic devices 110, 120, 130, and 140. As another example, the server 160 may provide, as a second service, a service of distributing files for installing and executing the above-mentioned application to the multiple electronic devices 110, 120, 130, and 140.

[0037] Fig. 2 is a block diagram for explaining the internal configuration of an electronic device and a server in an embodiment of the present invention. In Fig. 2, the internal configuration of electronic device 110 and the internal configuration of server 150 are explained as examples of electronic devices. In addition, other electronic devices 120, 130, 140 and server 160 may have the same or similar internal configuration as electronic device 110 or server 150 described above.

[0038] The electronic device 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input / output interfaces 214 and 224. The memories 211 and 221 may be non-transitory computer-readable recording media and may include persistent mass storage devices such as random access memory (RAM), read only memory (ROM), disk drives, solid state drives (SSD), flash memories, and the like. Here, persistent mass storage devices such as ROM, SSD, flash memories, and disk drives may be included in the electronic device 110 and the server 150 as persistent storage devices separate from the memories 211 and 221. In addition, the memories 211 and 221 may store an operating system and at least one program code (for example, a code for a browser installed and executed in the electronic device 110, an application installed in the electronic device 110 to provide a specific service, and the like). Such software components may be loaded from a computer-readable recording medium separate from the memories 211 and 221. Such another computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. In another embodiment, the software components may be loaded into the memory 211, 221 through a communication module 213, 223 that is not a computer-readable recording medium. For example, at least one program may be loaded into the memory 211, 221 based on a computer program (for example, the above-mentioned application) that is installed by a file provided via the network 170 by a developer or a file distribution system that distributes an installation file of the application (for example, the above-mentioned server 160).

[0039] The processors 212, 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processors 212, 222 by the memory 211, 221 or the communication modules 213, 223. For example, the processors 212, 222 may be configured to execute instructions received according to program code stored in a storage device such as the memory 211, 221.

[0040] The communication modules 213 and 223 may provide a function for the electronic device 110 and the server 150 to communicate with each other via the network 170, or may provide a function for the electronic device 110 and / or the server 150 to communicate with other electronic devices (for example, the electronic device 120) or other servers (for example, the server 160). For example, a request generated by the processor 212 of the electronic device 110 according to a program code recorded in a recording device such as the memory 211 may be transmitted to the server 150 via the network 170 under the control of the communication module 213. Conversely, a control signal, an instruction, a content, a file, etc. provided under the control of the processor 222 of the server 150 may be received by the electronic device 110 through the communication module 213 of the electronic device 110 via the communication module 223 and the network 170. For example, control signals, instructions, content, files, etc. from the server 150 received through the communication module 213 may be transmitted to the processor 212 or memory 211, and the content, files, etc. may be recorded on a recording medium (the above-mentioned permanent recording device) that the electronic device 110 may further include.

[0041] The input / output interface 214 may be a means for interfacing with the input / output device 215. For example, the input device may include a device such as a keyboard, a mouse, a microphone, or a camera, and the output device may include a device such as a display, a speaker, or a haptic feedback device. As another example, the input / output interface 214 may be a means for interfacing with a device in which functions for input and output are integrated into one, such as a touch screen. The input / output device 215 may be configured as one device together with the electronic device 110. Also, the input / output interface 224 of the server 150 may be a means for interfacing with a device for input or output (not shown) that may be connected to the server 150 or included in the server 150. As a more specific example, when the processor 212 of the electronic device 110 processes the instructions of the computer program loaded in the memory 211, a service screen or content configured using data provided by the server 150 or the electronic device 120 may be displayed on a display through the input / output interface 214.

[0042] Also, in other embodiments, the electronic device 110 and the server 150 may include more components than those in FIG. 2. However, it is not necessary to clearly show most of the conventional technical components in the figure. For example, the electronic device 110 may be realized to include at least some of the above-mentioned input / output devices 215, and may further include other components such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, a database, etc. As a more specific example, if the electronic device 110 is a smartphone, various components such as an acceleration sensor, a gyro sensor, a camera module, various physical buttons, a button using a touch panel, an input / output port, a vibrator for vibration, etc., which are generally included in a smartphone, may be realized to be further included in the electronic device 110.

[0043] Hereinafter, an embodiment will be described in which video data input during a video call is separated into an object image and a background image, and the separated background image is displayed on a video call screen in place of an optical viewing angle image. Also, an embodiment will be described in which an optical viewing angle image included in a background image of video data displayed on a video call screen is shared with a counterpart terminal during a video call. Here, the optical viewing angle may mean the maximum side angle at which a screen can be normally viewed on a video call screen, and an optical viewing angle image can be viewed on a video call screen according to an angle range including top, bottom, left, and right and normal brightness. An example of an optical viewing angle image may be a panoramic image. A panoramic image image is an image or an image provided in a panoramic form, and may be an image or an image that can be viewed from multiple directions (e.g., 360 degrees).

[0044] A background providing system implemented by a computer may be configured in the electronic device 110 according to the embodiment. For example, the background providing system may be implemented in the form of an independently operating program or may be configured in the form of an in-app of a specific application so as to be operable on the specific application, and in some cases, a messaging service may be provided in conjunction with the server 150.

[0045] Based on instructions provided by an application installed on the electronic device, the background providing system implemented in the electronic device 110 may perform a method including at least one feature associated with an embodiment.

[0046] Specifically, by sharing a background image substituted with a panoramic video image during a video call, the same background image may be displayed on the video call screen of each electronic device. Hereinafter, the electronic device 110 will be described by dividing it into a terminal and a counterpart terminal. In an embodiment, a video call environment may be configured by executing a video call through a call connection between a terminal (e.g., a first terminal, a calling terminal) and a counterpart terminal (e.g., a second terminal, a receiving terminal), and a background image substituted with a panoramic video image may be shared by a background providing service provided in the configured video call environment. In this case, the background providing service may be provided as a main function or an additional function of various services such as a video call service, a camera service, a chat service, etc., specific to the electronic device.

[0047] A background image for replacing the background image with a panoramic video image may be applied by selecting a user interface (e.g., tapping on a background image) for replacing the background image in the electronic device 110. For example, a background image list for replacing the background image may be provided through a background providing service operated in the electronic device, and a background image may be selected from the background image list and applied or unapplied video data may be confirmed, thereby replacing the background image with a panoramic video image. Hereinafter, a technology for providing a background image that can be used during a video call with a counterpart terminal will be described.

[0048] FIG. 3 is a block diagram illustrating components that may be included in a processor of an electronic device in one embodiment, and FIG. 4 is a flowchart illustrating a background providing method that may be executed by an electronic device in one embodiment.

[0049] The processor 212 of the electronic device may include a video separation unit 310, a background video replacement unit 320, a video data display unit 330, and a background video sharing unit 340. The components of the processor 212 may represent different functions executed by the processor 212 according to control instructions provided by program code recorded in the electronic device. The processor 212 and the components of the processor 212 may control the electronic device to execute steps 410 to 440 included in the background providing method of FIG. 4. In this case, the processor 212 and the components of the processor 212 may be realized to execute instructions according to the code of an operating system and the code of at least one program included in the memory.

[0050] The processor 212 may load a program code recorded in a program file for the background providing method into memory. For example, when the program is executed in the electronic device, the processor may control the electronic device to load the program code from the program file into memory under the control of an operating system. In this case, the processor 212 and the video separation unit 310, the background video replacement unit 320, the video data display unit 330, and the background video sharing unit 340 included in the processor 212 may each be a different functional representation of the processor 212 for executing instructions of corresponding parts of the program code loaded into memory to perform the following steps 410 to 440.

[0051] In step 410, the image separator 310 may separate an object image and a background image from image data input to a video call screen of a terminal during a call. FIG. 5 is an exemplary diagram for explaining separation of an object image and a background image from image data. A front camera mode may be operated by executing a camera in the electronic device 110. Image data input to the camera of the electronic device 110 may include an object image 510 including user information (e.g., a user's face) and a background image 520 including user surrounding information (e.g., buildings, roads, people unrelated to the user, etc.). Here, the camera mode operated during a video call of the electronic device 110 is described as the front camera mode, but is not limited thereto, and a back camera mode may be applied in the same manner, and the front camera mode or the back camera mode may be operated according to a user's selection. For example, the image separator 310 may separate the object image and the background image by outlining the boundary of an object in the image data through edge recognition, grouping pointers within the boundary, and grouping points having similar characteristics in a manner that keeps growing in the vicinity. Various other methods may be used to separate the object image and the background image from the image data.

[0052] In step 420, the background image replacement unit 320 may replace the background image separated from the image data with a panoramic image image. For example, the background image replacement unit 320 may replace the background image excluding the object image with the panoramic image image. As an example, the background image replacement unit 320 may replace a panoramic image image stored in the electronic device as the background image. If a panoramic image image is pre-stored in the electronic device, the background image replacement unit 320 may replace a panoramic image image selected for replacement with the background image among at least one panoramic image image stored in the electronic device as the background image. As another example, the background image replacement unit 320 may generate a panoramic image image using at least one image acquired by the electronic device, and replace the generated panoramic image image as the background image. In this case, the background image replacement unit 320 may acquire an image or image captured by the electronic device to generate the panoramic image image. Alternatively, the background image replacement unit 320 may acquire an image or image by the electronic device downloading an image or image uploaded by another electronic device. For example, the background image substitution unit 320 may provide a user interface (e.g., a tool) capable of generating an image or image acquired by the electronic device as a panoramic image, and a panoramic image may be generated by combining images or images according to a user's selection in the provided user interface. The background image substitution unit 320 may substitute the generated panoramic image as a background image. As another example, the background image substitution unit 320 may provide a list for background images, and may substitute a panoramic image selected from the provided background image list as a background image. In this case, the image substitution unit 320 may display a background image or image data to which the panoramic image selected by the user is applied. The user may determine whether to apply the panoramic image to the background image by checking the panoramic image displayed on the video call screen of the electronic device. The image substitution unit 320 may substitute the panoramic image as the background image by determining that the panoramic image is to be applied as the background image.

[0053] In step 430, the image data display unit 330 may display new image data generated by combining the background image replaced with the panoramic image image with the object image on the video call screen. FIGS. 7 and 8 are exemplary diagrams for explaining displaying the background image replaced with the panoramic image image on the video call screen. The image data display unit 330 may display new image data generated by combining the background image replaced with the panoramic image image 520 with the object image on the video call screen of the electronic device 110. New image data in which the background image is replaced with the panoramic image image 520 may be displayed while the object image photographed by the electronic device 110 is maintained on the electronic device 110. At this time, the object image photographed in real time by the camera operated in the electronic device 110 may be displayed in the image data. The image data display unit 330 may display new image data generated by combining the background image replaced with the panoramic image image with the object image on the video call screen and may play music at the same time. For example, the video data display unit 330 may play background music registered with the panoramic video image when displaying new video data on the video call screen. At this time, music to be played together with the panoramic video image may be registered in advance. Also, a music list to be played together with the panoramic video image may be provided, and music selected from the provided music list may be registered. Alternatively, the user may search for music to be played together with the panoramic video image, and the music may be registered. At this time, the volume of the music may be adjusted.

[0054] Also, the video data display unit 330 may set a partial area of ​​the panoramic video image to be initially displayed on the video call screen as a reference area. The video data display unit 330 may display the reference area of ​​the panoramic video image set to be initially displayed on the video call screen as a background video on the video call screen of the electronic device 110. Such a reference area may be changed according to a user's setting. Based on the reference area of ​​the panoramic video image initially displayed on the video call screen, the background video replaced with the panoramic video image may move according to the user's movement information 810, 811. For example, the video data display unit 330 may display a background video corresponding to at least one or more pieces of movement information 810, 811 including directional information of up, down, left, and right input by the user. Also, the background video replaced with the panoramic video image of the new video data displayed on the video call screen of the electronic device 110 may include a background object (e.g., a landmark 820). Here, the landmark 820 may mean an object present in the panoramic video image. An event may be set to occur by arranging an item in such landmark 820 or in a space within a panoramic video image. For example, assuming a panoramic video image including the Eiffel Tower in Paris, an item may be arranged in advance around the Eiffel Tower in Paris or the Eiffel Tower in the panoramic video image. In new video data displayed on the electronic device 110, a background image that moves according to the user's movement shows the Eiffel Tower or the Eiffel Tower's surroundings, thereby acquiring an item present around the Eiffel Tower or the Eiffel Tower. Alternatively, an event may be generated based on interaction information input to the Eiffel Tower or the Eiffel Tower's surroundings. For example, different types of events may be generated based on interaction information from the user input to the Eiffel Tower.

[0055] 6 is a flow chart for explaining a method for displaying a background image of a video call screen whose position moves based on user movement information. In step 610, the video data display unit 330 may receive that user movement information is input to new video data displayed on the video call screen. For example, directional information including up, down, left, and right may be input as the user movement information. In step 620, the video data display unit 330 may display a panoramic video image whose position moves according to the user movement information on the video call screen. For example, the video data display unit 330 may display a 360-degree panoramic video image as a background video by moving 360 degrees left and right from the user.

[0056] In step 440, the background image sharing unit 340 may share a panoramic image so that a background image of the new video data displayed on the video call screen is displayed on the video call screen of the other terminal. At this time, the background image sharing unit 340 may operate a process for sharing a panoramic image according to a request for sharing a panoramic image. The background image sharing unit 340 may transmit a panoramic image included in a background image of the new video data displayed on the video call screen to a other terminal during the video call. For example, when there are multiple other terminals during the video call, the background image sharing unit 340 may transmit a panoramic image to each of the multiple other terminals. At this time, whether or not to receive the panoramic image included in the background image may be set to on / off in the other terminal. When the other terminal is set to on, the panoramic image may be transmitted to the other terminal, and the transmitted panoramic image may be replaced with the background image to display the video data. On the other hand, when the other terminal is set to off, the panoramic image is not transmitted to the other terminal, and the panoramic image is not replaced as the background image.

[0057] FIG. 9 is a flowchart for explaining a method of controlling a background image of a remote terminal based on set time information. In step 910, the background image sharing unit 340 may set time information (e.g., 5 seconds) for maintaining the background image replaced with a panoramic image. The background image sharing unit 340 may provide a user interface for setting time information for maintaining the background image, and may generate a lock function based on the time information input to the provided user interface. For example, the background image sharing unit 340 may set a lock function in the remote terminal to prevent the background image from being changed or applied in the remote terminal within a preset time. During the time when the lock function is set, the background image cannot be changed or deleted in the remote terminal, and the video call cannot be stopped or ended. This allows the terminal to invite the remote terminal to the same space.

[0058] In step 920, the background image sharing unit 340 may control the display of image data including a background image substituted as a panoramic image on the video call screen of the counterpart terminal based on the set time information. The background image sharing unit 340 may transmit a panoramic image whose position moves according to the user's movement information to the counterpart terminal. As a result, the position of the panoramic image displayed on the video call screen of the counterpart terminal moves. FIG. 10 is an exemplary diagram for explaining how the background image of the counterpart terminal is maintained based on the time information set by the terminal. It is assumed that the terminal 110a and the counterpart terminal 110b are in a video call. In this case, the panoramic image, which is a background image displayed on the video call screen 1010 of the terminal 110a, may be transmitted to the counterpart terminal 110b for the time set by the terminal 110a. Even if the counterpart terminal 110b uses another background image, the background image substituted with the panoramic image of the terminal 110a may be transmitted. In other words, the background image of the video data displayed on the video call screen 1020 of the destination terminal 110b is converted to the panoramic video image transmitted from the terminal 110a. This makes it possible to display the same background image on the video call screen 1010 of the terminal 110a and the video call screen 1020 of the destination terminal 110b.

[0059] FIG. 11 is a flowchart for explaining a method of setting position information for displaying an object image. In step 1110, the background image sharing unit 340 may set position information for displaying an object image inputted to a video call screen captured by a camera of a counterpart terminal in a panoramic image image. For example, the background image sharing unit 340 may receive a selection of a specific point of a panoramic image image from a user. At this time, a specific point for displaying an object existing in each of the object images of the terminal and the counterpart terminal may be selected. For example, a point for displaying a user's face may be selected. In step 1120, the background image sharing unit 340 may transmit a panoramic image image including the position information set by the user to the counterpart terminal. The counterpart terminal may position an object existing in the object image based on the position information set in the panoramic image image transmitted from the terminal. At this time, the position information for displaying the object image of the counterpart terminal may be changed by the terminal or the counterpart terminal. The counterpart terminal may position an object image inputted to a video call screen of the counterpart terminal in the panoramic image image based on the position information set from the terminal. 12 is a diagram showing that electronic device 110a shares a background image with counterpart terminal 110b, so that objects input to terminal 110a and counterpart terminal 110b's video data exist in the same space. For example, users may be displayed as if terminal 110a and counterpart terminal 110b are sitting opposite each other at a specific table or a specific spot in a park.

[0060] FIG. 14 is a flowchart for explaining a method of generating an event in a background image based on interaction information. In step 1410, the image information sharing unit 340 may generate an event in a background object present in a panoramic image based on the interaction information. The image information sharing unit 340 may receive input of interaction information from a user. Here, the interaction information may mean that a landmark or a preset event occurs in a panoramic image according to input information from a user. Furthermore, it may mean interacting with a counterpart terminal according to an event occurring in a panoramic image. The image information sharing unit 340 may generate an event by inputting interaction information into an item arranged in a partial area of ​​a panoramic image that moves according to the movement of the terminal. For example, the image information sharing unit 340 may transmit a panoramic image including an event generated according to the interaction information to the counterpart terminal. Alternatively, the image information sharing unit 340 may transmit only event information generated according to the interaction information to the counterpart terminal. Event information may be applied to the background image of the counterpart terminal in the same manner as the terminal. For example, if a fountain is present in the background video, the fountain may be made higher or lower by inputting interaction information (e.g., zoom-in motion, zoom-out motion, etc.) from the user. As another example, a treasure chest may be hidden in the background video. When the user of the other terminal discovers the treasure chest, the treasure chest may be opened by inputting interaction information into the treasure chest. For example, opening the treasure chest may provide an item to the other terminal.

[0061] FIG. 13 is an exemplary diagram illustrating a background image of an electronic device being displayed on a video call screen of a partner terminal in an embodiment.

[0062] When the terminal and the other terminal 110b are in a video call, the terminals may be in the same area or in different areas. For example, when the terminal and the other terminal 110b are in a video call in different countries, the space in which the terminal exists may be daytime, but the space in which the other terminal 110b exists may be nighttime. In this way, a time difference may occur between the terminal and the other terminal 110b.

[0063] The electronic device may also record time information when the panoramic video image is replaced with the background video. The background video of the electronic device may be transmitted to the counterpart terminal 110b, and the counterpart terminal 110b may receive the background video transmitted from the electronic device. The panoramic video image of the video data displayed on the video call screen of the counterpart terminal 110b may be converted according to time difference information with the terminal based on the time information when the background video of the video data displayed on the video call screen of the counterpart terminal 110b is replaced with the panoramic video image.

[0064] The above-mentioned devices may be realized by hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments may be realized using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the OS. The processing device may also respond to the execution of the software to access, record, manipulate, process, and generate data. For convenience of understanding, a single processing device may be described as being used, but those skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0065] The software may include computer programs, codes, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or instruct the processing device, either independently or collectively. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium, or device to be interpreted based on the processing device or to provide instructions or data to the processing device. The software may be distributed and stored or executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0066] The method according to the embodiment may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions recorded on the medium may be specially designed and constructed for the embodiment, or may be available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, flash memories, and the like. Examples of program instructions include high-level language codes executed by a computer using an interpreter, etc., as well as machine language codes such as those generated by a compiler.

[0067] Although the embodiments have been described based on the limited embodiments and drawings as above, those skilled in the art will be able to make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from that described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a manner different from that described, or may be counterbalanced or replaced by other components or equivalents, and still achieve suitable results.

[0068] Therefore, different embodiments are within the scope of the appended claims provided they are equivalent to the claims.

[0069] The present invention further has the following objects.

[0070] To provide a method and device for sharing a panoramic video image so that a background video displayed on a video call screen is also displayed on the video call screen of a partner terminal. [Explanation of symbols]

[0071] 212: Processor 310: Video separation section 320: Background video alternative section 330: Video data display unit 340: Background video sharing section

Claims

1. A computer program for causing an electronic device to execute a background providing method, The method for providing a background includes: separating an object image and a background image from image data inputted to a video call screen of the terminal during the video call; replacing the separated background image with a panoramic image; and displaying, on the video call screen, a panoramic image in which new image data generated by synthesizing the background image replaced with the panoramic image with the object image moves in position in response to user movement information. sharing the new video data displayed on the video call screen so that the new video data is displayed on the video call screen of the other terminal; Including, The sharing step includes: and controlling the video data including the background image replaced with the panoramic video image to be displayed on the video call screen of the other terminal based on time information set for maintaining the background image replaced with the panoramic video image. A computer program comprising:

2. A computer program for causing an electronic device to execute a background providing method, comprising: The method for providing a background includes: separating an object image and a background image from image data inputted to a video call screen of the terminal during the video call; replacing the separated background image with a panoramic image; displaying, on the video call screen, a panoramic video image in which new video data generated by synthesizing the background video replaced with the panoramic video image with the object video image moves in accordance with user movement information; sharing the new video data displayed on the video call screen so that the new video data is displayed on the video call screen of the other terminal; Including, The sharing step includes: setting position information for displaying an object image inputted on a video call screen photographed by a camera of the other terminal in the panoramic image, and transmitting the panoramic image including the set position information to the other terminal. A computer program comprising:

3. A computer program for causing an electronic device to execute a background providing method, comprising: The method for providing a background includes: separating an object image and a background image from image data inputted to a video call screen of the terminal during the video call; replacing the separated background image with a panoramic image; displaying, on the video call screen, a panoramic video image in which new video data generated by synthesizing the background video replaced with the panoramic video image with the object video image moves in accordance with user movement information; sharing the new video data displayed on the video call screen so that the new video data is displayed on the video call screen of the other terminal; Including, The sharing step includes: transmitting the panoramic video image including the replaced time information to the other terminal so that the background video of the video data input to the video call screen of the terminal is displayed on the video call screen of the other terminal based on time difference information between the other terminal and the other terminal based on the time information in which the background video of the video data input to the video call screen of the terminal is replaced with the panoramic video image; A computer program comprising:

4. The sharing step includes: transmitting a panoramic video image included in a background video of the new video data displayed on the video call screen to a counterpart terminal during the video call; A computer program according to any one of claims 1 to 3, comprising:

5. The sharing step includes: generating an event on a background object present in the panoramic image based on the interaction information; A computer program according to any one of claims 1 to 4, comprising:

6. The sharing step includes: and generating an event by arranging an item in a portion of the panoramic image and inputting interaction information into the item arranged in the portion of the panoramic image that moves according to user movement information. A computer program according to any one of claims 1 to 5, comprising:

7. The motion information includes at least one of motion information including up, down, left, and right direction information of the electronic device. A computer program according to any one of claims 1 to 6, characterized in that

8. The displaying step includes: playing background music set for the panoramic image. A computer program according to any one of claims 1 to 7, comprising:

9. The replacing step includes: replacing the panoramic image stored in the electronic device as a background image; A computer program according to any one of claims 1 to 8, comprising:

10. The replacing step includes: generating a panoramic image using at least one image acquired by the electronic device, and replacing the generated panoramic image with a background image; A computer program according to any one of claims 1 to 8, comprising:

11. The replacing step includes: providing a background image list for the background image, and substituting a panoramic image selected from the provided background image list as the background image; A computer program according to any one of claims 1 to 8, comprising:

12. A background providing method executed by an electronic device, comprising: separating an object image and a background image from image data inputted to a video call screen of the terminal during the video call; replacing the separated background image with a panoramic image; and displaying, on the video call screen, a panoramic image in which new image data generated by synthesizing the background image replaced with the panoramic image with the object image moves in position in response to user movement information. sharing the new video data displayed on the video call screen so that the new video data is displayed on the video call screen of the other terminal; Including, The sharing step includes: and controlling the video data including the background image replaced with the panoramic video image to be displayed on the video call screen of the other terminal based on time information set for maintaining the background image replaced with the panoramic video image. A method for providing background, including:

13. A background providing method executed by an electronic device, comprising: separating an object image and a background image from image data inputted to a video call screen of the terminal during the video call; replacing the separated background image with a panoramic image; and displaying, on the video call screen, a panoramic image in which new image data generated by synthesizing the background image replaced with the panoramic image with the object image moves in position in response to user movement information. sharing the new video data displayed on the video call screen so that the new video data is displayed on the video call screen of the other terminal; Including, The sharing step includes: setting position information for displaying an object image inputted on a video call screen photographed by a camera of the other terminal in the panoramic image, and transmitting the panoramic image including the set position information to the other terminal. A method for providing background, including:

14. An electronic device, a video separation unit that separates an object video and a background video from video data input to a video call screen of the terminal during a video call; a background image replacement unit for replacing the separated background image with a panoramic image; a video data display unit for displaying, on the video call screen, a panoramic video image in which new video data generated by synthesizing the background video replaced with the panoramic video image with the object video image moves in position in response to user movement information; a background video sharing section for sharing the new video data displayed on the video call screen so that the new video data is displayed on the video call screen of the other terminal; Including, In the remote terminal, An electronic device characterized in that a panoramic video image of the video data displayed on the video call screen of the other terminal is converted based on time difference information with the other terminal based on time information at which the background image of the video data input to the video call screen is replaced with a panoramic video image.

15. In the remote terminal, Whether or not to receive the panoramic image is set.

15. The electronic device according to claim 14.

16. In the remote terminal, The panoramic video image is substituted for a background video of video data inputted to a camera of the other terminal, and the background video substituted for the panoramic video image is displayed on a video call screen of the other terminal.

16. The electronic device according to claim 14 or 15.

Images