Reaction display method, computer device, and computer-readable recording medium

Abstract

Disclosed are a reaction display method, a computer device, and a computer-readable recording medium. During a network phone call using a social graph, various media content can be shared between call participants, and in the environment of the shared media content, reactions exchanged between the call participants can be displayed in real time during the network phone call.

Description

TECHNICAL FIELD

[0001] The following description relates to a technology for providing a call function based on a voice over internet protocol (VoIP). BACKGROUND

[0002] Recently, communication devices provide various services such as a wireless network service, a terrestrial / satellite broadcasting service, etc. in addition to a voice call service and a text service.

[0003] In particular, as video compression and restoration technologies are developed and devices equipped with a camera are popularized, a video call service that enables a call to be made while confirming the face of the other party is provided.

[0004] As an example of a technology for providing a video call service, Korean Patent Laid-Open Publication No. 10-0401262 (date of authorization 2003.09.29) discloses a technology for providing a video call service between mobile phone terminals in a mobile phone network in a wireless environment. SUMMARY

[0005] In a voice over internet protocol (VoIP) call process using a social graph, various media contents can be shared between call participants.

[0006] In a VoIP call process, a reaction exchanged between call participants can be displayed in real time.

[0007] The present invention provides a reaction display method, which is executed in a computer device including at least one processor configured to execute computer readable instructions contained in a memory, the reaction display method including: a display step of displaying, by the at least one processor, a VoIP call screen and a content sharing screen in a case where media contents are shared in a VoIP call process in which a user of the computer device participates; a reception step of receiving, by the at least one processor, position information of a reaction input by the user of the computer device in the VoIP call process; a transmission step of transmitting, by the at least one processor, information related to the reaction and the position information to at least one other user participating in the VoIP call as an object; and a display step of displaying, by the at least one processor, the reaction on the VoIP call screen or the content sharing screen based on the position information.

[0008] According to an embodiment, the reception step can include a recognition step of recognizing a position coordinate at which the reaction is input as the position information on the VoIP call screen or the content sharing screen.

[0009] According to still another embodiment, the receiving step can include an identifying step of identifying, in a case where a reaction is input to the content sharing screen, a playback position of the content being played back on the content sharing screen at a time point of the user input reaction as the position information.

[0010] According to another embodiment, the reaction display method can further include a designating step of designating, by the at least one processor, a reaction object in the participant image included in the net phone call screen and the content sharing screen based on the position information.

[0011] According to still another embodiment, in the designating step, a screen corresponding to a position coordinate at which the reaction is input can be designated as the reaction object in the net phone call screen and the content sharing screen.

[0012] According to still another embodiment, in the designating step, in a case where a playback position of the content being played back on the content sharing screen is received as the position information, the content sharing screen can be designated as the reaction object in the net phone call screen and the content sharing screen.

[0013] According to still another embodiment, in the transmitting step, a user corresponding to the reaction object can be designated among other users participating in the net phone call, and the designated user can be transmitted as an object of the information related to the reaction and the position information.

[0014] According to still another embodiment, in the transmitting step, metadata of a net phone call data packet including the information related to the reaction and the position information can be generated and transmitted, and the reaction can be displayed on a screen of a computer device of the other user based on the metadata in the computer device of the other user.

[0015] According to still another embodiment, in the displaying step, an object matching the reaction can be displayed at a position on a screen corresponding to the position information in the net phone call screen and the content sharing screen.

[0016] According to still another embodiment, the reaction display method can further include a transmitting step of transmitting, by the at least one processor, information related to the media content shared during the net phone call and information related to the reaction received during the net phone call to a chat room related to the net phone call.

[0017] According to still another embodiment, the step of transmitting information to a chat room related to the above-mentioned Internet phone call can include a management step of managing information related to the above-mentioned media content and information related to the above-mentioned reaction by message type included in the above-mentioned chat room.

[0018] The present application provides a non-transitory computer-readable recording medium storing a computer program for causing a computer device to execute the above-mentioned reaction display method.

[0019] The present application provides a computer device including at least one processor implemented to execute computer-readable instructions included in a memory, the at least one processor processing a display process of displaying an Internet phone call screen and a content sharing screen in a case where media content is shared in an Internet phone call process in which a user of the computer device participates, a reception process of receiving position information of a reaction input by the user of the computer device in the Internet phone call process, a transmission process of transmitting information related to the reaction and the position information to at least one other user participating in the Internet phone call as an object, and a display process of displaying the reaction on the Internet phone call screen or the content sharing screen based on the position information. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A diagram to show an example of a network environment of an embodiment of the present application.

[0021] Figure 2 A block diagram to show internal structures of an electronic device and a server for explaining an embodiment of the present application.

[0022] Figure 3 A diagram to show an example of structural elements that a processor of an electronic device of an embodiment of the present application can include.

[0023] Figure 4 A diagram to show an example of a method that an electronic device of an embodiment of the present application can execute.

[0024] Figures 5 to 8 A diagram to show an example of a process of sharing content in an Internet phone call process for explaining an embodiment of the present application.

[0025] Figure 9 A flowchart to show an example of a method of displaying a reaction in an Internet phone call process of an embodiment of the present application.

[0026] Figures 10 to 12 A diagram to show an example of a process of displaying a reaction in an Internet phone call process for explaining an embodiment of the present application. DETAILED DESCRIPTION

[0027] Hereinafter, an embodiment of the present application will be described in detail with reference to the accompanying drawings.

[0028] An embodiment of the present application relates to a technology for providing a network phone-based call function.

[0029] In an embodiment including the contents specifically disclosed in the present specification, various media contents can be shared between call participants during a network phone call using a social graph, and reactions exchanged between the call participants can be displayed in real time during the network phone call.

[0030] Figure 1 A diagram for illustrating an example of a network environment of an embodiment of the present application. Figure 1 The network environment illustrates an example including a plurality of electronic devices 110, 120, 130, 140 and a plurality of servers 150, 160, and a network 170. The above-described Figure 1 The number of electronic devices or the number of servers is not limited to the example shown. Figure 1

[0031] The plurality of electronic devices 110, 120, 130, 140 can be fixed-type terminals or mobile-type terminals implemented by computer devices. As an example of the plurality of electronic devices 110, 120, 130, 140, there are included a smart phone, a mobile phone, a navigation device, a computer, a notebook computer, a digital broadcast terminal, a Personal Digital Assistant (PDA), a Portable Multimedia Player (PMP), a tablet computer, a game console, a wearable device, an internet of things (IoT) device, a virtual reality (VR) device, an augmented reality (AR) device, etc. As an example, Figure 1 In the example, the shape of a smart phone is illustrated as an example of the electronic device 110, and in an embodiment of the present application, the electronic device 110 can be substantially one of various physical computer devices capable of communicating with other electronic devices 120, 130, 140 and / or servers 150, 160 through the network 170 using wireless or wired communication methods.

[0032] ​The communication method is not limited, and can include a communication method using a communication network (e.g., a mobile communication network, a wired Internet, a wireless Internet, a broadcast network, a satellite network, etc.) that the network 170 can include and near distance wireless communication between a plurality of devices. For example, the network 170 can include one or more of any of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, etc. Also, the network 170 can include any one or more of network topologies having a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree network, a hierarchical network, etc., but is not limited thereto.

[0033] Each of the servers 150, 160 can be implemented as one or more computer devices that provide instructions, codes, files, contents, services, etc. through the network 170 in communication with the plurality of electronic devices 110, 120, 130, 140. For example, the server 150 can be a system that provides a first service to the plurality of electronic devices 110, 120, 130, 140 connected through the network 170, and the server 160 can be a system that provides a second service to the plurality of electronic devices 110, 120, 130, 140 connected through the network 170. As a more specific example, the server 150 can provide a service required by an application as a first service to the plurality of electronic devices 110, 120, 130, 140 by being installed as a computer program driven by the application, such as a network phone call service, etc. As another example, the server 160 can provide a service of distributing a file for installing and driving the above-described application to the plurality of electronic devices 110, 120, 130, 140 as a second service.

[0034] Figure 2 A block diagram for explaining internal structures of an electronic device and a server in an embodiment of the present application is shown. In Figure 2 In the block diagram, the internal structures of the electronic device 110 and the server 150 are explained as examples of the electronic device. Also, the other electronic devices 120, 130, 140 or the server 160 can have the same or similar internal structures as the above-described electronic device 110 or the server 150.

[0035] The electronic device 110 and the server 150 can include memories 211, 221, processors 212, 222, communication modules 213, 223, and input / output interfaces 214, 224. The memories 211, 221, as non-transitory computer-readable recording media, can include permanent mass storage devices such as random access memories (RAMs), read only memories (ROMs), hard drives, solid state drives (SSDs), flash memories, etc. Among them, the permanent mass storage devices such as the read only memories, the solid state drives, the flash memories, the hard drives, etc. can also be included in the electronic device 110 or the server 150 as separate permanent storage devices distinguished from the memories 211, 221. Also, the memories 211, 221 can store operating systems and at least one program code (as an example, a code of a browser installed to drive the electronic device 110 or an application installed to the electronic device 110 for providing a specific service, etc.). The above software structural elements can be loaded from a computer-readable recording medium separate from the memories 211, 221. The above separate computer-readable recording medium can include floppy disk drives, magnetic disks, magnetic tapes, DVD / CD-ROM drives, memory cards, etc. In another embodiment, the software structural elements can be loaded to the memories 211, 221 through the communication modules 213, 223, not through the computer-readable recording medium. For example, at least one program can be loaded to the memories 211, 221 based on a computer program (as an example, the above application) installed through a file provided by a developer or a file distribution system (for example, the above server 160) distributing an installation file of the application through the network 170.

[0036] The processors 212, 222 can be configured to perform basic arithmetic, logic, and input / output calculations, thereby processing instructions of a computer program. The instructions can be provided to the processors 212, 222 through the memories 211, 221 or the communication modules 213, 223. For example, the processors 212, 222 can be configured to execute received instructions according to program codes stored in recording devices such as the memories 211, 221.

[0037] The communication modules 213 and 223 can provide a function of enabling the electronic device 110 and the server 150 to communicate with each other through the network 170, and can provide a function of enabling the electronic device 110 and / or the server 150 to communicate with other electronic devices (e.g., the electronic device 120) or other servers (e.g., the server 160). For example, the processor 212 of the electronic device 110 can deliver a request generated according to a program code stored in a recording device such as the memory 211 to the server 150 through the network 170 according to the control of the communication module 213. In contrast, a control signal, an instruction, content, a file, etc. provided according to the control of the processor 222 of the server 150 can be provided to the electronic device 110 through the communication module 213 of the electronic device 110 via the communication module 223 and the network 170. For example, a control signal, an instruction, content, a file, etc. of the server 150 received through the communication module 213 can be delivered to the processor 212 or the memory 211, and the content or the file, etc. can be stored in a storage device (the above-mentioned permanent storage device) which the electronic device 110 can further include.

[0038] The input / output interface 214 can be a unit for interfacing with the input / output device 215. For example, the input device can include a keyboard, a mouse, a microphone, a camera, etc., and the output device can include a display, a speaker, a haptic feedback device, etc. As another example, the input / output interface 214 can also be a unit for interfacing with a device for inputting and outputting functions combined together such as a touch screen. The input / output device 215 can be configured as one device with the electronic device 110. Also, the input / output interface 224 of the server 150 can be a unit for interfacing with a device for inputting or outputting connected to the server 150 or included in the server 150 (not shown). As a more specific example, the processor 212 of the electronic device 110, in the process of processing an instruction of a computer program loaded into the memory 211, can display a service screen or content configured using data provided by the server 150 or the electronic device 120 on a display through the input / output interface 214.

[0039] Also, in another embodiment, the electronic device 110 and the server 150 can include more or less than Figure 2The electronic device 110 may include at least a portion of the input / output devices 215 described above, or may also include other structural elements such as a transceiver, a Global Positioning System (GPS) module, a camera, various sensors, a database, etc. As a more specific example, in the case of a smartphone, the electronic device 110 may also include various structural elements typically found in smartphones, such as an accelerometer, a gyroscope, a camera module, various physical buttons, buttons utilizing a touchpad, input / output ports, and a vibrator for vibration.

[0040] The following describes specific embodiments of methods and systems for sharing content during VoIP calls.

[0041] Figure 3 The figure is a diagram illustrating an example of the structural elements that a processor of an electronic device according to an embodiment of the present invention may include. Figure 4 A flowchart illustrating an example of a method executable by an electronic device according to an embodiment of the present invention.

[0042] The electronic device 110 in this embodiment can be configured as a computer-implemented VoIP system. As an example, the VoIP system can be implemented as a stand-alone program or as an in-app component of a specific application (e.g., instant messaging software), and can operate on the aforementioned specific application. Depending on the situation, it can provide VoIP services through interaction with the server 150.

[0043] A VoIP system implemented in electronic device 110 based on instructions provided by an application installed in electronic device 110 is executable. Figure 4 The content sharing method shown.

[0044] In order to execute Figure 4 Content sharing methods, such as Figure 3 As shown, as structural elements, the processor 212 of the electronic device 110 may include a call providing unit 310, an interface providing unit 320, a sharing request unit 330, a content display unit 340, and a response display unit 350. According to embodiments, structural elements of the processor 212 may be selectively included in or excluded from the processor 212. Furthermore, according to embodiments, structural elements of the processor 212 may be separated or combined to improve the functionality of the processor 212.

[0045] The processor 212 and its structural elements described above can control the electronic device 110 to perform operations. Figure 4The content sharing method includes a plurality of steps (steps S410 to S450). For example, the processor 212 and the structural elements of the processor 212 can be implemented to execute instructions of codes of an operating system and at least one program included in the memory 211.

[0046] The structural elements of the processor 212 can be a manifestation of different functions of the processor 212 performed by the processor 212 according to instructions provided by program codes stored in the electronic device 110 (as an example, instructions provided by an application driven in the electronic device 110). For example, the electronic device 110 can utilize the call providing part 310 as a manifestation of the functionality of the processor 212 for providing a network phone call according to the above instructions.

[0047] The processor 212 can read necessary instructions from the memory 211 in which instructions related to the control of the electronic device 110 are loaded. In this case, the above read instructions can include instructions for controlling the processor 212 to perform a plurality of steps (steps S410 to S450) described later.

[0048] Referring to Figure 4 In step S410, the call providing part 310 can provide a network phone call in which two or more users including a user of the electronic device 110 participate. The network phone call can include a voice call function and a video call function based on a network phone. In this case, the network phone call can support one-to-one call and can support group call within a predetermined maximum number of people. As an example, in the case where a network phone call is requested between users of the electronic devices 110, 120, 130, 140, the server 150 can deliver a call request to the counterpart electronic devices, and in response to this, can connect a call channel based on a network phone targeting a plurality of users who accept a call participation request.

[0049] In the case of a video call, the call providing part 310 can deliver a call participation intention of the user of the electronic device 110 to the server 150, and after receiving participant images of the network phone call as individual images from the server 150, render the received participant images into one screen to constitute a video call screen. The server 150 transmits each participant image as an individual image to the electronic device 110, and in the electronic device 110, constitutes a video call screen in which a plurality of participant images are one screen, rather than forming one image including a plurality of participant images on the server 150 side to provide to the electronic device 110.

[0050] In the case of a voice call, the call providing unit 310 can take the user who accepted the call participation request as an object, generate a participant list based on the introduction information, etc., and compose a voice call screen including the participant list.

[0051] In step S420, in the case where a user input of a call content sharing function is received from the user of the electronic device 110 during a network phone call, the interface providing unit 320 can provide a graphical user interface for specifying content to be shared with participants in the call. The user input can include a media type selected as a content sharing medium, for example, the media type can include screen sharing, a content provider providing a content sharing service such as youtube, etc. The content sharing function is a function of sharing media content between call participants through screen sharing or a content sharing service, etc. during a network phone call. The interface providing unit 320 is an interface for specifying content to be shared with call participants on a network phone call screen, and can provide a graphical user interface corresponding to the media type based on the user input.

[0052] As an example, in the case of sharing content of a content provider as one of the media types, the interface providing unit 320 can provide a graphical user interface for searching for content. After the interface providing unit 320 preferentially provides a provider selection screen for selecting a content provider, if a content provider is selected through the provider selection screen, a graphical user interface capable of searching for content within a platform of the selected content provider can be provided. In other words, the interface providing unit 320 can provide a content provider list providing a content sharing service, and can provide an interface capable of searching for content of a content provider selected in the content provider list. After a specific content provider is selected by the user as a media type, content to be shared with call participants can be specified through content search within a platform of the corresponding content provider.

[0053] As another example, as another one of the media types, in a case where the content is shared through screen sharing, the interface providing part 320 can provide a graphical user interface including a preview for the screen sharing object. In a case where the content of the content provider is shared through a uniform resource locator, a graphical user interface including a preview for the content of the corresponding uniform resource locator can also be provided. In a state where the uniform resource locator of the content of the content provider is copied in the clipboard enters the web phone call screen, a call instruction of the content sharing function can be identified, in which case, as the call instruction of the content sharing function is identified, the interface providing part 320 can provide a graphical user interface including a preview for the uniform resource locator copied in the clipboard. The user can designate the content to be shared with the call participant after confirming the media content of the screen sharing object or the uniform resource locator through the preview.

[0054] In providing the graphical user interface for the content search, the interface providing part 320 recommends a search term for searching the content based on the brief information set in the chat room during the web phone call or the data shared in the corresponding chat room in the past. Among them, the brief information can include the theme information of the chat room or the like. The search term can be recommended considering the brief information of the user participating in the chat room, for example, age, gender, residential area, or the like. In addition to the brief information, by analyzing the past shared data, for example, the messages or the content transmitted through the chat room, the interest can be derived, and the keyword related to the derived interest can be provided as the recommended search term. In a case where the proportion of the messages related to the new coronavirus in the chat room is above a prescribed ratio, a search term related to the new coronavirus can be recommended. Also, in a case where the statistical information related to the chat room, for example, the number of sharing times for the recent news is large, a search term related to the corresponding news can be recommended. Also, based on the content sharing history of the chat room, for example, in a case where the video of the LINE Friends channel is shared using LINE LIVE the most during the web phone call, a real-time popularity search term of LINE LIVE or a LINE Friends channel related search term can be recommended. In a case where the open chat room in which a plurality of users gather and transmit / receive content based on a specific theme on the instant messenger platform, the search term can be recommended based on the theme information set in the open chat room.

[0055] In step S430, in a case where the sharing request for the specific content is received from the user of the electronic device 110 through the graphical user interface, the sharing request part 330 transmits the sharing request for the corresponding content to the server 150 in the protocol internal metadata generation of the web phone call channel. In other words, the sharing request part 330 can generate the metadata of the web phone call data packet including the data corresponding to the content sharing request to transmit to the server 150.

[0056] The content sharing protocol for data exchange in the network phone call state can be defined in advance according to the media type. As an example, the content sharing protocol can include an identifier (identifier) representing a content sharing function and metadata defined according to the media type. The start (set) and end (unset) for content sharing can be processed by the event of the core. In the case of the media type being a content provider, an identifier of the media type, an identifier of the content designated by the user, a title, a thumbnail, a playback state (play, pause, etc.), a playback position, etc. can be added to the metadata of the network phone call packet, and in the case of sharing the content of the content provider through a uniform resource locator, an identifier of the media type, a content uniform resource locator, a playback state, a playback position, etc. can be added to the metadata of the network phone call packet. In the case of the media type being a screen sharing, an identifier of the media type, data required for screen sharing, etc. can be added to the metadata of the network phone call packet. The sharing request part 330 can describe all data required to play back the content designated through the graphical user interface (preview or search result) in synchronization between the user of the electronic device 110 and the call participant as metadata of the network phone and transmit to the server 150.

[0057] In step S440, the content display part 340 can display the network phone call screen and the content based on the sharing request of the user of the electronic device 110. After the sharing request for the content designated through the graphical user interface is delivered from the user of the electronic device 110 to the server 150, the content display part 340 can display the playback screen (hereinafter, referred to as "content sharing screen") of the corresponding content together with the network phone call screen. As an example, the content display part 340 can adjust one screen combined with the network phone call screen by including the content sharing screen in the network phone call screen through the interaction with the call providing part 310 and display. As another example, the content display part 340 can overlap and display the content sharing screen as an individual screen such as PIP on the network phone call screen.

[0058] The server 150 can identify the media type and the shared object content, etc. through the metadata of the network phone call packet, determine the electronic device 110 requesting the content sharing as a sender among the call participants, and determine the remaining electronic devices (at least one of 120, 130, 140) as a receiver.

[0059] The server 150 delivers metadata of a network phone call data packet containing data corresponding to the content sharing request of the sender to the electronic device (at least one of 120, 130, 140) corresponding to the receiver. The server 150 can function as a relay for directly delivering metadata of a network phone call data packet between the sender and the receiver for content sharing.

[0060] In the electronic device (at least one of 120, 130, 140) corresponding to the receiver, the content is shared in the same state as the sender based on the metadata of the network phone call data packet received from the sender through the server 150. In the case of a video as a sharing object, in the sender side, only information (e.g., uniform resource locator information (or content identifier) and information about a start time point) related to the video is contained in the metadata, or a video data packet that needs to be shared is contained together to be delivered to the receiver, and in the receiver side, the video is loaded from a content provider based on the information contained in the metadata or directly played back from the network phone call data packet received from the sender in the playback state of the sender side without an additional loading job. In the case of a sharing request for a playback list, in the receiver side, the state of the sender side is received through the metadata of the network phone call data packet, and the video information contained in the playback list is directly inquired to fill in the corresponding information and start playback.

[0061] For a network phone call, the server 150 (e.g., a relay server) that delivers a network phone call data packet is used, but is not limited thereto, and in the absence of the server 150, a peer to peer (P2P) network as communication between individuals can be used. The electronic device 110 can directly deliver metadata of a network phone call data packet containing data corresponding to the content sharing request of the user to the electronic device 120 corresponding to the receiver through a P2P network of one-to-one communication. Accordingly, in the absence of the server 150, between the sender and the receiver, a network phone call data packet can be transmitted and received through a P2P network, and metadata of a network phone call data packet can be used to share content during a network phone call.

[0062] An operation authority for content shared with the receiver basically exists in the sender, and in the case where the user at the sender side changes a playback position or a playback state, etc., metadata of a network phone call data packet containing the changed data is delivered to the receiver, and the state of content being shared between the sender and the receiver is synchronized according to the operation of the sender. As another example, the receiver can obtain consent from the sender or other call participants and change the state of content in sharing or propose sharing of other content based thereon.

[0063] In step S450, the reaction display part 350 can display a reaction corresponding to a user input on the network phone call screen or the content sharing screen in the environment of sharing content during a network phone call. Hereinafter, a specific method of displaying a reaction on the network phone call screen or the content sharing screen will be described in detail.

[0064] Figures 5 to 8 An exemplary diagram for explaining a process of sharing content during a network phone call of an embodiment of the present application.

[0065] Figures 5 to 8 An interface screen example displayed on the display of the electronic device 110 is illustrated.

[0066] Referring to Figure 5 , the processor 212 receives participant images of a network phone call as individual images, respectively, and then renders the received participant images into one screen to display a video call screen 510. During a network phone call, in the case where a user input of a call content sharing function is received from a user of the electronic device 110, the processor 212 can provide a graphical user interface for designating content to be shared with participants in a call.

[0067] The video call screen 510 can be adjusted according to a user's environment setting or the number of participants, for example, can be constituted in a form in which participant images are all displayed in one screen or in a form in which participant images are arranged in one direction and then can be slid in the arrangement direction.

[0068] In the case where a user desires to share content of a content provider such as youtube, which is one of media types, the processor 212 can provide a graphical user interface 520 for searching content, as illustrated in Figure 5 . In the case where connectable content providers are plural, the processor 212 can provide a provider selection screen for selecting a content provider in priority, and then, if a content provider is selected through the provider selection screen, can provide a graphical user interface 520 capable of searching content within a platform of the selected content provider.

[0069] In the case where a user shares content through screen sharing, which is another of media types, the processor 212 can provide a graphical user interface 630 including a preview of a screen, which is executed in the electronic device 110, for a screen designated as a screen sharing object, as illustrated in Figure 6 .

[0070] The processor 212 can display the content search graphical user interface 520 and the preview graphical user interface 630 on the video call screen 510 in overlap as a graphical user interface for designating content (refer to Figure 5), or adjust the video call screen 510 to be displayed in one screen in combination with the video call screen 510 (see Figure 6 ).

[0071] In a case where the user requests the designated content sharing after designating the content through the content search graphical user interface 520 or the preview graphical user interface 630, the processor 212 can deliver the metadata of the network phone call data packet containing data for the corresponding content sharing to the receiver. At the same time, as shown in Figure 7 , the processor 212 can display the video call screen 510 and the content sharing screen 740 which is a playback screen of the requested sharing content.

[0072] In a case where the content sharing screen 740 is displayed in one screen in combination with the video call screen 510, the video call screen 510 can display the participant images combined in a remaining area other than the content sharing screen 740. In this case, the video call screen 510 can be constituted in a form of displaying all or a part of the participant images which provide own images. Or, the video call screen 510 can be constituted in a form of arranging the participant images in one direction and being slidable in the arranged direction.

[0073] In a case of a voice call other than a video call, as shown in Figure 8 , the processor 212 can provide a voice call screen 810 containing a participant list participating in the call as a network phone call screen. In a case where content is shared during the voice call, the processor 212 can display the voice call screen 810 containing the participant list and the content sharing screen 740.

[0074] Therefore, in the present embodiment, various media content can be shared between the call participants during the network phone call (voice call or video call) using the social graph.

[0075] Figure 9 A flowchart showing an example of a reaction display method during a network phone call according to an embodiment of the present application. Figure 9 The reaction display method of Figure 4 may be executed by the reaction display part 350 and can include the step S450 included in the content sharing method of

[0076] In step S901, the reaction display part 350 can receive position information of a user input reaction of the electronic device 110 during a network phone call (voice call or video call). As an example, the reaction display part 350 can recognize position coordinates where the user actually input the reaction by touch or the like as position information on a screen (network phone call screen or content sharing screen) where the input user reacted. The position coordinates where the reaction was input can be formed of a pair of XY coordinates according to the reaction type, and in addition, as trajectory information that tracks one or more than two touches, can be formed of a set of XY coordinates. As another example, in the case of a reaction input to the content sharing screen, the reaction display part 350 can recognize a playback position of the content being played back on the content sharing screen at the time point when the user input the reaction as position information of the reaction of the input user.

[0077] In step S902, the reaction display part 350 can specify an object (hereinafter, referred to as a "reaction object") to which the user of the electronic device 110 input the reaction in the participant image included in the network phone call screen and the content sharing screen on the basis of the position information received in step S901. The user can input a reaction to any one of the participant images included in the network phone call screen or the content sharing screen, and in this case, the reaction display part 350 specifies the reaction object on the basis of the position information of the reaction of the input user. As an example, the reaction display part 350 can specify a screen corresponding to the position coordinates where the reaction was actually input as the reaction object in the network phone call screen and the content sharing screen. In the case where the user input the reaction to the image of a specific participant in the network phone call screen, the reaction object can be specified by an identifier of the corresponding participant. As another example, in the case where the playback position of the content is recognized as the reaction position information, the reaction display part 350 can specify the content sharing screen as the reaction object. As another example, the reaction display part 350 can analyze at least a part of the screen corresponding to the playback position of the content to specify the reaction object. In this case, the reaction display part 350 can transmit image data related to at least a part of the screen to the server 150, can receive information related to things, characters, places, or the like recognized from the image data from the server 150 using an image learning model constructed on the server 150 or a platform capable of interacting with the server 150, and thus can specify the reaction object.

[0078] In step S903, the reaction display part 350 can transmit information about the reaction input by the user of the electronic device 110 and the reaction object and position information about the corresponding reaction to other users participating in the network telephone call, i.e., call participants. The information about the reaction can include a reaction type, and the information about the reaction object can include information indicating whether it is a reaction to one participant image in the network telephone call screen or a reaction to the content sharing screen. The reaction display part 350 can generate the information about the reaction and the reaction object and the reaction position information as metadata in a protocol of the network telephone call channel and transmit it to the call participants through the server 150 or the peer-to-peer network. As an example, the reaction display part 350 transmits metadata of a network telephone call data packet including the information about the reaction and the reaction object and the reaction position information to all users participating in the network telephone call as an object. As another example, the reaction display part 350 specifies a participant corresponding to the reaction object among the users participating in the network telephone call and transmits metadata of a network telephone call data packet including the information about the reaction and the reaction position information to the corresponding participant.

[0079] In step S904, the reaction display part 350 can display the reaction on the screen corresponding to the reaction object in the network telephone call screen and the content sharing screen based on the reaction position information at the corresponding position. According to the reaction, an object matching each reaction can be defined in advance, and the reaction display part 350 can display an object matching the reaction input by the user in association with the reaction object at the position where the corresponding reaction is input. In this case, the object can include visual elements such as images, effect animations, etc. defined in advance corresponding to the reaction, and in addition to such visual elements, can include tactile elements such as haptics or audio-based auditory elements.

[0080] The electronic devices 120, 130, and 140 of the other users participating in the network telephone call display the network telephone call screen and the content sharing screen, and in this case, based on the metadata of the network telephone call data packet received from the electronic device 110, the reaction input by the user of the electronic device 110 and the reaction object and the reaction position information can be analyzed. In the electronic devices 120, 130, and 140 of the other users, the reaction input by the user of the electronic device 110 is displayed on the screen corresponding to the reaction object in the network telephone call screen and the content sharing screen based on the reaction position information at the corresponding position. According to an embodiment, among the electronic devices 120, 130, and 140 of the other users participating in the network telephone call, only the electronic device of the participant corresponding to the reaction object (e.g., 120 among 120, 130, and 140) can receive the reaction-related metadata from the electronic device 110 and display the reaction input by the user of the electronic device 110 on the participant image screen of the own participant in the network telephone call screen.

[0081] Figures 10 to 12 FIG. 10 is an example diagram illustrating a process of displaying a reaction in a web call according to an embodiment of the present disclosure.

[0082] Figure 10 and Figure 11 FIG. 11 is a diagram illustrating a scenario in which a reaction is delivered between users in a video call.

[0083] Referring to Figure 10 , the user of the electronic device 110 can share the same video with other users participating in the video call through the content sharing screen 740, in which case the video is watched together and a reaction can be left at a specific playback position.

[0084] In the case in which a reaction is input through the reaction input interface 1001 (e.g., a like button) related to the content sharing screen 740, the processor 212 can identify the playback position of the content at the point in time at which the user input the reaction as position information of the input user's reaction.

[0085] The processor 212 can display an image 1002 of the reaction input by the user on the content sharing screen 740, as an example, on a progress bar 1041 indicating the playback position of the content, in which case the reaction image 1002 can be displayed at a position corresponding to the playback position at the point in time at which the user input the reaction.

[0086] The electronic devices 120, 130, 140 of the other users participating in the web call can each display the reaction input by the user of the electronic device 110 on the content sharing screen 740 shared with the user of the electronic device 110 at the input position of the reaction, whereby the users participating in the web call can share the position at which the reaction is left.

[0087] Referring to Figure 11 , the user of the electronic device 110 can share the shopping website screen 1150 designated as a screen sharing object in a screen executed in the electronic device 110 as a content sharing screen with other users participating in the video call, in which case a reaction to a specific matter can be left in the shopping website screen 1150 shared through the screen.

[0088] The processor 212 can display a reaction input interface 1101 according to a user input to the shopping website screen 1150 corresponding to the content sharing screen, in which case the user can input a reaction at the position of the specific matter within the shopping website screen 1150 through the reaction input interface 1101.

[0089] The processor 212 can display an image 1102 of the user's input response on the shopping website screen 1150 at the location of the user's input response.

[0090] The electronic devices 120, 130, and 140 of other users participating in the VoIP call can display the responses entered by the user of electronic device 110 on the shopping website screen 1150 shared with the user of electronic device 110, thereby allowing users participating in the VoIP call to share the positions where responses are left.

[0091] The processor 212 can identify the user who inputs the response among the participants in the VoIP call, and display the response image 1002, 1102 on the corresponding user's image screen within the video call screen 510. Thus, it is easy to grasp which user's response is among the participants in the VoIP call.

[0092] In addition to the scenarios mentioned above, collaboration using screen sharing can also be achieved through... Figure 11 The scenario described is similar, where users participating in a VoIP call share a screen displaying collaborative documents and exchange questions or opinions about specific parts of the documents displayed on the screen.

[0093] In addition to responding to content shared during a VoIP call, a specific participant within the video call screen 510 can also be designated to submit a response. When a user of the electronic device 110 selects a specific participant to input a response within the video call screen 510, the processor 212 can display the response image on the corresponding participant's video screen within the video call screen 510.

[0094] During a VoIP call, responses can be sent and displayed to one party, or sent and displayed bidirectionally. For example, if user 1 in a VoIP call draws part of a heart and sends it, user 2 can respond by drawing the rest of the heart.

[0095] Furthermore, the processor 212 can transmit and manage information related to the media content shared during the VoIP call and information related to the responses received during the VoIP call to the chat room where the VoIP call is being made.

[0096] Reference Figure 12 ,like Figure 10 In a scenario where a user participating in a VoIP call leaves a response to a video shared during the VoIP call, the processor 212 can receive the corresponding video thumbnail, the respondent, the time of the response, and the user's nickname, etc., as a message type 1201 included in the chat room 1200 and display it.

[0097] And, asFigure 11 In the case where the user participating in the net phone call leaves a reaction with respect to the screen shared during the net phone call, the processor 212 can receive a capture image of the shared screen, a reaction object, a time at which the reaction is input, and a user nickname, etc. as a message type 1202 included in the chat room 1200 and display. In this case, in the case where the user participating in the net phone call inputs a reaction with respect to another user during the net phone call, an image of the user who is the reaction object and the reaction of the user who inputs the reaction can be displayed.

[0098] Accordingly, the processor 212 can manage the media content and the information related to the reaction exchanged during the net phone call through the chat room 1200 and construct a history related to the net phone call. In other words, even after the net phone call is ended, the media content and the information related to the reaction exchanged during the net phone call can be saved in the chat room 1200.

[0099] As described above, according to the embodiment of the present application, the reaction exchanged between the participants of the net phone call can be displayed in real time during the net phone call.

[0100] The apparatus described above can be implemented as a hardware configuration element, a software configuration element, and / or a combination of the hardware configuration element and the software configuration element. For example, the apparatus and the configuration elements described in the embodiments can be implemented using one or more general-purpose computers or special-purpose computers such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor (DSP), a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing apparatus can execute an operating system (OS) and one or more software applications executed on the operating system. Also, the processing apparatus accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For the convenience of understanding, a case where only one processing apparatus is used is described, but a person skilled in the art to which the present application pertains can know that the processing apparatus can include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing apparatus can include a plurality of processors or one processor and one controller. Also, other processing configurations such as a parallel processor can be used.

[0101] The software can include a computer program, code, instructions, or a combination of one or more thereof, which can configure a processing device or give an instruction to a processing device, either independently or collectively, in a manner of operating as needed. The software and / or data can be embodied in any type of machine, component, physical device, computer storage medium or means, in order to be analyzed by a processing device or in order to provide an instruction or data to a processing device. The software can be dispersed on computer systems connected through a network, so as to be stored or executed by a dispersed method. The software and data can be stored in one or more computer-readable recording media.

[0102] The method of the embodiments can be realized as program instructions that can be executed by various computer units, recorded in a computer-readable medium. In this case, the medium can continue to store a program that can be executed by a computer, or temporarily store for execution or download. Also, the medium can be a variety of recording units or storage units in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to one computer system, but can be dispersed on a network. As an example of the medium, there can be a medium configured to store program instructions, including a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a read-only memory, a random access memory, a flash memory, etc. Also, as an example of other media, an application store for circulation of applications or a website for supply or circulation of other various software, a recording medium or a storage medium managed in a server, etc. can be cited.

[0103] As described above, although a plurality of embodiments are explained through limited embodiments and drawings, as long as a person skilled in the art to which the present invention pertains, various modifications and changes can be made from the above-described description. For example, even if the explained technology is executed in a different order from the explained method and / or the structural elements of the explained system, structure, device, circuit, etc. are combined or integrated in a different form from the explained method, or are replaced or substituted by other structural elements or equivalents, an appropriate result can be achieved.

[0104] Therefore, other examples, other embodiments, and equivalents to the claimed scope belong to the appended claims.

Claims

1. A reaction display method, executed in a computer device, characterized in that, The aforementioned computer device includes at least one processor configured to execute computer-readable instructions contained in memory. The above-mentioned reaction display methods include: The display step involves, through at least one of the aforementioned processors, displaying the VoIP call screen and the content sharing screen during a VoIP call in which the user of the aforementioned computer device is involved, while sharing media content. The receiving step involves receiving location information input by the user of the computer device during the VoIP call via at least one of the processors described above. The transmission step involves transmitting information related to the above-mentioned response and the above-mentioned location information to at least one other user participating in the above-mentioned VoIP call, through at least one processor. The display step involves using at least one processor to display the aforementioned reaction on the VoIP call screen or the content sharing screen based on the aforementioned location information; Through at least one of the processors, as the user inputs the response, a chat room message is automatically generated. The chat room message includes information related to the media content shared during the VoIP call and information related to the response received during the VoIP call. The at least one processor described above transmits the chat room messages to a chat room provided via an instant messaging service in connection with the aforementioned VoIP call; and The aforementioned chat room messages will be displayed in the aforementioned chat room. The steps for automatically generating the chat room messages include generating a message containing visual information about the media content and a response object representing the response. The visual information regarding the aforementioned media content includes at least one of a thumbnail of the aforementioned media content and a screenshot of the aforementioned media content. The above-mentioned reaction display method further includes the step of managing visual information containing the above-mentioned media content and messages representing the reaction objects according to the message types included in the above-mentioned chat room.

2. The reaction display method according to claim 1, characterized in that, The above receiving steps include the following identification steps: identifying the location coordinates of the input reaction as location information on the above VoIP call screen or the above content sharing screen.

3. The reaction display method according to claim 1, characterized in that, The above receiving step includes the following identification step: when responding to the above content sharing screen input, the playback position of the content being played on the above content sharing screen at the time of the user input response on the above computer device is identified as the above position information.

4. The reaction display method according to claim 1, characterized in that, The above-described reaction display method further includes the following specified step: using at least one processor, based on the location information, specifying the reaction object in the participant image included in the above-described VoIP call screen and the above-described content sharing screen.

5. The reaction display method according to claim 4, characterized in that, In the specified steps described above, in the aforementioned VoIP call screen and the aforementioned content sharing screen, the screen corresponding to the location coordinates of the input reaction is designated as the reaction object.

6. The reaction display method according to claim 4, characterized in that, In the specified steps described above, when the playback position of the content being played back on the content-sharing screen is received as the location information, the content-sharing screen is designated as the response object in both the VoIP call screen and the content-sharing screen.

7. The reaction display method according to claim 4, characterized in that, In the above transmission steps, a user corresponding to the above response object is designated from among the other users participating in the above VoIP call, and the designated user is used as the object to transmit information related to the above response and the above location information.

8. The reaction display method according to claim 1, characterized in that, In the above transmission steps, metadata containing information related to the above reaction and the above location information is generated and transmitted. On the computer devices of the other users, the above-mentioned reaction is displayed on the screen of the other users' computer devices based on the above-mentioned metadata.

9. The reaction display method according to claim 1, characterized in that, In the above display steps, in the above VoIP call screen and the above content sharing screen, an object matching the above reaction is displayed at the position on the screen corresponding to the above location information.

10. A non-transitory computer-readable recording medium, characterized in that, The device contains a computer program for executing the reaction display method according to any one of claims 1 to 9 in a computer device.

11. A computer device, characterized in that, Includes at least one processor configured to execute computer-readable instructions contained in memory. At least one of the processors described above processes the following procedure: During the display process, when media content is shared during a VoIP call involving a user of the aforementioned computer device, the VoIP call screen and the content sharing screen are displayed. During the aforementioned VoIP call, the location information input by the user on the aforementioned computer device is received. During the transmission process, at least one other user participating in the aforementioned VoIP call is targeted with information related to the aforementioned response and the aforementioned location information; The display process, based on the aforementioned location information, displays the aforementioned reaction on the aforementioned VoIP call screen or the aforementioned content sharing screen; As the user inputs the above-mentioned response, a chat room message is automatically generated. The chat room message includes information related to the media content shared during the above-mentioned VoIP call and information related to the above-mentioned response received during the above-mentioned VoIP call. Transmit the aforementioned chat room messages to chat rooms provided through instant messaging services in connection with the aforementioned VoIP calls; and The aforementioned chat room messages will be displayed in the aforementioned chat room. The process of automatically generating the chat room messages mentioned above includes the process of generating visual information containing the aforementioned media content and a message representing the reaction object. The visual information regarding the aforementioned media content includes at least one of a thumbnail of the aforementioned media content and a screenshot of the aforementioned media content. The processor further processes the process of managing visual information containing the aforementioned media content and messages representing the aforementioned reactions according to the message types included in the aforementioned chat room.

12. The computer apparatus according to claim 11, characterized in that, The processor at least once identifies the location coordinates of the input reaction as location information on the VoIP call screen or the content sharing screen.

13. The computer apparatus according to claim 11, characterized in that, In response to the input on the content-sharing screen, the at least one processor will identify the playback position of the content being played on the content-sharing screen at the time of the user input response on the computer device as the position information.

14. The computer apparatus according to claim 11, characterized in that, Based on the aforementioned location information, at least one processor designates a response object in the participant image included in the aforementioned VoIP call screen and in the aforementioned content sharing screen.

15. The computer apparatus according to claim 14, characterized in that, The processor at least once designates a user corresponding to the reaction object from among other users participating in the VoIP call, and transmits information related to the reaction and the location information to the designated user as the object.

16. The computer apparatus according to claim 11, characterized in that, The at least one processor generates metadata for transmitting VoIP call data packets containing information related to the aforementioned reaction and the aforementioned location information. On the computer devices of the other users, the above-mentioned reaction is displayed on the screen of the other users' computer devices based on the above-mentioned metadata.

17. The computer apparatus according to claim 11, characterized in that, In the aforementioned network telephony call screen and the aforementioned content sharing screen, at least one of the aforementioned processors displays an object at a position on the screen corresponding to the aforementioned location information that matches the aforementioned response.

Images