Electronic device and control method thereof

Abstract

An electronic device is disclosed. This electronic device may comprise: a display; a memory for storing one or more instructions; and one or more processors operatively connected to the display and the memory, wherein the one or more processors execute the one or more instructions to: control the display to display a UI screen including additional information related to content while the content is shared with an external device; obtain a UI screen in which multiple pieces of additional information are rearranged on the basis of user history information about multiple pieces of different types of additional information included in the UI screen; and share the UI screen including multiple pieces of information rearranged according to a preconfigured event with the external device.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device performing a video call function and a method for controlling the same.

[0002] Advances in electronic technology have led to the development and proliferation of various types of electronic devices. In particular, display devices such as TVs and laptops are being developed in various forms.

[0003] Recently, services have been provided that allow multiple users to watch content together while making video calls.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0005] An electronic device according to one or more embodiments includes: a display; a memory storing one or more instructions; and one or more processors; wherein the one or more processors control the display to display a UI screen including additional information related to content while the content is shared with an external device by executing the one or more instructions, obtain a UI screen in which the plurality of additional information is rearranged based on user history information about a plurality of different types of additional information included in the UI screen, and share the UI screen including the plurality of rearranged pieces of information with the external device according to a preset event.

[0006] According to one or more embodiments, the plurality of additional information may be rearranged on the UI screen based on the priority of the plurality of additional information based on the score of each of the plurality of additional information, when a score corresponding to each of the plurality of additional information is calculated based on the number of different types of activities for each of the plurality of additional information.

[0007] According to one or more embodiments, the different types of activities include sharing, viewing and searching, and the one or more processors can, by executing the one or more commands, assign different weights to the sharing, the viewing and the searching to produce a score corresponding to each of the plurality of additional information.

[0008] According to one or more embodiments, the one or more processors, by executing the one or more commands, provide a content report including information about user reactions and commentary information when viewing of the content is finished, wherein the commentary information can be obtained while the identified user reaction occurs when the user reaction is identified while the content is shared with the external device.

[0009] According to one or more embodiments, the user reaction may include at least one of a voice reaction, a chat reaction, a facial expression reaction, or a gesture reaction, and the commentary information may include at least one of screen capture information, conversation information, or shared information.

[0010] According to one or more embodiments, the content report includes a scene-by-scene rating calculated in timeline order based on information about the user reaction and the commentary information, and a content section from the time a specific user reaction is identified to the time a reaction related to the specific user reaction ends can be identified as one scene.

[0011] According to one or more embodiments, the one or more processors, by executing the one or more commands, obtain commentary information related to the content while the user reaction occurs, if the user reaction is identified as corresponding to a preset emotion, wherein the preset emotion may include at least one of a positive emotion or a negative emotion.

[0012] According to one or more embodiments, the one or more processors may control the display to display a user-captured image on one side of the content area by executing the one or more commands, and, when the user reaction is identified as corresponding to the preset emotion, control the display to temporarily switch and display the user-captured image and the content area including the user reaction.

[0013] According to one or more embodiments, the plurality of additional information of different types includes at least one of casting, plot, content background, producer, director, or review score, and the plurality of additional information may be rearranged on the UI screen based on the updated matrix when a matrix representing scores for the plurality of additional information is obtained based on a preset algorithm and the obtained matrix is ​​updated based on user history information for the plurality of additional information.

[0014] A server according to one or more embodiments includes: a communication interface; a memory storing one or more instructions; and one or more processors operatively connected to the communication interface and the memory, wherein the one or more processors execute the one or more instructions to provide a UI screen including additional information related to content to one of the plurality of external devices based on a preset event that occurs while the plurality of external devices are sharing content, obtain a UI screen in which the plurality of additional information is rearranged based on user history information about a plurality of different types of additional information included in the UI screen, and share the UI screen including the rearranged plurality of pieces of information with the remaining devices among the plurality of external devices based on the preset event. The plurality of pieces of additional information may be rearranged on the UI screen based on a priority of the plurality of pieces of additional information based on a score of each piece of the plurality of pieces of additional information when a score corresponding to each piece of the plurality of pieces of additional information is calculated based on a number of different types of activities for each piece of the plurality of pieces of additional information.

[0015] A method of controlling an electronic device according to one or more embodiments includes: displaying a UI screen including additional information related to content while the content is shared with an external device; obtaining a UI screen in which the plurality of additional information is rearranged based on user history information about a plurality of different types of additional information included in the UI screen; and sharing the UI screen including the plurality of rearranged pieces of information with the external device according to a preset event.

[0016] A non-transitory computer-readable medium storing computer instructions that, when executed by a processor of an electronic device according to one or more embodiments, cause the electronic device to perform an operation, the operation comprising: displaying a UI screen including additional information related to content while the content is shared with an external device; obtaining a UI screen in which a plurality of additional information are rearranged based on user history information about a plurality of different types of additional information included in the UI screen; and sharing the UI screen including the rearranged plurality of pieces of information with the external device according to a preset event.

[0017] The above and other aspects and features of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0018] FIG. 1 is a diagram illustrating a video call system according to one or more embodiments.

[0019] FIG. 2A is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention.

[0020] FIG. 2b is a drawing for explaining a detailed configuration of an electronic device according to one or more embodiments.

[0021] Figure 3a is a block diagram showing the configuration of a server according to one embodiment of the present invention.

[0022] FIG. 3b is a block diagram illustrating the operation of a server according to one embodiment of the present invention.

[0023] FIG. 4 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments.

[0024] FIG. 5 is a drawing illustrating an example of a UI screen according to one or more embodiments.

[0025] FIG. 6 is a flowchart illustrating a method for rearranging additional information according to one or more embodiments.

[0026] FIGS. 7A and 7B are drawings illustrating examples of UI screens according to one or more embodiments.

[0027] FIGS. 8A to 8E are drawings for explaining a method of rearranging additional information according to one or more embodiments.

[0028] FIG. 9 is a flowchart illustrating a method for providing a content report according to one or more embodiments.

[0029] FIG. 10 is a flowchart detailing a method for providing a content report according to one or more embodiments.

[0030] FIG. 11 is a diagram illustrating a method for generating viewing commentary information according to one or more embodiments.

[0031] FIG. 12 is a diagram illustrating a method for providing viewing commentary information according to one or more embodiments.

[0032] FIG. 13 is a diagram detailing a method for providing a user reaction highlighting effect according to one or more embodiments.

[0033] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0034] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions or cases of those skilled in the art, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0035] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0036] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to cases where (1) only A is included, (2) only B is included, or (3) both A and B are included.

[0037] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0038] When it is said that a component (e.g., a first component) is “operatively or communicatively coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0039] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0040] In some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0041] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0042] In the embodiments, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of "modules" or "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "module" or "part" that needs to be implemented as specific hardware.

[0043] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0044] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0045] FIG. 1 is a diagram illustrating a video call system according to one or more embodiments.

[0046] Referring to FIG. 1, a plurality of electronic devices (10, 20, 30) can communicate via a server (40). As an example, as illustrated in FIG. 1, a plurality of electronic devices (10, 20, 30) can view content together (or simultaneously) via the server (40). For example, the plurality of electronic devices (10, 20, 30) can be implemented as various types of electronic devices, such as a mobile phone, a smart phone, an electronic picture frame, a DID (Digital Information Display), a kiosk, a PMP (Portable Media Layer), an MP3 player, a game console, a LFD (Large Format Display), a notebook, a laptop, a TV, a monitor, a projector system, etc.

[0047] For example, multiple electronic devices (10, 20, 30) can execute an application (or software or program) (hereinafter, “joint viewing application”) that enables video calls and viewing content together, and transmit captured images acquired through a camera to a server (40). The joint viewing application may be provided in the form of an icon interface on the screens of the multiple electronic devices (10, 20, 30). For example, when the joint viewing application is executed, the camera may be turned on automatically, but in some cases, the camera may be turned on according to a user command.

[0048] For example, the server (40) may communicate with a content sharing application and provide a content sharing service through the content sharing application. For example, when a viewing group for sharing content is set up through the content sharing application, the server (40) may transmit the content for sharing to multiple electronic devices (10, 20, 30) set up as a group.

[0049] In one example, multiple electronic devices (10, 20, 30) and / or servers (40) may share interest information with a viewing group while viewing content together and generate and provide viewing commentary based on major events that occurred while viewing content together.

[0050] Accordingly, the following describes various embodiments of sharing information of interest with viewing groups and creating and providing viewing comments.

[0051] FIG. 2A is a block diagram showing the configuration of an electronic device according to an embodiment of the present invention.

[0052] According to FIG. 2A, the electronic device (100) includes a display (110), a memory (120), and one or more processors (130). According to an example, the electronic device (100) may be implemented as any one of the plurality of electronic devices (10, 20, 30) illustrated in FIG. 1.

[0053] The display (110) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The display (110) may also include a driving circuit, a backlight unit, etc., which may be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. Meanwhile, the display (110) may be implemented as a flexible display, a rollable display, a 3D display, a display in which a plurality of display modules are physically connected, etc. For example, if the electronic device (100) is implemented as a projector device, it may include an image projection unit instead of a display (110).

[0054] The memory (120) can store data required for various embodiments. Depending on the purpose of data storage, the memory (120) may be implemented as a memory embedded in the electronic device (100) or as a memory detachable from the electronic device (100). For example, data for operating the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding the functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100). Meanwhile, in the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD). In addition, in the case of memory that can be attached or detached to the electronic device (100'), it may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc. It can be implemented in the form of.

[0055] One or more processors (130) control the overall operation of the electronic device (100). Specifically, one or more processors (130) may be connected to each component of the electronic device (100) to control the overall operation of the electronic device (100). For example, one or more processors (130) may be operatively connected to the display (130) and the memory (120). The processor (130) may be composed of one or more processors.

[0056] One or more processors (130) may perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in the memory (120).

[0057] The one or more processors (130) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (130) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors (130) may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform a method according to one or more embodiments of the present disclosure by executing one or more instructions stored in a memory.

[0058] When a method according to one or more embodiments of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0059] One or more processors (130) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (130) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to one or more embodiments of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to one or more embodiments of the present disclosure.

[0060] When a method according to one or more embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0061] In the embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. Hereinafter, for the convenience of explanation, one or more processors (130) will be referred to as a processor (130).

[0062] According to one embodiment, the processor (130) may control the display (110) to display a UI screen including additional information related to the content while the content is shared with an external device. In one example, the content area and the UI screen may be received from an external server (e.g., server (40) of FIG. 1).

[0063] In one example, the additional information may include multiple types of additional information. For example, different types of additional information may indicate different attributes of the information. For example, the additional information may include different types of information, such as at least one of casting, plot, content background, producer, director, or review rating.

[0064] For example, a UI screen can be divided into sub-areas containing each type of additional information. For example, if the UI screen is vertically long, the sub-areas containing each type of additional information can be arranged vertically. For example, if the UI screen is horizontally long, the sub-areas containing each type of additional information can be arranged horizontally.

[0065] According to one embodiment, the processor (130) may acquire a UI screen in which multiple additional information is rearranged based on user history information regarding multiple types of additional information included in the UI screen. In one example, the UI screen in which multiple additional information is rearranged may be acquired via the server (40).

[0066] According to one embodiment, the processor (130) may share a UI screen containing a plurality of rearranged pieces of information with an external device based on a preset event. In one example, the UI screen (or information about the UI screen) containing a plurality of rearranged pieces of information may be shared with devices belonging to the same viewing group via the server (40).

[0067] Meanwhile, according to one embodiment, a UI screen with multiple additional information rearranged may be acquired through a server (40), but it is also possible to acquire the UI screen from an electronic device (100). Hereinafter, an embodiment of acquiring the UI screen from an electronic device (100) will be described. However, at least some of the operations performed on the electronic device (100) may be performed through the server (40) even without special mention.

[0068] According to one embodiment, a score corresponding to each of the plurality of pieces of additional information is calculated based on the number of times each type of activity occurs for each of the plurality of pieces of additional information, and then the priority of the plurality of pieces of additional information can be identified based on the score of each piece of additional information. In this case, the plurality of pieces of additional information can be rearranged on the UI screen according to the priority. In one example, the different types of activities may include at least one of sharing, viewing, and searching.

[0069] In one embodiment, a score corresponding to each of the plurality of additional pieces of information may be calculated by assigning different weights to different types of activities, for example, sharing, viewing, and searching.

[0070] For example, when a matrix representing scores for a plurality of pieces of additional information is obtained based on a preset algorithm and the obtained matrix is ​​updated based on user history information for the plurality of pieces of additional information, the plurality of pieces of additional information can be rearranged on a UI screen based on the updated matrix.

[0071] In one embodiment, when viewing of content is terminated, a UI screen containing commentary information (or commentary content or report) related to the content may be provided.

[0072] Meanwhile, according to one embodiment, commentary information may be obtained through a server (40), but it is also possible to obtain commentary information from an electronic device (100). Hereinafter, an embodiment of obtaining commentary information from an electronic device (100) will be described. However, at least some of the operations performed in the electronic device (100) may be performed through the server (40) even without special mention.

[0073] According to one embodiment, when a user reaction is identified while content is shared with an external device, commentary information related to the content may be acquired while the user reaction occurs. After viewing of the content is terminated, the processor (130) may provide a content report including information about the user reaction and commentary information. According to one example, the content report may include at least one of a voice reaction, a chat reaction, a facial expression reaction, or a gesture reaction. According to one example, the commentary information may include at least one of screen capture information, conversation information, or shared information. For example, the screen capture information may be information captured from the screen of the electronic device (100) while the user reaction occurs. For example, the conversation information may include at least one of conversation information between users including input text entered through an input window while the user reaction occurs or conversation information between users via voice. For example, shared information may include information shared between users while a user reaction occurs, for example, information shared through a shared UI screen.

[0074] For example, the electronic device (100) and / or the server (40) can detect the location of the user's face included in the captured image acquired through the camera and identify the user's facial expressions, such as the user's eyes, nose, and mouth included in the user's face. Various conventional methods can be used as a face region detection method. Specifically, a direct recognition method and a statistical method can be used. The direct recognition method creates rules using physical features such as the outline, skin color, and size of components of the face image or the distance between them, and compares, inspects, and measures according to the rules. The statistical method can detect the face region according to a pre-learned algorithm. That is, it is a method of converting unique features included in the input face into data and comparing and analyzing them with a large prepared database (shapes of faces and other objects). In particular, the face region can be detected according to a pre-learned algorithm, and methods such as a multi-layer perceptron (MLP) and a support vector machine (SVM) can be used.

[0075] For example, the electronic device (100) and / or the server (200) can obtain user conversation information using speech recognition technology. Speech recognition refers to a process in which a computer interprets a person's spoken language and converts the content into text data, and is also called STT (Speech-to-Text). A representative algorithm is the HMM (Hidden Markov Model), which statistically models the voices spoken by various speakers to construct an acoustic model and can construct a language model through corpus collection.

[0076] For example, a content report may include a scene-specific rating, calculated in timeline order, based on user reaction information and commentary information. For example, a content segment from the time a specific user reaction is identified to the time the reaction associated with that specific user reaction ends may be identified as a single scene.

[0077] In one example, if a user reaction is identified as corresponding to a preset emotion, commentary information related to the content can be obtained while the user reaction occurs. For example, the preset emotion may include at least one of a positive or negative emotion.

[0078] For example, when a user image is input, an emotion corresponding to the user reaction can be identified using an artificial intelligence model trained to classify emotional information.

[0079] For example, when a user's voice is input, an emotion corresponding to the user's reaction can be identified using an artificial intelligence model trained to classify emotional information.

[0080] According to one embodiment, the processor (130) may control the display (110) to display a user area including a user-captured image on one side of the content area. In one example, the user area may be a shared viewing user area including multiple user-captured images accessed through a shared viewing application. For example, the content area may be provided at the top of the screen and the user area at the bottom of the screen, but this is not limited thereto.

[0081] According to one embodiment, if the processor (130) identifies a user reaction as corresponding to a preset emotion, the processor (130) may control the display (110) to temporarily switch and display a user-captured image and content area including the user reaction. For example, the preset emotion may include at least one of a positive emotion and a negative emotion.

[0082] FIG. 2b is a drawing for explaining a detailed configuration of an electronic device according to one or more embodiments.

[0083] According to FIG. 2b, the electronic device (100') includes an output unit (110), a memory (120), one or more processors (130), a communication interface (140), a user interface (150), a sensor (160), a camera (170), a speaker (180), and a power supply unit (190). Among the configurations illustrated in FIG. 2b, a detailed description of configurations that overlap with those illustrated in FIG. 2a will be omitted.

[0084] The communication interface (140) includes a circuit and can communicate with an external device (server or user terminal). For example, the processor (130) can receive various data or information from an external device connected via the communication interface (140) and can also transmit various data or information to the external device.

[0085] The communication interface (140) may include at least one of a WiFi module, a Bluetooth module, a wireless communication module, an NFC module, and a UWB module (Ultra Wide Band). At this time, the wireless communication module may perform communication according to various communication standards such as IEEE, Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), 5G (5th Generation), etc.

[0086] In addition, the communication interface (140) may perform communication according to a communication method such as Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc., depending on the implementation example of the electronic device (100').

[0087] The user interface (150) may be implemented by devices such as buttons, touch pads, mice, and keyboards, or may be implemented by a touch projection surface, a remote control transmitter / receiver, etc. that can also perform the above-described display function and operation input function. The remote control transmitter / receiver may receive a remote control signal from an external remote control device or transmit a remote control signal through at least one of infrared communication, Bluetooth communication, and Wi-Fi communication.

[0088] The speaker (160) can output audio signals. In one example, the speaker (160) may include multiple speaker units (or amplifiers) capable of outputting multi-channels. For example, the speaker (180) may include an L channel speaker and an R channel speaker that reproduce the L channel and the R channel, respectively.

[0089] The camera (170) can be turned on and take pictures according to a preset event. The camera (170) can convert the captured image into an electrical signal and generate image data based on the converted signal. For example, the subject can be converted into an electrical image signal through a semiconductor optical element (CCD; Charge Coupled Device), and the converted image signal can be amplified and converted into a digital signal and then signal processed.

[0090] According to one embodiment, the processor (130) can identify a user's gesture, gaze, etc. based on a captured image acquired through a camera (170), and can identify whether a user command is input based on the identified gesture, gaze, etc.

[0091] In addition, the electronic device (100') may include a microphone, a sensor, etc., depending on the implementation example. The microphone is a configuration for receiving a user's voice or other sounds and converting them into audio data. However, according to another embodiment, the electronic device (100') may receive a user's voice input through an external device through a communication interface (140). The sensor may include various types of sensors, such as a distance sensor, an illuminance sensor, an image sensor, a touch sensor, a proximity sensor, an acceleration sensor, a geomagnetic sensor, a gyro sensor, a pressure sensor, a position sensor, etc.

[0092] Figure 3a is a block diagram showing the configuration of a server according to one embodiment of the present invention.

[0093] According to FIG. 3A, the server (200) includes a communication interface (210), a memory (220), and one or more processors (230). As an example, the electronic device (100) may be implemented as the server (40) illustrated in FIG. 1. The configuration of the communication interface (210), the memory (220), and the one or more processors (230) is identical / similar to the configuration of the communication interface (140), the memory (120), and the one or more processors (130) illustrated in FIGS. 2A and 2B, and thus a detailed description thereof will be omitted. For the convenience of explanation, the one or more processors (230) will be referred to as a processor (230) hereinafter.

[0094] According to one embodiment, the processor (130) can configure a DB for managing video chat users and chat rooms and store it in the memory (120).

[0095] FIG. 3b is a block diagram illustrating the operation of a server according to one embodiment of the present invention.

[0096] According to FIG. 3b, the processor (230) can provide a content viewing service using an analysis module (241), a screen layout dynamic management module (242), a group management module (243), and a connection management module (244). Here, each module can be implemented with at least one software, at least one hardware, and / or a combination thereof.

[0097] For example, at least one of the analysis module (241), the screen layout dynamic management module (242), the group management module (243), and the connection management module (244) may be implemented to utilize a predefined algorithm, a predefined formula, and / or a learned artificial intelligence model. The analysis module (241), the screen layout dynamic management module (242), the group management module (243), and the connection management module (244) may be included within the server (200), but may be distributed to at least one external device (e.g., the electronic device (100)) according to an example.

[0098] According to one example, the processor (230) can analyze real-time viewing situations based on content analysis, content recognition, content usage history, content surrounding information, etc. using the analysis module (241).

[0099] For example, the analysis module (241) may analyze real-time viewing situations based on at least one of the user's content usage history (e.g., history of sharing key interest information), commentary creation history (e.g., history of reactions occurring during viewing together), content recognition information (e.g., image recognition information), or surrounding environment information (e.g., viewer voice and facial expression information).

[0100] According to one example, the processor (230) can dynamically configure / manage a screen provided through the electronic device (100) using the screen layout dynamic management module (242).

[0101] For example, the screen layout dynamic management module (242) can perform operations related to the connected user area, additional information overlay to be displayed on my screen, selection and / or sharing of information to be shared, content area information, and provision of real-time reaction effects based on viewer reactions.

[0102] According to one example, the processor (230) can perform remote access user identification and image information processing operations using the group management module (243).

[0103] In one example, the processor (230) can create and manage a group of users invited to view together using the access management module (244).

[0104] FIG. 4 is a flowchart illustrating a method for controlling an electronic device according to one or more embodiments.

[0105] According to one embodiment, in operation 410, the electronic device (100) may display a UI screen including additional information related to the content while the content is being shared with the external device. In one example, the electronic device (100) may share a content area with the external device (100) by executing a shared viewing application.

[0106] For example, a UI screen containing additional information may be presented through a single-view screen. For example, the content area may be shared with the user of the external device, but the UI screen containing the additional information may not be shared.

[0107] In operation 420, the electronic device (100) can obtain a UI screen in which a plurality of additional pieces of information are rearranged based on user history information for a plurality of different types of additional pieces of information included in the UI screen.

[0108] In operation 430, the electronic device (100) may share a UI screen including a plurality of pieces of information rearranged according to a preset event with an external device. For example, the preset event may include an event in which a user command for sharing the UI screen is input.

[0109] Meanwhile, in Fig. 4, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0110] FIG. 5 is a drawing illustrating an example of a UI screen according to one or more embodiments.

[0111] FIG. 5 is a drawing for explaining a first screen provided to a user when a view-to-view application is executed according to one embodiment.

[0112] According to an example, as illustrated on the left side of FIG. 5, the first screen (510) may include a content area (511) and a user area for viewing together (512). For example, the first screen (510) may include a GUI (or indicator) (513) for providing a UI screen including additional information. For example, the content area (511) and the user area for viewing together (512) may be provided to all users who have accessed the application for viewing together.

[0113] For example, the first screen (510) may include a theme image as a background area. For example, the background areas of the content area (511) and the user viewing area (512) may include various types of theme images. For example, the theme images may include various themes such as a theater theme, a nature theme, and a city theme. In some cases, the theme images may be theme images related to the content type provided in the content area (511).

[0114] In one example, a second screen may be provided containing information that is visible only to the user of the electronic device (100) according to a user command.

[0115] According to an example, as illustrated on the right side of FIG. 5, the second screen (520) may additionally provide a UI screen (514) including additional information. For example, the UI screen (514) including additional information may be additionally provided in response to a user command of clicking and swiping the GUI (513) for providing the UI screen to the left. In this case, the UI screen (514) may be provided in a viewable form only to the user of the electronic device (100) until a preset event occurs.

[0116] For example, the UI screen (514) may be provided in a form that is overlaid on the content area (511) and the user viewing area (512) while maintaining the sizes of the content area (511) and the user viewing area (512).

[0117] For example, the electronic device (100) may output the content area (511) and the user area for viewing together (512) as a first display layer, and output the UI screen (514) as a second display layer on the front of the first display. According to an example, the electronic device (100) may obtain an output image by alpha blending the first display layer and the second display layer. Here, alpha blending refers to a method of mixing the background RGB value and the RGB value above it by assigning a new value called A (Alpha) to the color value RGB in order to produce a transparent effect when overlaying another image on top of the image.

[0118] For example, the Alpha value is divided into 0~255 values ​​or 0.0~1.0 values, and 0 can mean completely transparent, and the opposite, 255 (or the highest value such as 1.0) can mean fully opaque. Or, 0 can mean completely opaque, and the opposite, 255 (or the highest value such as 1.0) can mean fully transparent. For example, when 8 bits are allocated to the Alpha value and can express values ​​from 0 to 255, the larger the value, the higher the ratio of the corresponding pixel, and the lower the value, the lower the ratio. For example, by increasing the transparency of the area corresponding to the content area (511) and the user area to view together (512), the area other than the UI screen (514) can be provided as the original image.

[0119] In another example, the UI screen (514) may be provided in the free space created as the size of the content area (511) and the view user area (512) are reduced.

[0120] For example, a UI screen (514) including additional information may include multiple types of UI information. For example, as illustrated in FIG. 5 , the UI screen (514) may include casting information (514-1), review rating information (514-2), director information (514-3), and a more screen (514-4).

[0121] Meanwhile, a UI screen (514) including additional information may disappear from the screen according to a user command to right-click and swipe the GUI (513) to provide a UI screen.

[0122] FIG. 6 is a flowchart illustrating a method for rearranging additional information according to one or more embodiments.

[0123] In one embodiment, the rearrangement of additional information may be performed by the server (200). However, in some cases, at least some of the operations may be performed by the electronic device (100). For convenience of explanation, the following description assumes that the rearrangement of additional information is performed by the server (200).

[0124] According to FIG. 6, in operation 610, the server (200) can calculate a score corresponding to each of the plurality of additional information based on the number of different types of activities for each of the plurality of additional information.

[0125] In operation 620, the server (200) can identify the priority of the plurality of additional information based on the score of each of the plurality of additional information.

[0126] In operation 630, the server (200) may rearrange the plurality of pieces of additional information on the UI screen based on the priorities of the plurality of pieces of additional information. In this case, the electronic device (100) may receive and display the UI screen with the rearranged additional information from the server (200).

[0127] Meanwhile, in Fig. 6, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0128] FIGS. 7A and 7B are drawings illustrating examples of UI screens according to one or more embodiments.

[0129] According to an example, as illustrated in FIG. 7a, the single-view screen (520) may include a content area (511), a user area (512), and a UI screen (514) including additional information.

[0130] For example, as illustrated in FIG. 7a, the UI screen (514) may include casting information (514-1), review rating information (514-2), director information (514-3), and a more screen (514-4).

[0131] According to an example, when a specific sub-area included in the UI screen (514), for example, casting information (514-1), is selected by the user as illustrated in FIG. 7a, the selected sub-area may be displayed in an enlarged manner (514-2).

[0132] According to an example, a specific sub-area included in the UI screen (514), for example, the supervisory information area (514-3), as illustrated in FIG. 7a, may be displayed as a hidden area after scrolling.

[0133] For example, when the More screen (514-4) is selected by the user, a sub-area (515) containing additional information, such as ending information, may be additionally displayed. For example, the microphone may be temporarily turned off when searching for additional information (or new information). For example, if an additional sub-area is provided, the user may additionally search for new information through text search, button hold, voice search, etc.

[0134] According to an example, as illustrated in the upper part of FIG. 7B, the viewing screen (710) may include a content area (711) and a viewing user area (712). According to an example, as illustrated in the upper part of FIG. 7B, when a notification displayed on the user's image (712-1) of the electronic device (100) is selected in the viewing user area (712), information (721) to be shared with other users may be displayed on the viewing screen (720), as illustrated in the lower part of FIG. 7B. For example, the user may say "How about this information?" and select a notification displayed on his / her image (721-2) to share desired information with other users.

[0135] FIGS. 8A to 8E are drawings for explaining a method of rearranging additional information according to one or more embodiments.

[0136] According to one embodiment, the server (200) may acquire key interest information based on content recognition and rearrange the key interest information based on user history information. In another example, the server (200) may acquire a key interest information profile by generating a Similarity Matrix based on keywords and metadata based on content recognition, such as keyword extraction.

[0137] According to an example, as illustrated in FIG. 8a, the server (200) can construct a Simliarity Matrix (or Scene Similarity Matrix) of metadata for each keyword based on the content recognition result. For example, the score included in the Simliarity Matrix may be a score normalized from 0 to 1. For example, as illustrated in FIG. 8a, a Collaborative Filtering (CF) algorithm such as TFID may be used as the basic matrix of the Simliarity Matrix, but is not limited thereto. The CF algorithm is an algorithm based on the assumption that users who like the same content will have similar content tastes, and if 'user a' likes content A and B and 'user b' likes content B and C, C may be recommended to user a and A to user b. Machine learning technology may be applied so that the recommendation accuracy increases as the number of users participating in content evaluation increases and data accumulates.

[0138] For example, as illustrated in FIG. 8b, when the server (200) acquires an interest information profile based on the content recognition result, the server (200) can rearrange the order of the interest information profile according to the user search. For example, the server (200) configures the user's query corresponding to the user search in the form of "content title" (prefix) + "search word", and if the field corresponding to the query result for the user search does not exist in the MetaData DB, after providing the web crawling result, the server can add the information selected and viewed by the user to the MetaData DB field. For example, the DB Field name can use the word with the highest frequency, and the existing Similarity Matrix can be updated by the keyword extracted when the user directly searches.

[0139] In one example, as illustrated in FIG. 8c, the server (200) can update an interest information profile based on user history. For example, the user history information may include activities and / or frequency of interest in the interest information. For example, user activities may include at least one of viewing, sharing, and new searches.

[0140] According to an example, as illustrated in FIG. 8d, the server (200) can calculate weights for interest information Profiles based on user history. For example, Scene Profile Score (Weight Sum X Similarity) can be calculated after adding weight ratios (Click X Weight). For example, the weights can be calculated as Share xi, View x (i+1), and New Search (i+2). For example, it can be calculated as Share x 1, View x 2, Search x 3. For example, since the purpose is to place the item to be shared at the front, a greater weight can be given to the activity of interest before sharing. Here, View can mean an activity of selecting to view information provided by the system, Sharing can mean an activity of selecting a share button to share with others, and Search can mean an activity of selecting a search button while performing a text-based search or selecting a Voice Assistant button for a voice-based search.

[0141] For example, as illustrated in FIG. 8e, the server (200) can rearrange the interest information layout based on the calculated weights for the interest information Profiles. For example, if the Scene Profile Score is tied, a relatively higher priority can be given to the Profile added through search. If this is also tied, the interest information Profiles can be arranged in the order in which they were first added to the Profile. For example, the order of addition can be identified based on the Time Stamp added to the MetaData DB.

[0142] FIG. 9 is a flowchart illustrating a method for providing a content report according to one or more embodiments.

[0143] According to FIG. 9, in operation 910, the electronic device (100) can identify user reactions while content is being shared with an external device. For example, various user reactions, such as user gestures, user voice, and user input text, can be identified through various sensors, such as a camera, microphone, and touch sensor.

[0144] In operation 920, the server (200) can obtain information about user reactions and commentary information related to content while the user reactions occur.

[0145] In operation 930, when viewing of content is finished (S930:Y), the electronic device (100) may provide a content report including information about user reaction and commentary information.

[0146] Meanwhile, in Fig. 9, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0147] FIG. 10 is a flowchart detailing a method for providing a content report according to one or more embodiments.

[0148] According to one embodiment, each step of the content report providing method illustrated in FIG. 10 may be performed through an electronic device (100) and / or a server (200), and below, each step will be described as being performed by the electronic device (100) and / or the server (200) without distinguishing the subject of each step.

[0149] According to FIG. 10, in operation 1005, the electronic device (100) can execute a co-viewing application according to a user command.

[0150] In operation 1010, the electronic device (100) may collect environmental information while the viewing application is running. For example, the environmental information may include user video, user voice, conversation content, etc.

[0151] In operation 1015, the electronic device (100) and / or the server (200) may identify whether a key event has occurred. For example, the key event may include a user reaction.

[0152] In operation 1015, if the electronic device (100) and / or the server (200) identifies that a major event has occurred (S1015:Y), in operation 1020, the electronic device (100) and / or the server (200) can dynamically generate a view-along reaction effect.

[0153] For example, the electronic device (100) and / or the server (200) may classify emotions based on user reactions. For example, the electronic device (100) and / or the server (200) may classify emotions based on user reactions as “positive” or “negative.”

[0154] For example, when a user reaction occurs, the electronic device (100) and / or the server (200) can switch and display the content area and the viewer area.

[0155] In one example, the electronic device (100) and / or server (200) may update the commentary event history based on the user reaction when a user reaction occurs.

[0156] In one example, the electronic device (100) and / or server (200) can add / edit commentary content per scene in timeline order.

[0157] In one example, the electronic device (100) and / or server (200) may calculate a rating per scene in timeline order.

[0158] In operation 1025, the electronic device (100) and / or the server (200) may collect commentary content elements when a reaction occurs. For example, the electronic device (100) and / or the server (200) may collect screen captures, conversation content, user responses, etc. when a reaction occurs.

[0159] In operation 1030, when a major event occurs (S1030:Y), the electronic device (100) and / or the server (200) may collect commentary content elements at the time of reaction occurrence in operation 1035. For example, the major event may include an interest information sharing event. For example, the commentary content elements may include screen captures, conversation content, shared information, user reactions, etc. at the time of interest information sharing. For example, the commentary content elements may include screen captures, conversation content, shared information, user reactions, etc. at the time of interest information sharing related to content properties. For example, the content properties may be various properties such as the content itself, content episode, detailed genre (e.g., baseball, real variety), broadcast time, characters, characters, items, emotions, videos, background images, sounds, scenes, etc.

[0160] In one example, the electronic device (100) and / or server (200) may update the commentary event history based on the sharing of interest information.

[0161] In one example, the electronic device (100) and / or server (200) may add / edit commentary content per scene in timeline order based on shared interest information.

[0162] In one example, the electronic device (100) and / or server (200) may calculate a rating per scene in timeline order based on shared interest information.

[0163] In operation 1040, the electronic device (100) and / or the server (200) may update the user history. For example, the user history may include information about the sharer of the information of interest.

[0164] In operation 1045, the electronic device (100) and / or the server (200) may rearrange and provide additional information provided on the UI screen.

[0165] In operation 1050, when viewing of the content is finished, in operation 1055, the electronic device (100) and / or the server (200) may provide co-viewing viewing commentary information for the content.

[0166] In one example, the electronic device (100) and / or server (200) may calculate an average commentary rating for the entire content upon completion of viewing, and mark the viewing ratings of users who viewed the content together (e.g., “our viewing ratings”) to make the content public.

[0167] Meanwhile, in Fig. 10, the order is mapped for all steps for convenience of explanation, but it is of course not necessarily limited to the order of steps that are not related to the order or can be performed in parallel.

[0168] FIG. 11 is a diagram illustrating a method for generating viewing commentary information according to one or more embodiments.

[0169] In one embodiment, commentary information may be generated per scene in timeline order while content viewing is in progress.

[0170] According to Figure 11, a total of three events can occur while viewing content. In this case, a scene can be identified at the time of each event occurrence, and commentary information for each scene can be generated.

[0171] For example, when an additional information sharing event (1111) occurs in scene#1, commentary information including shared information (1112) and each user's reaction information (1113) regarding the shared information (1112) may be generated. For example, scene#1 may be identified from the point in time when a share button is selected and interest information is shared in a shared video. For example, commentary information corresponding to scene#1 may include a snapshot of the shared video at which the event occurred, a snapshot of the user's face image / photo shared at the time of sharing and the information shared, a user's face image / photo mentioned about the user shared at the time of sharing, a voice text record, etc.

[0172] For example, when a major user reaction occurs in scene #2, the user video and content video may be switched and provided (1114), and commentary information including a shared viewing video snapshot (1114) and information about the major user reaction (1115) may be generated. For example, the major user reaction may include a reaction expressing a specific emotion such as joy, anger, sleepiness, or jeer.

[0173] For example, the electronic device (100) and / or the server (200) may analyze the video captured by the camera and / or the voice received through the microphone during the shared viewing session to classify the reaction emotion as a positive or negative emotion. For example, the electronic device (100) and / or the server (200) may analyze the viewer's facial expression and / or the conversation content included in the voice included in the captured video. For example, positive emotions may include joy (including laughter, cheering, and cheering sounds). For example, negative emotions may include anger (for example, an angry expression of dissatisfaction, a swearing conversation, and a jeering conversation with another viewer), boredom (for example, a sleepy face, a jeering sound or conversation), etc.

[0174] For example, the electronic device (100) and / or the server (200) may use a trained artificial intelligence model to classify the emotion of a reaction as positive or negative. For example, when an image is input, the artificial intelligence model may be trained to classify the user's emotion contained in the image and output a specific type of emotional information. For example, when a voice is input, the artificial intelligence model may be trained to classify the user's emotion contained in the voice and output a specific type of emotional information.

[0175] Here, the learning of an artificial intelligence model means that a basic artificial intelligence model (e.g., an artificial intelligence model including any random parameters) is trained using a learning algorithm using a large number of training data, thereby creating a predefined set of operation rules or an artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed through a separate server and / or system, but is not limited thereto, and may also be performed in a cooking device. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0176] Here, the artificial intelligence model can be implemented as, for example, a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Restricted Boltzmann Machine (RBM), a Deep Belief Network (DBN), a Bidirectional Recurrent Deep Neural Network (BRDNN), or a Deep Q-Network, but is not limited thereto.

[0177] For example, in scene #3, if a significant user reaction occurs from at least one viewer among multiple viewers, the content area and viewer area can be switched to provide a reaction highlighting effect. In this case, not only is the display location switched, but the user reaction screen is also enlarged, further emphasizing the user reaction.

[0178] In one embodiment, the viewing commentary information for the viewing together may include "Our Viewing Rating."

[0179] For example, the server (200) can calculate a rating based on the positive / negative sentiment classification each time a major reaction event occurs in a scene. For example, a plus point of 1 point can be calculated for each user who shared information, and a plus point of 1 point can be calculated for each user who reacted positively. For example, even if a user reacted negatively, a minus point of 0 points can be calculated. For example, a negative reaction can be assigned a score of 0 points, thereby affecting the overall rating average.

[0180] For example, if an event occurs for N scenes per piece of content viewed, our rating per scene where the event occurs can be calculated based on the "(Plus Point + Minus Point) X (5-star rating) conversion factor." For example, the conversion factor can be calculated as = 5 points / total number of connected viewers N. For example, if the number of connected viewers is 4, the conversion factor can be calculated as = 5 / 4 = 1.25.

[0181] For example, in the example illustrated in FIG. 11, the rating of scene #1 can be calculated as plus point 1 X 1.25 (conversion factor) = 1.25, the rating of scene #2 can be calculated as plus point 4 X 1.25 (conversion factor) = 5, and the rating of scene #3 can be calculated as minus point 0 X 1.25 (conversion factor) = 0. In this case, the rating for the entire content can be calculated as the average value after adding up the ratings for each scene. For example, in the example illustrated in FIG. 11, the rating for the entire content can be calculated as (1.25 + 5 + 0) / 3 = 2.08 (rounded to two decimal places).

[0182] FIG. 12 is a diagram illustrating a method for providing viewing commentary information according to one or more embodiments.

[0183] According to one embodiment, when content viewing is completed, as illustrated in FIG. 12, co-viewing commentary information generated for the content may be provided. For example, the co-viewing commentary information may be provided in the form of a commentary document (1211) on a screen (1210) or in the form of a commentary video (e.g., an image-based automatic video editing) (1212).

[0184] FIG. 13 is a diagram detailing a method for providing a user reaction highlighting effect according to one or more embodiments.

[0185] According to the right drawing of FIG. 13, if the electronic device (100) and / or the server (200) identify that the user reaction corresponds to a preset emotion, the electronic device (100) and / or the server (200) can temporarily switch and display the user-captured image (1312) and content area (1311) corresponding to the user reaction included in the screen (1310). For example, the preset emotion may include at least one of a positive emotion or a negative emotion. In this case, not only the display location is switched, but the user reaction screen is also enlarged and provided, so that the user reaction can be further emphasized.

[0186] According to the left drawing of FIG. 13, the electronic device (100) and / or the server (200) can switch and provide the user-captured video (1312) and the content area (1311), and then additionally provide the remaining user-captured videos (133, 1314) included in the viewing group to the content area. For example, if the user included in the user-captured video (1312) is dozing, the users included in the remaining user-captured videos (133, 1314) can show reactions that tease or jeer at the target user. In this case, the reaction emphasis effect can be provided through video switching.

[0187] According to the various embodiments described above, video chat allows viewers to share key information related to their viewing experience while watching content together. Furthermore, key reactions from viewers can be detected and shared with co-viewers after the viewing session concludes. This enhances the UX experience associated with co-viewing.

[0188] Meanwhile, the methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device and / or server.

[0189] Additionally, the various embodiments of the present disclosure described above can also be performed through an embedded server provided in an electronic device, or an external server of the electronic device.

[0190] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0191] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0192] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0193] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In electronic devices, display; A memory that stores one or more instructions; and comprising one or more processors; The one or more processors, by executing the one or more instructions, Controlling the display to display a UI screen containing additional information related to said content while the content is being shared with an external device; Obtain a UI screen in which the plurality of additional information is rearranged based on user history information for the plurality of additional information of different types included in the UI screen, An electronic device that shares a UI screen including the plurality of rearranged pieces of information with an external device according to a preset event.

2. In paragraph 1, The above multiple additional information, An electronic device in which a score corresponding to each of the plurality of additional information is calculated based on the number of different types of activities for each of the plurality of additional information, and the plurality of additional information is rearranged on the UI screen based on the priority of the plurality of additional information based on the score of each of the plurality of additional information.

3. In paragraph 2, The above different types of activities include sharing, browsing and searching, The one or more processors, by executing the one or more instructions, An electronic device that calculates a score corresponding to each of the plurality of additional information by assigning different weights to the sharing, viewing and searching.

4. In paragraph 1, The one or more processors, by executing the one or more instructions, When viewing of the above content is finished, a content report is provided including information on user reactions and commentary information. The above commentary information is, An electronic device, wherein a user reaction is identified while the content is shared with the external device, and the identified user reaction is acquired while the identified user reaction occurs.

5. In paragraph 4, The above user reaction is, Contains at least one of a voice reaction, a chat reaction, a facial expression reaction, or a gesture reaction; The above commentary information is, An electronic device comprising at least one of screen capture information, conversation information, or shared information.

6. In paragraph 4, The above content report is, Includes a scene-by-scene rating calculated in timeline order based on the information about the above user reactions and the above commentary information, An electronic device, wherein a content section from the time a specific user reaction is identified to the time the reaction related to said specific user reaction ends is identified as one scene.

7. In paragraph 4, The one or more processors, by executing the one or more instructions, When the above user reaction is identified as corresponding to a preset emotion, commentary information related to the content is obtained while the user reaction occurs. The above preset emotions are, An electronic device comprising at least one positive or negative emotion.

8. In paragraph 7, The one or more processors, by executing the one or more instructions, Controlling the display to display a user-captured image on one side of the content area; An electronic device that controls the display to temporarily switch and display the user-captured image and the content area including the user reaction when the user reaction is identified as corresponding to the preset emotion.

9. In paragraph 1, The above different types of additional information are: Includes at least one of the following: casting, plot, content background, producer, director, or review ratings; The above multiple additional information, An electronic device, wherein a matrix representing scores for the plurality of additional information is obtained based on a preset algorithm, and the obtained matrix is ​​updated based on user history information for the plurality of additional information, and the matrix is ​​rearranged on the UI screen based on the updated matrix.

10. On the server, communication interface; A memory that stores one or more instructions; and comprising one or more processors; The one or more processors, by executing the one or more instructions, Providing a UI screen including additional information related to the content to one of the plurality of external devices based on a preset event that occurs while the plurality of external devices are sharing the content; Obtain a UI screen in which the plurality of additional information is rearranged based on user history information for the plurality of additional information of different types included in the UI screen, Sharing a UI screen containing the above rearranged plurality of pieces of information with the remaining devices among the plurality of external devices according to a preset event; The above multiple additional information, A server, wherein a score corresponding to each of the plurality of additional information is calculated based on the number of different types of activities for each of the plurality of additional information, and the plurality of additional information is rearranged on the UI screen based on the priority of the plurality of additional information based on the score of each of the plurality of additional information.

11. In a method for controlling an electronic device, A step of displaying a UI screen including additional information related to said content while the content is shared with an external device; A step of obtaining a UI screen in which the plurality of additional information are rearranged based on user history information for a plurality of different types of additional information included in the UI screen; and A control method, comprising: a step of sharing a UI screen including the plurality of rearranged pieces of information with the external device according to a preset event.

12. In paragraph 11, The above multiple additional information, An electronic device in which a score corresponding to each of the plurality of additional information is calculated based on the number of different types of activities for each of the plurality of additional information, and the plurality of additional information is rearranged on the UI screen based on the priority of the plurality of additional information based on the score of each of the plurality of additional information.

13. In paragraph 12, The above different types of activities include sharing, browsing and searching, The one or more processors, by executing the one or more instructions, An electronic device that calculates a score corresponding to each of the plurality of additional information by assigning different weights to the sharing, viewing and searching.

14. In paragraph 11, The one or more processors, by executing the one or more instructions, When viewing of the above content is finished, a content report is provided including information on user reactions and commentary information. The above commentary information is, A control method, wherein a user reaction is identified while the above content is shared with the external device, and the identified user reaction is acquired while the identified user reaction occurs.

15. A non-transitory computer-readable medium storing computer instructions that, when executed by a processor of an electronic device, cause the electronic device to perform an operation, The above actions are, A step of displaying a UI screen including additional information related to said content while the content is shared with an external device; A step of obtaining a UI screen in which the plurality of additional information are rearranged based on user history information for a plurality of different types of additional information included in the UI screen; and A non-transitory computer-readable medium comprising: a step of sharing a UI screen including the plurality of rearranged pieces of information with an external device according to a preset event;

Images