Electronic device and information processing methods thereof

Abstract

An electronic device is disclosed. The present device and method provide, when emoticons input by multiple users watching the same content are received through a communication unit, an event corresponding to a watching atmosphere identified on the basis of label information of the received emoticons to the multiple users, and update the label information according to the degree of participation in the event. Accordingly, the accuracy of a context recognition technology of real-time content increases, and a technology for collecting and analyzing user input data can be implemented more easily than before.

Description

Electronic devices and information processing methods thereof

[0001] The present invention relates to an electronic device and method for processing information to recognize the viewing atmosphere of content.

[0002] In content like live sports matches and competition programs, viewer participation is increasing, with viewers cheering on the action or voting in real time. Unlike traditional recorded video, this type of content is gaining immense popularity because viewers can influence the ending of the broadcast by intervening in real time.

[0003] For content providers offering this type of content, it's crucial to understand the viewer's mood and take appropriate action accordingly. For example, if viewers are bored with the content currently playing, the nature of real-time content allows for immediate changes to the content.

[0004] Therefore, there was a need to accurately recognize the viewing atmosphere, but in conventional technologies, it was difficult to accurately grasp the viewing atmosphere of real-time content.

[0005] According to at least one embodiment of the present disclosure, an electronic device includes a memory, a communication unit, and a processor.

[0006] The processor controls to provide an event related to an atmosphere corresponding to label information for each emoticon input by multiple users viewing the same content received through the communication unit, and updates the label information according to the degree of participation in the event.

[0007] In addition, in a method for processing information using an electronic device, the method includes a step of controlling to provide an event corresponding to a viewing mood for the content identified based on label information of emoticons input by multiple users viewing the same content, and a step of updating label information previously stored for each of the multiple emoticons according to the degree of participation in the event.

[0008] In addition, in a non-transitory computer-readable recording medium storing one or more instructions executed by a control unit of an electronic device to cause the electronic device to perform an operation, the operation includes a step of providing an event corresponding to a viewing mood for the content identified based on label information of emoticons input by a plurality of users viewing the same content, and a step of updating label information previously stored for each of the plurality of emoticons according to the degree of participation in the event.

[0009] FIG. 1 is a drawing for explaining the operation of an electronic device according to at least one embodiment of the present disclosure.

[0010] FIG. 2 is a block diagram illustrating a configuration of an electronic device according to at least one embodiment of the present disclosure.

[0011] FIGS. 3 and 4 are diagrams showing examples of emoticons and their label information used in an electronic device according to at least one embodiment of the present disclosure.

[0012] FIG. 5 is a drawing illustrating an example of a new emoticon according to at least one embodiment of the present disclosure.

[0013] FIG. 6 and FIG. 7 are drawings for explaining an example of a method for inputting an emoticon in one external device (200).

[0014] Figure 8 shows an example of a chat window in which an emoticon selected by a user is displayed.

[0015] Figures 9 to 12 are drawings for explaining various event providing methods of an electronic device.

[0016] FIG. 13 is a block diagram showing the configuration of a display device according to at least one embodiment of the present disclosure.

[0017] FIG. 14 is a flowchart illustrating an information processing method of an electronic device according to at least one embodiment of the present disclosure.

[0018] FIG. 15 is a flowchart illustrating an information processing method according to another embodiment of the present disclosure.

[0019] FIG. 16 is a flowchart illustrating an information processing method according to another embodiment of the present disclosure.

[0020] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0021] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0022] The embodiments of the present disclosure may be modified and have various embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the scope of the disclosed concepts and techniques. In describing the embodiments, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the main point.

[0023] Terms such as "first" and "second" may be used to describe various components, but the components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0024] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprises" or "consists of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0025] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted for clarity of description of the present disclosure, and similar parts are designated with similar reference numerals throughout the specification.

[0026] FIG. 1 is a drawing for explaining the operation of an electronic device according to at least one embodiment of the present disclosure.

[0027] According to FIG. 1, the electronic device (100) can communicate with a plurality of external devices (200-1 to 200-n). The electronic device (100) can receive various types of information from the external devices (200-1 to 200-n).

[0028] The electronic device (100) may be a device for providing a chat service or emoticon usage service available from a plurality of external devices (200-1 to 200-n). Specifically, the electronic device (100) may be implemented as a server device, a cloud server device, etc., but is not limited thereto, and may be implemented as various devices such as a PC or a laptop PC. In the present disclosure, the electronic device (100) is illustrated and described as if it were a single server device, but the electronic device (100) may also be implemented as a plurality of devices.

[0029] The plurality of external devices (200-1 to 200-n) may be electronic devices for providing content. Specifically, the plurality of external devices (200-1 to 200-n) may be composed of a TV, a set-top box, a one-connect box, a PC, a laptop PC, a smart monitor, a mobile phone, a tablet PC, a kiosk, an electronic whiteboard, and other home appliances. The plurality of external devices (200-1 to 200-n) may be display devices directly equipped with a display, or electronic devices connected to an external display device. Hereinafter, the expression “providing content” may be understood to mean directly displaying content when implemented as a display device, and when implemented as an electronic device that does not include a display device and is connected to an external display device, it may be understood as an operation of transmitting content and a control signal for controlling the output of the content to the external display device.

[0030] Content can be provided from various content sources. For example, the electronic device (100) of FIG. 1 may directly provide content to a plurality of external devices (200-1 to 200-n), but is not limited thereto. The plurality of external devices (200-1 to 200-n) may receive content as external inputs such as an antenna port, a broadcast cable, the Internet, etc., or may receive content from various sources such as a multimedia player, PC, laptop PC, or console game machine connected to the external devices (200-1 to 200-n).

[0031] External devices (200-1 to 200-n) can download content in advance and play it for output, or stream it in real time and output it.

[0032] The electronic device (100) can receive input data when a user of each external device (200-1 to 200-n) inputs an emoticon or chat message while outputting content from each external device (200-1 to 200-n).

[0033] When the same content is being output from multiple external devices (200-1 to 200-n), the server device (100) can receive emoticon information entered by users of the external devices (200-1 to 200-n), i.e., viewers, information on content being viewed, text information entered while performing a chat function, etc.

[0034] Specifically, users of each external device (200-1 to 200-n) can input emoticons or chat messages to express their emotions while watching content. Emoticons refer to graphic objects that can express emotions. Emoticons can be referred to by various terms such as graphic objects, icons, thumbnail images, and mini images, but are referred to as emoticons in this disclosure. Users can express the emotions they feel while watching content using appropriate emoticons.

[0035] The method of inputting emoticons can be implemented in various ways depending on the characteristics of the external device (200-1 to 200-n).

[0036] For example, if the external device is a TV or a set-top box, the user can display an emoticon UI on the screen using the remote control of the TV or set-top box and select at least one of a plurality of emoticons displayed on the emoticon UI. Data for the emoticon UI may be stored in each external device or may be provided from the electronic device (100).

[0037] Emoticon selection can be done using the directional buttons, confirmation button, or number buttons on the remote control. Alternatively, if the user speaks identification information (e.g., emoticon ID, number, etc.) to identify an emoticon, the external device can input the spoken voice through a microphone on the external device, remote control, AI speaker, etc., and select an emoticon based on that spoken voice.

[0038] Emoticons input from each external device (200-1 to 200-n) can be displayed through the screen of each external device (200-1 to 200-n) or its connected device. In addition, information about the emoticons input from each external device (200-1 to 200-n) is transmitted to the electronic device (100). The electronic device (100) determines the viewing mood of the content being played in real time based on the emoticon information received from the plurality of external devices (200-1 to 200-n) and provides the user with an event that matches the determined viewing mood. The viewing mood can be an emotional state expressed by a plurality of users who are viewing the content. For example, the viewing mood can be analyzed into various states such as anger, sadness, boredom, joy, pleasure, fun, and disinterest. The viewing moods can be broadly classified into positive, negative, and neutral. Anger, sadness, boredom, etc. can be classified as negative states, joy, pleasure, fun, etc. can be classified as positive states, and disinterest, etc. can be classified as neutral states.

[0039] The electronic device (100) can identify a mood corresponding to label information for each emoticon input based on information about the emoticon input by the user or at least one other user viewing the same content as the electronic device (100). Specifically, emoticons used by users may be individually assigned label information. The label information may be information stored by predicting the usage intention of each emoticon. The label information may include emotion label information indicating at least one emotion among positive, negative, and neutral. For example, in the case of an emoticon with a smiling expression, label information having a positive value (e.g., +1) may be matched. Alternatively, in the case of an emoticon with a crying expression, label information having a negative value (e.g., -1) may be matched. In the case of an emoticon with a neutral expression, label information having a neutral value (e.g., 0) may be matched.

[0040] The method of generating label information and other examples are described in detail later based on other drawings.

[0041] The electronic device (100) can identify the viewing mood of the content, i.e., the viewing mood, based on the label information. For example, if there are multiple emoticons expressing positivity, the viewing mood of the currently displayed content can be identified as positive. Conversely, if there are multiple emoticons expressing negativity, the viewing mood can be identified as negative.

[0042] The electronic device (100) can provide events related to the identified mood. A mood-related event can be an event in which users with an emotional state identical to or similar to the identified mood can easily participate. Specifically, the event can be at least one of a game, quiz, message, vote, advertisement, cheering, or video playback that allows user participation. If the viewing mood is positive, the electronic device (100) can provide a quiz or other event in which users in a positive mood can easily participate. Conversely, if the viewing mood is negative, the electronic device (100) can provide events such as a message or cheering to alleviate negative feelings.

[0043] The electronic device (100) can induce user participation by providing guidance information, which is determined differently depending on the viewing atmosphere, along with an event. Specific examples of event provision will be described in detail later.

[0044] The electronic device (100) can provide an event related to the mood corresponding to the label information for each emoticon input by the user, and identify the user's participation information for the provided event. The electronic device (100) can update the label information for each emoticon based on the degree of participation. Specifically, if users who used positive emoticons participated in many events corresponding to a positive viewing mood, the electronic device (100) can determine that the used emoticons correspond to positive label information. Accordingly, the previous label information can be maintained. In this case, the electronic device (100) can increase the reliability of the emotional label information assigned to the emoticon. On the other hand, if users who used negative emoticons participated in many events corresponding to a positive viewing mood, the label information for the used negative emoticons can be updated to positive label information.

[0045] In this way, the electronic device (100) can continuously update and manage the label information of the emoticon. Accordingly, the viewing mood of the content can be accurately and efficiently identified.

[0046] Meanwhile, in order for the electronic device (100) to accurately recognize the mood of real-time content from the emotional label information of the input emoticon, it is advantageous when many people actively use emoticons while watching the content. The electronic device (100) can measure the reliability of the label information corresponding to each emoticon differently based on the number of users watching the content, the number of users using emoticons, the number of emoticons used, the frequency of use, etc. When many users use multiple emoticons over a certain period of time (for example, when 30 or more users use emoticons over 10 minutes), the electronic device (100) can measure the reliability of the viewing mood identified based on those emoticons to be high.

[0047] Traditionally, intelligent context awareness technology has been used to perform situational awareness through video. Intelligent context awareness technology empowers computers to understand and interpret their surroundings. Because it combines artificial intelligence and big data technologies and requires a heavy-duty system based on deep learning, it is difficult to use on electronic devices with poor hardware or software performance. Furthermore, because conventional intelligent context awareness technology only utilizes information recognizable through video, it lacks high accuracy for real-time content. In contrast, various embodiments of the present disclosure identify situations based on the immediate reactions of users watching real-time content, enabling easier and more accurate identification of viewing atmospheres compared to conventional technologies. Below, these various embodiments will be described in detail.

[0048] FIG. 2 is a block diagram illustrating a configuration of an electronic device according to at least one embodiment of the present disclosure.

[0049] According to FIG. 2, the electronic device (100) includes a communication unit (110), a processor (120), and a memory (130).

[0050] The communication unit (110) is configured to perform communication with multiple external devices. In an environment such as FIG. 1, the communication unit (110) can perform communication with at least one external device (200-1 to 200-n). Specifically, the communication unit (1100) can receive information about emoticons input by multiple users viewing the same content through their respective external devices, information about the content being viewed, and information about text input during the execution of a chat function.

[0051] The communication unit (110) can transmit and receive various signals and data with external devices through various wired and wireless communication methods such as Bluetooth, AP-based Wi-Fi (Wi-Fi, Wireless LAN network), Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, HDMI (High-Definition Multimedia Interface), USB (Universal Serial Bus), MHL (Mobile High-Definition Link), AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc.

[0052] The processor (120) is a configuration for controlling the overall operation of the electronic device (100).

[0053] The processor (120) may include one or more of a digital signal processor (DSP), a microprocessor, a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, or an artificial intelligence (AI) processor, or may be defined by the terms thereof. In addition, the processor (120) may be implemented as a system on chip (SoC) or large scale integration (LSI) having a built-in processing algorithm, or may be implemented in the form of a field programmable gate array (FPGA). The processor (120) may perform various functions by executing computer executable instructions stored in the memory (130).

[0054] The memory (130) is a configuration for storing various software, commands, control codes, and data required for the operation of the electronic device (100). The memory (130) may be implemented as at least one of various memories, such as DRAM (dynamic RAM), SRAM (static RAM), SDRAM (synchronous dynamic RAM), OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory, hard drive, or solid state drive (SSD).

[0055] The memory (130) may store not only data of the electronic device (100), but also data of each external device received through the communication unit (110). In particular, the memory (130) may store data on emoticons that can be provided to each external device, label information stored to match each emoticon, etc. In addition, depending on the embodiment, the memory (130) storing an artificial intelligence model may be used.

[0056] The processor (120) performs various operations using the memory (130). Specifically, the processor (120) can store various information, data, and signals transmitted via the communication unit (110) in the memory (130). The processor (120) can use the information stored in the memory (130) to identify the viewing atmosphere of the content and provide events at appropriate times.

[0057] In addition, the processor (120) can generate a control signal for controlling the operation of each of the plurality of external devices (200-1 to 200-n). The control signal can include various identification information such as an IP address or SSID (Session ID) corresponding to each of the plurality of external devices (200-1 to 200-n), various control information such as a digital control code corresponding to each of the plurality of external devices (200-1 to 200-n), etc. The control signal can be configured in various forms according to a communication standard method. The processor (120) transmits the generated control signal and data to the plurality of external devices (200-1 to 200-n) through the communication unit (110).

[0058] As described above, when the electronic device (100) supports an emoticon provision service or a chat service, the processor (120) can provide a chat window that can be used by multiple external devices (200-1 to 200-n) viewing the same content. The chat window can be a display area for displaying chat messages and emoticons. The chat window can be described in various ways, such as a chat area, a viewer area, a participation space, an additional display area, a related information display area, etc., but is described as a chat window in the present disclosure.

[0059] The electronic device (100) does not necessarily provide only one chat window, and may additionally provide a separate display area capable of displaying only emoticons. However, for convenience of explanation, the following description will assume that both emoticons and chat messages are displayed in a single display area, i.e., a chat window.

[0060] The processor (120) can transmit data regarding the chat window and a control signal for commanding the display thereof to each external device (200-1 to 200-n) so that a chat window including various emoticons or chat messages input from each external device (200-1 to 200-n) can be commonly displayed on multiple external devices (200-1 to 200-n). Accordingly, users using multiple external devices (200-1 to 200-n) can share their emotions with others through emoticons or chat messages.

[0061] As described above, input of emoticons can be performed through external devices (200-1 to 200-n). For input of emoticons, an emoticon UI including at least one emoticon can be provided by the external devices (200-1 to 200-n). Each external device (200-1 to 200-n) may directly store data for the emoticon UI, but is not necessarily limited thereto, and the electronic device (100) may also transmit data for the emoticon UI to each external device (200-1 to 200-n).

[0062] For example, the processor (120) may transmit data on a plurality of emoticons stored in the memory (130) and a control signal for displaying the emoticons on one side of the screen to each external device (200-1 to 200-n) through the communication unit (110). The data on the emoticons may include various information such as the size, shape, display location, and text of the emoticon.

[0063] When the total number of emoticons is too large to be displayed on a single screen, the processor (120) may sequentially provide data of a preset number of emoticons to each external device (200-1 to 200-n).

[0064] Alternatively, the processor (120) may select emoticons that are related to the content currently being output from each external device (200-1 to 200-n) and provide data of the selected emoticons to each external device (200-1 to 200-n). Specifically, when information about content being provided from multiple external devices (200-1 to 200-n) is received through the communication unit (110), the processor (120) selects an emoticon that matches the content based on the content information.

[0065] For example, when watching a baseball game, the processor (120) may select an emoticon containing a character wearing a baseball uniform, a baseball ball, a bat, or other objects or text related to baseball (e.g., home run, strikeout, etc.) from among a plurality of emoticons stored in the memory (130). The selected emoticons may become recommended icons recommended to the user. The processor (120) may transmit data regarding the recommended emoticons to each external device (200-1 to 200-n).

[0066] In this way, when an emoticon is input from each external device, the processor (120) can receive it through the communication unit (110).

[0067] When multiple users viewing the same content input emoticons through the communication unit (110), the processor (120) can identify the viewing mood based on the label information of the received emoticons. Specifically, when a certain number or more of emoticons are input, the processor (120) checks the label information stored in the memory (130) and identifies the ratio of emotional label information included in the label information of the input emoticons. As a result of the identification, if the positive label information is 80%, the negative label information is 10%, and the neutral label information is 10%, the processor (120) can identify the viewing mood as a positive mood.

[0068] The processor (120) can identify the viewing mood based on the maximum number of emotional label information among multiple emotional label information. For example, if positive label information accounts for 40%, negative label information accounts for 35%, and neutral label information accounts for 25%, the processor (120) can determine that the viewing mood is positive.

[0069] However, this is merely an example, and a minimum standard for the label information ratio for identifying viewing atmosphere may be set and used. The set minimum standard information may be stored in advance in the memory (130). For example, if the minimum standard is stored as 50%, the processor (120) will suspend viewing atmosphere determination if the positive label information, which accounts for the maximum ratio, falls below the minimum standard at 40%.

[0070] Once the viewing mood is identified, the processor (120) can provide multiple users with events corresponding to the identified viewing mood. As described above, the events may include at least one of various events, such as games, quizzes, phrases, votes, advertisements, cheering, and video playback. The processor (120) can select at least one event corresponding to the current viewing mood from among these events. For example, events such as games or quizzes can be provided to users who are losing interest in the content or feeling bored, thereby drawing attention to the content. Alternatively, if the users are identified as being in a negative state, the processor (120) can provide events such as phrases or cheering videos that can reverse the mood. Alternatively, if the users are identified as being in a positive state, such as excited or joyful, the processor (120) can provide events such as playing past highlight videos or conducting voting to select a winner or outstanding player, which can enhance the joy. The processor (120) can also provide events along with guidance information corresponding to the current viewing mood. The guidance information may include text or images. When providing guidance information together, the processor (120) can provide the same type of event in various viewing environments. For example, even for the same quiz event, in a negative environment, the processor (120) can provide the quiz event with comforting phrases that can soothe users' emotions, and in a positive environment, the processor (120) can provide the quiz event with cheering phrases that can further enhance users' mood. When an event to be provided to users is configured, the processor (120) can generate data for the event and a control signal to display the event in a chat window, and transmit the data to each external device (200-1 to 200-n) via the communication unit (110).

[0071] The processor (120) can determine the timing of providing an event based on various criteria. For example, if minimum standard information is stored in the memory (130) as described above, the processor (120) can provide an event when the maximum ratio of emotional label information satisfies the minimum standard information.

[0072] As another example, the processor (120) may provide events at preset time intervals. For example, the processor (120) may identify a viewing mood every 10 minutes and provide an event corresponding to that viewing mood. If the maximum emotional level information falls below the minimum standard at the time of viewing mood identification, the processor (120) may provide an event corresponding to the previously identified viewing mood.

[0073] As another example, the processor (120) may identify the viewing mood and provide an event each time a certain number of emoticons are input. For example, if the number is set to 50, the processor (120) may identify the viewing mood based on the emoticons input between each of the 50 emoticons and provide an event corresponding to the identified mood.

[0074] As another example, the processor (120) may provide an event if no emoticons are input for a certain period of time. In this case, the processor (120) may provide the event along with guidance information encouraging the use of emoticons.

[0075] In addition, the processor (120) may identify the viewing atmosphere and provide an event corresponding thereto when the operator of the electronic device (100) inputs an event provision command, when there is a change in the content being output (in the case of exercise content, when a break time or operation time arrives, when there is an intermission commercial time, when there is a time to tally the results of a contest content, etc.).

[0076] Each external device (200-1 to 200-n) can provide an event while performing a chat function based on the received control signal. After checking the event information, the user can decide whether to participate. If participation is selected, the external device (200-1 to 200-n) transmits a participation signal to the electronic device (100) to notify participation in the event. When the communication unit (110) receives the participation signal, it transmits it to the processor (120).

[0077] The processor (120) can identify the level of participation in an event based on the received participation signal. The level of participation may include information such as the number of users participating in the event. The processor (120) can identify the number of users participating in the event and their user information based on the number of external devices that transmitted the participation signal and the information thereon. In addition, the processor (120) can check information on emoticons or chat messages previously entered by users participating in the event from the memory (130).

[0078] The processor (120) compares the emotional label information among the label information of the emoticon used by the user with the emotional label information corresponding to the viewing mood. If the comparison results match, the processor (120) retains the label information of the emoticon. In this case, the processor (120) may also increase the reliability score of the label information of the corresponding emoticon.

[0079] On the other hand, if the comparison results are inconsistent, the processor (120) updates the label information stored in the memory (130). For example, if an emoticon representing joy is used in a sad mood, the processor (120) may change the emotional label information of the emoticon from a positive label to a negative label. For convenience of explanation, the label information initially stored in the present disclosure may be referred to as predicted label information, and the label information updated through an event may be referred to as confirmed label information. The confirmed label information may be periodically updated.

[0080] FIG. 3 and FIG. 4 are drawings showing examples of emoticons and their label information used in electronic devices.

[0081] The emoticon (300) of FIG. 3 includes an image (301) of a person trembling and text (302), and the emoticon (400) of FIG. 4 includes an image (401) of a person raising one hand and text (402). The texts (302, 402) may be displayed together around the image of the person (401).

[0082] According to FIGS. 3 and 4, label information (303, 403) matching each emoticon (300, 400) may include an ID, a timestamp, and emotion label information (Negative, Positive) for identifying the emoticon.

[0083] Although still emoticons are shown in FIGS. 3 and 4, each emoticon may be a dynamic emoticon with added motion. The processor (120) can convert emoticon images into dynamic emoticons using an animation model. In this case, each emoticon (300, 400) can move in a certain pattern on the emoticon UI provided to the user.

[0084] Users can select emoticons to express their emotions by looking at the shape, motion, and text of the emoticon.

[0085] As described above, the emotional labels in Figures 3 and 4 can be broadly categorized into "positive," "negative," and "neutral." Specifically, emotions corresponding to the "positive" label include "joy," "satisfaction," and "excitement," while emotions corresponding to the "negative" label include "sadness," "anger," and "disappointment." Emotions corresponding to the "neutral" label include "boring" and "indifference."

[0086] The processor (120) can identify the emoticon selected by the user based on identification information, such as an ID corresponding to the emoticon selected by the user. The processor (120) can estimate the viewing mood based on emotional label information matching the identification ID included in the identification information.

[0087] Emotion label information can be arbitrarily selected by the emoticon creator based on the emoticon's motion, shape, text, etc., and then matched to the emoticon. For example, the emoticon (300) in FIG. 3 can be matched with a "negative" emotion label because it contains an angry motion and text, and the emoticon (400) in FIG. 4 can be matched with a "positive" emotion label because it contains a powerful and positive motion and text. As described in the above section, such emotion label information can be updated through events, etc.

[0088] Meanwhile, new emoticons whose label information is not separately matched may be used. The new emoticons may be created in the electronic device (100) and provided to each external device (200-1 to 200-n), may be created directly in the external devices (200-1 to 200-n) and used, or may be created by a separate third party for emoticon production and provided to the electronic device (100) or external devices (200-1 to 200-n).

[0089] A new emoji may be, but is not limited to, a new emoji that has not been used before, and an emoji that does not have label information may also be referred to as a new emoji.

[0090] The creation of new emoticons can be performed in various ways. For example, the processor (120) can generate emoticons using an image-generating AI model. The user can input information about various characteristics of the character to be created, such as shape, color, size, and text, through a separate input method. The processor (120) can generate an input prompt containing the input information and input it into the image-generating AI model to obtain an emoticon. If a dynamic emoticon is to be created, the processor (120) can also generate a dynamic emoticon by adding motion information to the emoticon using an animating AI model.

[0091] However, this is only an example and is not necessarily limited thereto, and the processor (120) provides a graphic tool that allows the user to draw directly, and can also identify an image drawn by the user as an emoticon using the graphic tool.

[0092] Alternatively, the processor (120) may capture a photo input by the user and draw an emoticon along the boundaries of the captured image.

[0093] When creating a new emoticon, the creator may directly input label information for the new emoticon. However, this is not limited to the above, and the processor (120) may also generate label information based on the characteristics of the new emoticon.

[0094] Figure 5 illustrates an example of a new emoticon. According to Figure 5, the new emoticon (1000) includes an image (501) of a person raising both hands and crying and text (502).

[0095] When a new emoticon without label information is identified, the processor (120) can estimate label information based on at least one of the shape, color, movement, and text included in the new emoticon. In FIG. 5 , if the image (501) is in the form of raising both hands and crying, the processor (120) can estimate a negative emotion label. Conversely, if the text (502) is “Hurrah” as in FIG. 5 , the processor (120) can estimate a positive emotion label. Since people are likely to shed tears when they are both happy and sad, the processor (120) can prioritize the text and determine the emotion label. Accordingly, for FIG. 5 , emotion label information indicating “positive” can be generated.

[0096] In addition, the processor (120) can generate a new ID for a new emoticon based on the ID assigned to existing emoticons, add a timestamp indicating the time at which the label information for the new emoticon is generated, and generate label information for the new emoticon (500). The processor (120) matches the generated label information to the new emoticon (500) and stores it in the memory (130).

[0097] When users of a plurality of external devices (200-1 to 200-n) wish to use emoticons, the processor (120) can provide at least some of the stored emoticons to each of the external devices (200-1 to 200-n).

[0098] FIG. 6 and FIG. 7 are drawings for explaining an example of a method for inputting an emoticon in one external device (200).

[0099] According to FIG. 6, the external device (200) can display an emoticon UI (602) on an area of ​​the screen while outputting content (601) on the screen. Information about the emoticon UI (602) can be provided in real time from the electronic device (100), or can be generated based on data that the external device (200) stores on its own. The external device (200) can automatically generate and display the emoticon UI (602) according to the type of content selected by the user. For example, if the electronic device (100) has determined content that provides an emoticon service or a chat service, the emoticon UI (602) can be displayed when the content is output. However, the present invention is not limited thereto, and the external device (200) can also display the emoticon UI (602) according to a user command.

[0100] On the emoticon UI (602) of FIG. 6, emoticons selected based on various criteria, such as emoticons frequently used by the user, emoticons that match the content currently being displayed, and emoticons available to the user, may be displayed. Alternatively, all emoticons may be displayed. When the user selects an emoticon, the selected emoticon may be displayed in an area within the screen. FIG. 6 illustrates a case where selected emoticons (611, 612) are overlapped and displayed on the content screen (601). The external device (200) may also overlap and display emoticons input by other users on the content screen (601) as shown in FIG. 6, based on emoticon information provided from the electronic device (100).

[0101] The emoticon UI (602) of FIG. 6 is provided when the size of the content screen (601) is maximized, and according to various embodiments, the user can change the size of the emoticon UI. For example, the external device (200) may additionally display a transition menu (603) within the emoticon UI (602).

[0102] When a user selects a transition menu (603), the external device (200) can change the shape or size of the emoticon UI (602). In an embodiment where the electronic device (100) generates and provides an emoticon UI, the external device (200) can transmit the fact that the transition menu (603) has been selected to the electronic device (100) to notify the electronic device (100), and can also receive corresponding emoticon UI data from the electronic device (100) to display the changed emoticon UI.

[0103] Fig. 7 illustrates a case where a different type of emoticon UI is displayed. When a user selects the transition menu (603), the emoticon UI (602) of Fig. 6 may be changed to the emoticon UI (700) of Fig. 7. However, this is not limited thereto, and the emoticon UIs of Figs. 6 and 7 may be provided on separate devices. In this case, the transition menu display may be omitted.

[0104] According to FIG. 7, the external device (200) can display a content screen (601) and an emoticon UI (700), respectively. The content screen (601) of FIG. 7 is reduced in size compared to the content screen of FIG. 6, but the area covered by the emoticon UI is eliminated.

[0105] The emoticon UI (700) of FIG. 7 may include an emoticon display area (701) at the bottom of the content screen (601), a mood display area (702) at the upper right, an emoticon selection area (703), an area (704) for displaying frequently used emoticons, etc.

[0106] Emoticons input by multiple users are displayed in the emoticon display area (701). Emoticons may be displayed sequentially and changed according to the input time, but Fig. 7 illustrates a form in which the emoticons input by each user are displayed in each user's location in a state in which multiple users are displayed as if they are watching the same content. If the user of the external device (200) is USER 2, the emoticon selected by that user may be displayed in the USER 2 area within the emoticon display area (701).

[0107] The mood display area (702) may display a viewing mood determined based on the label information of emoticons entered by multiple users. Fig. 7 illustrates a state where emotional labels such as sadness and joy are displayed. However, if the content is a sports or competition program, the viewing mood may also include a cheering mood for a specific team, player, or participant.

[0108] The emoticon selection area (703) displays various emoticons provided by the electronic device (100) or external device (200). Users can select and input emoticons in this area (703). The emoticon selection area (703) may also display popular emoticons frequently used by other users as recommended emoticons. If an emoticon is particularly frequently used by users, a special mark, such as a star, may be added to the top of the area for selecting the emoticon.

[0109] The area (704) at the bottom right may display frequently used emoticons or recently used emoticons. Users may also register frequently used emoticons in the favorites menu. Although not separately illustrated in FIG. 7, a switching menu may also be displayed in the emoticon UI (700) of FIG. 7. In this case, when the user selects the switching menu, the external device (200) may switch to the emoticon UI of FIG. 6. However, this is merely an example, and if FIG. 6 and FIG. 7 are implemented as separate embodiments, the switching menu may be omitted.

[0110] In Figures 6 and 7, an emoticon UI that only displays emoticons is illustrated, but emoticons may also be displayed in a chat window, etc. Figure 8 shows an example of a chat window in which an emoticon selected by a user is displayed.

[0111] According to FIG. 8, an external device (200) can display a chat window (600) on one side of a content screen (601). The chat window can be an area where various texts, symbols, numbers, emoticons, etc. entered by various users are displayed.

[0112] When a chat participation request is transmitted from a plurality of connected external devices, the electronic device (100) can transmit data and control signals for displaying a chat window for the external devices. A user can input a chat participation request from his / her terminal device (i.e., external device) using a remote control or various input means.

[0113] For example, a user can request to participate in a chat by selecting the chat function menu on the settings menu provided by the external device.

[0114] Alternatively, a user may request to participate in a chat by selecting a chat participation menu temporarily displayed on one side of the content screen on an external device. Figure 8 illustrates an example of a chat window configuration provided by such a request.

[0115] According to FIG. 8, in the chat window (600), text (602, 605), emoticons (603, 606), etc. entered by each user are displayed in a manner that matches each user's ID (607, 604).

[0116] The processor (120) of the electronic device (100) can identify the viewing atmosphere based on various texts and emoticons uploaded to the chat window (600). For example, label information can be used as described above.

[0117] As another example, the processor (120) can recognize text as well as label information. The memory (130) can store a dictionary database that classifies and stores multiple words by emotional labels. The processor (120) can identify the viewing mood by searching the dictionary database for emotional labels corresponding to words contained in the text. In this case, the processor (120) may utilize an artificial intelligence model. The use of an artificial intelligence model will be further described in detail in the following section.

[0118] If one of the emoticons (603, 606) uploaded to the chat window (600) (e.g., 606) does not have label information stored in the memory (130), the processor (120) may treat the emoticon (606) as a new emoticon. In this case, as described in FIG. 5, the label information of the new emoticon may be generated by estimating an emotion label based on at least one of the shape, color, movement, and text included in the new emoticon.

[0119] Alternatively, the processor (120) may estimate the emotional label of a new emoticon based on the mood of the chat window in which the new emoticon is used. In the case of FIG. 8, if the frequency of texts corresponding to positive labels, such as "home run" and "finally," is high, the processor (120) may estimate that the viewing mood is positive and estimate a positive emotional level for the new emoticon. The processor (120) may match label information including the estimated positive emotional level to the new emoticon and store it in the memory (130).

[0120] Meanwhile, if a specific emoticon (e.g., 603) is used more than a preset number of times while performing a chat function, the processor (120) may compare the emotional label information assigned to the emoticon with the emotional information estimated by the text uploaded to the chat window (600). In the case of the emoticon (603) of FIG. 8, since it is an image of shedding tears, a negative emotional label may be matched. The processor (120) may update the emotional label information of the emoticon (603) matched with the negative emotional label when the emoticon (603) is used many times in a positive-mood chat window.

[0121] As described above, if the emotion label information has already been updated according to the degree of event participation, if there is a discrepancy between the updated label information and the emotion information estimated based on the text, the processor (120) can re-update the label information based on the estimated emotion information.

[0122] In this case, the processor (120) may store previously updated label information along with the updated label information without deleting it. Accordingly, multiple label information may be matched to a single emoticon. If multiple label information items are matched, the processor (120) may assign a weight to each label information item.

[0123] For example, if multiple users input emoticons with multiple emotion labels such as “positive,” “negative,” and “neutral,” the processor (120) analyzes the text uploaded while performing the chat function to predict the emotions of the users using the emoticons.

[0124] Based on the predicted emotional results, it is assumed that the weight for the “positive” label is set to 0.7, the weight for the “negative” label is set to 0.2, and the weight for the “neutral” label is set to 0.1. When the corresponding emoticon is used later, the processor (120) can determine the overall viewing atmosphere by considering the weights for each label.

[0125] For example, if a total of 10 emoticons are used by multiple users while performing a chat function, and 3 of them are emoticons with the aforementioned weights, and the label information of the remaining 7 emoticons is 2 “positive,” 3 “negative,” and 2 “neutral,” the processor (120) determines that the positive label is 2 + 2.1 = 4.1, the sum of the weights for “negative” is 3 + 0.6 = 3.6, and the sum of the weights for “neutral” is 2 + 0.3 = 2.3. Accordingly, the processor (120) can determine that the viewing mood is “positive” even if there are more emoticons with negative labels or they are the same as emoticons with positive labels.

[0126] There are various ways to match multiple labels to a single emoticon. For example, the emoticon creator could match various emotional labels based on the intended use of the emoticon. Specifically, a crying expression could be used for both sadness and happiness. In this case, the positive and negative labels could each be weighted to 0.5 to match them to the corresponding emoticon.

[0127] As another example, the processor (120) may directly generate a plurality of label information based on the mood of the chat window, match the information to emoticons, and store the information in the memory (130). For example, if one emoticon is used in both positive and negative chat windows, the processor (120) may match both positive and negative labels to one emoticon. In this case, the processor (120) may determine weights for the plurality of label information by considering the frequency or number of times the emoticon is used in each chat window. Specifically, if the emoticon is used 10 times in a positive chat window and 30 times in a negative chat window, the weight for the positive label information may be determined as 0.25, and the weight for the negative label information may be determined as 0.75. The processor (120) may match the weights with the label information and store the information in the memory (130).

[0128] Although the descriptions in FIGS. 6 through 8 assume that the external device is implemented as a display device equipped with a display, the present invention is not limited thereto. For example, if the external device is a device connected to an external display, the external device can configure a screen as shown in FIGS. 6 through 8 and provide it to the external display.

[0129] Figures 9 through 12 are diagrams illustrating various event provision methods for an electronic device. As described above, the screens of Figures 9 through 12 can also be displayed on an external device or an external display connected to an external device. The following description assumes that the external device directly displays the screens.

[0130] According to FIG. 9, the external device (200) can display a chat window (910) on one side of the content screen (601) and display event information (920) within the chat window (910). In FIG. 9, only the content screen (601) and the chat window (910) are illustrated, but in an actual product, an emoticon UI for inputting emoticons may be additionally displayed separately.

[0131] As described above, the processor (120) can provide a corresponding event based on the viewing mood determined by emoticons, etc. The processor (120) can control each external device to first display event information (920) to inquire about participation in the event. The event information (920) can be information to indicate which events can be provided. The event information (920) can be provided along with guidance information. The guidance information can be determined differently depending on the viewing mood.

[0132] FIG. 9 illustrates a situation where text, which is guidance information inquiring about game participation in a positive mood (i.e., Would you like to play a wave surfing game suitable for an exciting cheering mood?), is provided together with event information (i.e., a game). On the other hand, in a negative mood, the processor (120) may provide text, which is guidance information inquiring about game participation in a negative mood (e.g., Would you like to play a wave surfing game to freshen up the mood?), together with event information. That is, the processor (120) may configure event information by providing different guidance information for the same type of event. However, the present invention is not limited thereto, and the processor (120) may also select and provide different types of events depending on the viewing mood.

[0133] Figures 10 and 11 are diagrams showing examples of further event information.

[0134] According to FIGS. 10 and 11, event information (930, 940) may be displayed within a chat window (910) along with text or emoticons (911, 912) entered by various users. FIG. 10 illustrates event information (930) provided in a negative atmosphere. FIG. 11 illustrates event information (940) provided in a boring atmosphere.

[0135] As described above, the processor (120) may identify the viewing mood based on the maximum number of emotion label information among the emotion label information of emoticons input by multiple users, or may identify the viewing mood based on the content of text input by each user.

[0136] Users can decide whether or not to participate in an event by selecting a menu (i.e., yes or no) provided with each event information.

[0137] If the user selects the Yes menu (941) among the menus (941, 942) displayed in the event information of Fig. 11, the processor (120) provides an event screen.

[0138] Fig. 12 illustrates an example of an event screen corresponding to the event information of Fig. 11. When an intent to participate in a quiz event is received, the processor (120) may provide data on an event screen (950) including a quiz to an external device (200). Fig. 12 illustrates an event screen (950) displayed on a single external device (200), but when multiple users input an intent to participate, the processor (120) provides the same event screen (950) to each of the external devices of the multiple users.

[0139] The processor (120) may provide a reward of some kind, such as virtual points or an electronic coupon, when a user participates in a quiz event and enters the correct answer.

[0140] Meanwhile, when a user participates in an event, the processor (120) identifies the label information of the emoticon used by the user. The processor (120) compares the identified label information with the mood of the event information and updates the label information based on the comparison result.

[0141] For example, if the label information matches the mood, the reliability score of the label information of the emoticon used by the user can be increased, or changed from a predicted label level to a confirmed label. On the other hand, if the identified label information and the mood of the event information do not match, the processor (120) can change the label information of the emoticon used by the user to label information corresponding to the mood of the event information, or update the reliability score of the previous label information to a lower level. The reliability score can be information that can indicate the accuracy of the viewing mood determined based on the label information. The reliability score can be information that is identical to or includes the above-described weight, but is not necessarily limited thereto, and can also be information used separately from the weight. That is, the processor (120) can use the above-described weight when summing the emotion label information of a specific emoticon, and use the reliability score to determine whether to change the emotion label information. If the reliability score of a specific emoticon is below a threshold, the processor (120) can change the emotion label information of the emoticon, or exclude the emoticon from determining the viewing mood.

[0142] The various operations of the electronic device (100) have been described above. The processor (120) of the electronic device (100) can perform the various operations described above by executing a program or command stored in the memory (130).

[0143] For example, the processor (120) may perform the various operations described above using an artificial intelligence model. An artificial intelligence model is a computer system or software module that implements human-level intelligence. The artificial intelligence model has the characteristics of a machine learning and judgment that improves its recognition rate with use.

[0144] Artificial intelligence models are composed of machine learning (deep learning) technology that uses algorithms that classify / learn the characteristics of input data on their own, and element technologies that use machine learning algorithms to simulate the cognitive and judgment functions of the human brain.

[0145] The element technologies may include, for example, at least one of a linguistic understanding technology that recognizes human language / characters, a visual understanding technology that recognizes objects as if they were human vision, an inference / prediction technology that judges information and logically infers and predicts, and a knowledge representation technology that processes human experience information into knowledge data.

[0146] The artificial intelligence model in the present disclosure can perform operations such as logically inferring and predicting label information that matches the characteristics of an emoticon based on previously stored data by executing the processor (120), analyzing emoticons or texts, etc. and identifying the viewing mood through probability-based inference based on the analysis results and usage history thereof.

[0147] These various operations of the artificial intelligence model according to the present disclosure can be performed by the processor (120) and the memory (130).

[0148] The processor (120) may be comprised of one or more processors. As described above, the processor (120) may be implemented in various forms. In particular, one or more processors may be implemented as a dedicated artificial intelligence processor. The dedicated artificial intelligence processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0149] Meanwhile, the artificial intelligence model can be stored in memory (130). The artificial intelligence model can be created through learning.

[0150] Creating through learning means that a basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating a predefined operation rule or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed in the electronic device (100) itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0151] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations by calculating the results of previous layers and the multiple weights. The multiple weights of the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated during the learning process to reduce or minimize the loss or cost values ​​obtained by the artificial intelligence model.

[0152] Artificial neural networks may include deep neural networks (DNNs), such as, but not limited to, convolutional neural networks (CNNs), deep neural networks (DNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), generative adversarial networks (GANs), or deep Q-networks.

[0153] In various embodiments of the present disclosure, the processor (120) must provide the user with an appropriate emoticon, which may be provided based on the results of an inference and prediction process utilizing an artificial intelligence model. Inference and prediction are technologies for logically inferring and predicting information by judging it, and include knowledge and probability-based inference, optimized prediction, preference-based planning, and recommendations.

[0154] For example, when a new emoticon without label information is used in a chat window used by multiple users, the processor (120) can identify the mood of the chat window using texts used in the chat window and an artificial intelligence model. The mood of the chat window may be a viewer's mood or emotional information.

[0155] Specifically, the processor (120) inputs texts used during the execution of the chat function into an artificial intelligence model. The artificial intelligence model can output probability values ​​of multiple moods inferred based on the text. The processor (120) can identify the mood with the maximum probability value among the output probability values ​​as the mood of the chat window. To increase accuracy, the processor (120) can use multiple texts as input values, and can also determine the mood of the chat window when the difference between the maximum probability value and other probability values ​​exceeds a threshold value.

[0156] Alternatively, the processor (120) may re-update the label information if there is a discrepancy between the estimated emotional information of users based on the artificial intelligence model and texts and the updated label information.

[0157] Alternatively, when an emoticon matching multiple label information is used more than a preset number of times in a chat window, the processor (120) may estimate the emotional information of users using the text and artificial intelligence model uploaded to the chat window, and set weights for the multiple label information accordingly.

[0158] Alternatively, when a new emoticon is discovered, the processor (120) may analyze the characteristics of the new emoticon using an artificial intelligence model, obtain emotional label information corresponding to the new emoticon, and then match and store the label information including the same with the new emoticon. Meanwhile, in the above description, the electronic device (100) is illustrated and described as performing various operations such as providing or recommending emoticons to various external devices, or updating or adding label information of emoticons according to the emoticon usage status of each external device, but at least some of these operations may be performed by an external device or another electronic device.

[0159] Below, the configuration of another electronic device corresponding to one of the external devices of Fig. 1 is specifically described with drawings.

[0160] FIG. 13 is a block diagram showing the configuration of a display device according to at least one embodiment of the present disclosure.

[0161] According to FIG. 13, the display device (800) includes a communication unit (810), a processor (820), a memory (830), and a display (840).

[0162] The communication unit (810) is a configuration for performing communication with at least one external device. For example, the display device (800) can perform communication with the electronic device (100) of FIG. 1 or other external devices (200-1 to 200-n) through the communication unit (810). The communication unit (810) may include an IR signal receiving module capable of receiving a remote control signal. However, if the display device (800) is implemented to be connected to a remote control according to a wireless communication standard such as Bluetooth or Wi-Fi, the IR signal receiving module may be omitted. Since an implementation example of the communication unit (810) has been exemplified in the section regarding FIG. 2, redundant description thereof will be omitted.

[0163] The processor (820) is a configuration for controlling the overall operation of the display device (800). Specifically, the processor (820) can control the display (840) to display content.

[0164] In addition, the processor (820) may control the display (840) to display at least one of various types of emoticon UIs and chat windows as illustrated in FIGS. 6 to 8 when specific content is displayed or when a user selects a menu for emoticon input. Since an implementation example of the processor (820) has been exemplified in the section regarding FIG. 2, redundant description thereof will be omitted.

[0165] The memory (830) is configured to store various software, commands, control codes, and data required for the operation of the display device (800). For example, the memory (830) can store various data (e.g., layout data, graphic data, emoticon information, etc.) for configuring an emoticon UI or chat window. Since an example of the memory (830) has been described in FIG. 2, a detailed description thereof will be omitted.

[0166] The display (840) is a configuration for displaying content or other various screens under the control of the processor (830). The display (840) may be implemented as a display including a self-luminous element or a display including a non-luminous element and a backlight. For example, it may be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, an LED (Light Emitting Diodes), a micro LED, a Mini LED, a PDP (Plasma Display Panel), a QD (Quantum dot) display, a QLED (Quantum dot light-emitting diodes), etc. The display (130) may also include a driving circuit, a backlight unit, etc., which may be implemented in a form such as an a-si TFT, an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc.

[0167] When various user commands are input, the processor (820) can perform operations corresponding to the user commands. The user commands may be received via a remote control corresponding to the display device (800), input via buttons or other operation panels provided on the main body of the display device (800), or received via an external electronic device connected to the display device (800). In the present disclosure, when a user command for viewing content is input, the processor (820) can control the display to output content.

[0168] Alternatively, the processor (820) may receive a user voice signal through a microphone built into the display device (800) or built into an external device, and recognize a user command included in the user voice signal. For example, if a microphone is built into the remote control, the remote control may receive an analog voice signal, digitize it, and transmit it to the display device (800). The processor (820) may receive a voice signal through a Wi-Fi module, a Bluetooth module, or the like of the communication unit (810).

[0169] Alternatively, the processor (820) may receive a user voice signal through a mobile phone with a remote control application installed, other terminal device, or AI speaker.

[0170] In addition, when the chat function is executed, the processor (820) may control the display (840) to display a chat window in an area of ​​the screen where content is displayed, on one side of the content, or in a form overlapping the upper side of the content screen, and to display various texts or emoticons entered by the user of the electronic device and other users in the chat window. The processor (820) may identify the mood, i.e., the viewing mood, based on the text or emoticons displayed in the chat window.

[0171] Although only one communication unit (810) is illustrated in FIG. 13, the number and type of communication units are not limited. For example, the display device (800) may separately include a communication unit for communicating with a server device and a communication unit for communicating with other external devices, such as a remote control. Specifically, communication with the server device may be accomplished via an Ethernet modem or Wi-Fi module, and communication with a remote control or the like may be accomplished via a Bluetooth module. However, the present disclosure is not limited thereto, and the display device (800) may also include an integrated communication unit equipped with multiple communication modules.

[0172] When a user command to view specific content is input, the processor (820) receives content from a content source and controls the display (840) to display the received content. As described above, the content source can be implemented in various types. In FIG. 13, only the communication unit (810) is illustrated, but the display device (800) may further include an interface such as an HDMI port, DP, RGB, DVI, USB, or Thunderbolt for receiving video / audio signals by being connected to external content sources. HDMI, DP, and Thunderbolt are ports that can transmit video and audio signals simultaneously. The processor (820) performs various processes such as demuxing, decoding, and scaling on content received from the content source through the communication unit (810) and these various interfaces to configure screen data, and provides the configured screen data to the display (840).

[0173] Meanwhile, when a user command for displaying an emoticon UI or a chat window is input, the processor (820) configures a screen as described in the various drawings described above and provides it to the display (840). When a chat function is executed while content is being displayed, the processor (820) can control the display (840) to display a chat window in an area (e.g., the right or bottom) of the screen where the content is being displayed. When an emoticon input function is executed, the processor (820) can also control the display (840) to display an emoticon UI within the screen. The emoticon UI and the chat window may be displayed separately according to separate user commands, but are not necessarily limited thereto, and the emoticon UI and the chat window may be displayed together when the chat function is executed.

[0174] Alternatively, the processor (820) may control the display (840) to display at least one of an emoticon UI and a chat window superimposed on the content screen being played, while maintaining the size or shape of the content screen without any change. In this case, the processor (820) may increase the transparency of at least one of the superimposed emoticon UI and the chat window so that the content portion of the superimposed area is identifiable. The user may select an emoticon through the emoticon UI. The processor (820) may display the selected emoticon on the emoticon UI or the chat window, and may transmit information about the selected emoticon to a server device or another device viewing the same content through the communication unit (810). When transmitting to the server device, the server device may relay the received emoticon information to the other device. As a result, the emoticon entered by the user may be displayed on the other devices as well. When the processor (820) receives information about an emoticon entered on another device from the server device or the other device, it may display it on the emoticon UI or the chat window. In this manner, the user may share his or her feelings about the content with others.

[0175] When a certain number of emoticons are displayed in the emoticon UI or chat window, the processor (820) can identify the viewing mood based on the label information matching the emoticons. Once the viewing mood is identified, the processor (820) can control the display (840) to display a message asking the user whether to proceed with a corresponding event. If the user wishes to proceed with an event, the user can directly determine the type of event.

[0176] For example, if a user decides on a wave cheering event, the processor (820) can transmit information about the event related to the decided mood to a server device or other devices. The server device or other devices can transmit information about whether or not to participate in the event or the result of participation to the display device (800). The processor (820) can update label information for the emoticons used by each user and store the updated information in the memory (830) based on the emoticons used by each user and the degree of participation in the event.

[0177] When a server device receives an event request, the server device may directly provide an event when event requests with the same content are received from a certain number of external devices or more. The server device may transmit the participation results of each user for the event to the display device (800). The processor (820) may update label information based on the participation results provided by the server device.

[0178] Since the specific renewal method, the method for judging the viewing atmosphere, and the method for processing new emoticons have been described in other embodiments described above, a duplicate description will be omitted.

[0179] Although FIG. 13 illustrates a method of processing information in a display device including a display (840), as described above, these operations may also be performed in an electronic device that does not include a display.

[0180] In this case, the processor may generate screen data for content screens, emoticon UIs, chat windows, etc., and then transmit the screen data and a control signal for controlling the display of the screen based on the screen data to a connected external display device (e.g., a TV, monitor, projector, etc.). Such data and control signals may be transmitted through an input / output interface connected between the electronic device and the external display device, or may be transmitted through a communication session if a communication session is connected between the electronic device and the external display device.

[0181] FIG. 14 is a flowchart illustrating an information processing method of an electronic device according to at least one embodiment of the present disclosure.

[0182] According to FIG. 14, the electronic device provides an event corresponding to the viewing mood of the content identified based on label information of emoticons input by multiple users viewing the same content to multiple users (S1410).

[0183] Specifically, the electronic device verifies the label information of various emoticons entered by the user while viewing content and identifies the viewing mood of the content based on that label information. For example, if a certain number of emoticons are entered, the electronic device adds up the emotional label information contained in the label information of each emoticon and compares the number of emotional labels, such as positive, negative, and neutral. Accordingly, the viewing mood can be identified based on a relatively large number of emotional labels. The electronic device then provides multiple users with events corresponding to the viewing mood.

[0184] Events that correspond to the viewing mood can be selected from an event pool. Electronic devices may pre-store information about the event pool. An event pool can be a collection of information about various events. The event pool can include various events such as cheering, games, quizzes, advertisements, and voting. There are two ways to create events: a system-driven method, in which the system automatically generates events based on the user's emoticon input, and a user-driven method, in which events are provided based on clustered emoticon usage. Furthermore, the guidance text provided when an event is provided varies depending on the viewing mood. For example, a "positive" mood might be accompanied by a message like, "Would you like to play a wave-surfing game, perfect for an exciting cheering mood?", while a "negative" mood might be accompanied by a message like, "The cheering mood seems to be bad. Are you not feeling well either?"

[0185] Next, the electronic device updates (S1420) the label information stored for each of the plurality of emoticons according to the degree of participation in the provided event.

[0186] The degree of participation in an event can be determined based on the number of user selections for the YES / NO menu asking whether the user will participate. Since user participation is high when an event is provided that suits the viewing atmosphere of real-time content, a high participation rate for the provided event can be determined as an appropriate understanding of the viewing atmosphere. In this case, the electronic device can update the initial label information assigned to the emoticon, i.e., the predicted label, to a confirmed label. If the participation rate is high, the electronic device can also update the predicted label of the emoticon used by the user to a different predicted label depending on whether the user participated in the event. For example, if a user using a negative emoticon participates in a positive event, the predicted label of the emoticon used by the user, which is a negative emotion label, can be changed to a positive emotion label. Depending on the embodiment, the electronic device can also update the positive emotion label to a confirmed label based on the atmosphere of the chat window, etc.

[0187] Conversely, if engagement is low, the device may conclude that it has not properly captured the viewing atmosphere. In this case, the device may retain the predicted labels.

[0188] Repeated updates of label information enable the creation of more accurate emotional labels, which in turn allows for a more accurate understanding of the viewing mood.

[0189] FIG. 15 is a flowchart illustrating an information processing method according to another embodiment of the present disclosure.

[0190] According to Fig. 15, since existing emoticons are assigned a predictive label, the viewing mood of the content can be identified based on the assigned label. However, when a new emoticon is added, there may be cases where a label is not assigned. Accordingly, the electronic device determines whether an emoticon input by a user has an emotional label assigned (S501), and when it determines that a new emoticon without label information has been added, the electronic device estimates label information corresponding to the emoticon based on at least one of the shape, color, and movement of the emoticon or the text included in the emoticon (S503), and matches the estimated label information to the new emoticon and stores it (S504).

[0191] For example, when a user uses a new emoticon that has not been assigned an emotion label, the electronic device analyzes the motion or the text of the emoticon using an artificial intelligence model. For example, as shown in FIG. 3 , the electronic device can recognize that the motion is negative based on the trembling human figure of the emoticon (301). Furthermore, the electronic device can recognize negative text phrases such as “quitting baseball” or “angry.” The artificial intelligence model can estimate the emotion label of the emoticon by combining the motion and text of the emoticon. Since the emoticon (301) of FIG. 3 has a negative motion and text, the artificial intelligence model can assign a predicted label corresponding to the “negative” emotion to the emoticon.

[0192] As described above, the AI ​​model can assign predictive labels by analyzing the motion and text of the emoticon itself, but it can also assign predictive labels by analyzing the text that the user directly enters while performing the chat function.

[0193] When an emoticon without label information is used in a chat window used by multiple users, the artificial intelligence model of the electronic device can match label information corresponding to the mood of the chat window identified based on texts used in the chat window to the emoticon and store it in the memory of the electronic device.

[0194] For example, when a large number of users use unlabeled new emoticons while performing a chat function and write positive phrases such as “fighting” or “well done,” the AI ​​model recognizes this as a positive mood and assigns a predicted label corresponding to the “positive” sentiment to the emoticon.

[0195] FIG. 16 is a flowchart illustrating an information processing method according to another embodiment of the present disclosure.

[0196] According to FIG. 16, if a user uses an emoticon with an intention different from the assigned emotion label, the electronic device can estimate the label information of the emoticon using an artificial intelligence model. Specifically, if an emoticon with updated label information is used more than a preset number of times during the chat function (S601), the electronic device estimates the emotional information of users using the emoticon based on the text uploaded to the chat window (S602) (S603). If the estimated emotional information and the updated label information do not match (S604), the electronic device re-updates the label information based on the estimated emotional information (S605). Here, the updated label information refers to a previously confirmed label. If the estimated emotion and the label of the used emoticon match, the electronic device determines the viewing mood based on the previously confirmed label (S606).

[0197] Specifically, when multiple users input emoticons with a confirmed label during a chat function, the electronic device recognizes the chat phrases entered by the users and checks whether the viewing mood matches the confirmed label of the entered emoticon. For example, if most of the multiple users use emoticons with a "positive" label, but the text phrase entered during the actual chat function is a "negative" response, the AI ​​model updates the label assigned to the emoticon from "positive" to "negative."

[0198] If multiple users use an emoticon for an intention different from the emotion label assigned to it, in addition to the method of updating the label assigned to the emoticon as described above, a method of assigning multiple emotion labels may also be used.

[0199] Specifically, when the same emoticon is used in chat windows with different viewing moods, the electronic device can generate multiple label information according to the different viewing moods, match it to the emoticon, and store it in the memory.

[0200] When an emoticon matching multiple label information is used more than a preset number of times during a chat function, the electronic device estimates the emotional information of users using the emoticon based on the text uploaded to the chat window. Furthermore, the electronic device sets a weight for each of the multiple label information based on the estimated emotional information and stores it in memory. When multiple users viewing the same content input multiple emoticons, the weights are reflected in the label information of the multiple emoticons, and the viewing mood is identified based on the result of the aggregation.

[0201] For example, if an emoticon can express both "positive" and "negative" emotions, it can be assigned both "positive" and "negative" labels. In this case, an AI model analyzes text entered by multiple users during chat functions and assigns weights to each emotional label.

[0202] The control methods of FIGS. 14 to 16 may be performed by the electronic device (100) of the configuration described in FIGS. 1 and 2 or the display device of FIG. 13, but are not necessarily limited thereto, and may also be performed by devices of different configurations. Programs or commands for performing the various information processing methods described above may be provided in a state stored in a non-transitory readable medium. The non-transitory readable medium may be loaded and used in a device capable of calling the commands stored in the storage medium and operating according to the called commands. Accordingly, when the program or command stored in the non-transitory readable medium is executed by a processor, the processor may directly, or under the control of the processor, use other components to perform the operations described in the various embodiments described above.

[0203] A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0204] Instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily on the storage medium.

[0205] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be implemented as a product distributed online through an application store, in addition to the non-transitory readable recording medium described above. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0206] Accordingly, in a non-transitory computer-readable medium storing one or more instructions executed by a control unit of an electronic device to cause the electronic device to perform an operation,

[0207] The operation of the recording medium includes the steps of receiving emoticons input by multiple users viewing the same content, identifying a viewing mood based on label information of the received emoticons, identifying a viewing mood based on label information of the received emoticons, providing an event corresponding to the identified mood to the multiple users, and updating label information for each emoticon based on the degree of participation in the event.

[0208] While the present invention has been described with reference to the attached drawings, the scope of the present invention is determined by the claims described below and should not be construed as being limited to the aforementioned embodiments and / or drawings. Furthermore, it should be clearly understood that improvements, modifications, and variations apparent to those skilled in the art, as defined in the claims, are also included within the scope of the present invention.

[0209]

Claims

1. In electronic devices, memory; Department of Communications; Processor; including; The above processor, Based on information about emoticons input by multiple users watching the same content, received through the above communication unit, control is provided to provide an event related to the mood corresponding to the label information for each of the input emoticons, An electronic device that updates the label information based on the degree of participation in the above event.

2. In paragraph 1, The above label information is, Contains emotion label information indicating at least one emotion among positive, negative, and neutral, The above processor, Identify the mood based on the maximum number of emotion label information among the emotion label information corresponding to the input emoticon, Events related to the above atmosphere are: Includes at least one event among games, quizzes, phrases, polls, advertisements, and cheering. The above processor, An electronic device that controls the event to provide different information according to the atmosphere.

3. In paragraph 1, The above memory stores the artificial intelligence model, The above processor, An electronic device that matches the shape, color, movement, and text included in the new emoticon and label information estimated by the artificial intelligence model to the new emoticon for which the label information does not exist, and stores the same in the memory.

4. In paragraph 1, The above memory stores the artificial intelligence model, The above processor, An electronic device that, when a new emoticon for which the label information does not exist is used during a chat function, matches the texts used during the chat function and the label information corresponding to the mood identified by the artificial intelligence model to the new emoticon and stores the same in the memory.

5. In paragraph 1, The above memory stores the artificial intelligence model, The above processor, An emoticon with the above updated label information is used more than a preset number of times while performing a chat function, If the emotional information estimated by the artificial intelligence model based on the text used during the execution of the chat function does not match the updated label information, Re-updating the label information based on the estimated emotional information. Electronic devices.

6. In paragraph 1, The above memory stores the artificial intelligence model, The above processor, An emoticon that matches multiple label information is used more than a preset number of times during the chat function. Store the weight set for each of the plurality of label information based on the text used during the execution of the chat function and the emotional information of the users estimated by the artificial intelligence model in the memory, An electronic device that identifies the mood based on the result of applying the weight to the label information of the plurality of emoticons when multiple users viewing the same content input multiple emoticons.

7. In paragraph 6, The above processor, An electronic device that, when the same emoticon is used to perform chat functions with different moods, matches multiple label information acquired according to the different moods to the emoticon and stores it in the memory.

8. In paragraph 1, including display; The above processor, When a user command for viewing content is input, the display is controlled to output the content, Based on the execution of the chat function, the display is controlled to output text or emoticons input by the user or another user viewing the same content as the content in a chat window output in an area of the screen where the content is output, An electronic device that identifies the mood based on the text or emoticon.

9. In paragraph 1, The above processor, An electronic device that controls the communication unit to transmit information about an event related to the atmosphere to at least one external device, and when a participation signal for the event is received through the communication unit, identifies the degree of participation in the event based on the number of external devices that transmitted the participation signal.

10. In paragraph 1, The above memory stores the minimum standard information for providing an event, The above processor, An electronic device that controls to provide the event when the label information occupying the largest proportion among the label information of emoticons used during viewing of the same content satisfies the minimum standard information.

11. In a method of processing information using an electronic device, A step of providing an event corresponding to the viewing mood of the content identified based on label information of emoticons entered by multiple users viewing the same content; and An information processing method comprising a step of updating label information stored for each of a plurality of emoticons according to the degree of participation in the event.

12. In paragraph 11, The above label information is, Contains emotion label information indicating at least one emotion among positive, negative, and neutral, The above viewing mood is identified based on the maximum number of emotion label information among the emotion label information of the emoticons input by the plurality of users. Events corresponding to the above atmosphere are: Includes at least one event among user-participatory games, quizzes, phrases, polls, advertisements, and cheering; An information processing method in which an event corresponding to the above atmosphere is provided with different information according to the above atmosphere.

13. In paragraph 11, An information processing method further comprising: a step of matching and storing, for a new emoticon for which the label information does not exist, at least one of the shape, color, movement, and text included in the new emoticon and label information estimated by an artificial intelligence model to the new emoticon.

14. In paragraph 11, An information processing method further comprising the step of matching and storing label information corresponding to texts used during the chat function and a mood identified by an artificial intelligence model to the new emoticon when a new emoticon for which the label information does not exist is used during the chat function.

15. In paragraph 11, An emoticon with the above updated label information is used more than a preset number of times while performing a chat function, If the emotional information estimated by the artificial intelligence model based on the text used during the above chat function does not match the updated label information, An information processing method further comprising a step of re-updating the label information based on the estimated emotional information.

Images