Electronic device and operation method thereof

Abstract

Provided may be an operation method of an electronic device, comprising the steps of: receiving a notification message while first content is displayed; determining the priority of the received notification message by using a first machine learning model trained to output the priority determined for an input notification message on the basis of an action history of a user with respect to at least one notification message corresponding to the input notification message; acquiring an immersion level of the user predicted with respect to first content by inputting information about the first content to a second machine learning model trained to predict an immersion level of the user with respect to content being played on the basis of input information about the content being played; determining a notification time point and a notification method of the received notification message on the basis of the determined priority of the notification message and the predicted immersion level; and notifying the notification message according to the determined notification time point and the determined notification method of the notification message.

Description

Electronic device and method of operation thereof

[0001] Various embodiments relate to an electronic device and a method of operation thereof. More specifically, the invention relates to an electronic device and a method of operation thereof that determine the timing and method of notification by considering the importance of the notification message and the user's immersion in the content when notifying a user of a notification message.

[0002] As the use of digital devices in daily life increases, so do the notification messages received from these devices. However, it is not easy to determine whether these notification messages contain essential information for the user or are merely noise. While notification messages sometimes provide users with important information in a timely manner, they can occasionally disrupt immersion in content being watched or cause stress. Therefore, to address this issue, research is needed on methods to eliminate unnecessary notification messages and provide only important messages to the user in an appropriate manner by learning the user's electronic device usage patterns.

[0003] An electronic device according to one embodiment may include at least one processor including a processing circuit.

[0004] An electronic device according to one embodiment may include a memory that stores one or more instructions.

[0005] One or more instructions may be executed individually or collectively by one or more processors to enable the electronic device to receive a notification message while the first content is displayed.

[0006] One or more instructions may be executed individually or collectively by one or more processors to determine the priority of a received notification message using a first machine learning model learned to output a priority determined for said input notification message based on a user's response history to at least one notification message corresponding to the input notification message.

[0007] One or more instructions are executed individually or collectively by one or more processors, thereby inputting information obtained for the first content into a second machine learning model trained to obtain a predicted level of immersion for the user's content being played based on input information for the content being played by the electronic device, so as to obtain a predicted level of immersion for the first content.

[0008] One or more instructions may be executed individually or collectively by one or more processors to enable the electronic device to determine the timing and method of notification of a received notification message based on the importance and predicted level of immersion of the determined notification message.

[0009] One or more instructions may be executed individually or collectively by one or more processors to enable the electronic device to notify a notification message according to the notification time and method of notification of the determined notification message.

[0010] A method of operation of an electronic device according to one embodiment may include the step of receiving a notification message while the first content is displayed.

[0011] A method of operation of an electronic device according to one embodiment may include the step of determining the priority of a received notification message using a first machine learning model trained to output a determined priority for the input notification message based on a user's response history for at least one notification message corresponding to an input message.

[0012] A method of operation of an electronic device according to one embodiment may include the step of acquiring the user's immersion in the first content by inputting information acquired for the first content to a second machine learning model trained to acquire the user's predicted immersion in the content being played based on input information for the content being played.

[0013] A method of operation of an electronic device according to one embodiment may include the step of determining the notification time and notification method of the received notification message based on the importance of the determined notification message and the predicted level of immersion.

[0014] A method of operating an electronic device according to one embodiment may include the step of generating state change data when a state change of the electronic device is detected.

[0015] A method of operation of an electronic device according to one embodiment may include a step of notifying the notification message according to the notification time and notification method of the determined notification message.

[0016] A computer-readable recording medium according to one embodiment may be a computer-readable recording medium having a program recorded thereon for implementing a method of operation of an electronic device comprising the step of receiving a notification message while the first content is displayed.

[0017] A computer-readable recording medium according to one embodiment may be a computer-readable recording medium having a program recorded thereon for implementing a method of operation of an electronic device comprising the step of determining the importance of a received notification message using a first machine learning model learned to output an importance determined for said input notification message based on a user's response history to at least one notification message corresponding to an input message.

[0018] A computer-readable recording medium according to one embodiment may be a computer-readable recording medium having a program recorded thereon for implementing a method of operation of an electronic device comprising the step of acquiring a user's immersion level predicted for the first content by inputting information acquired for the first content to a second machine learning model trained to acquire a user's immersion level predicted for the content being played based on input information for the content being played.

[0019] A computer-readable recording medium according to one embodiment may be a computer-readable recording medium having a program recorded thereon for implementing a method of operation of an electronic device comprising the step of determining the notification time and notification method of the received notification message based on the importance of the determined notification message and the predicted user immersion.

[0020] A computer-readable recording medium according to one embodiment may be a computer-readable recording medium having a program recorded thereon for implementing a method of operation of an electronic device comprising the step of notifying the notification message according to the notification time and notification method of the determined notification message.

[0021] FIG. 1 is a drawing illustrating an example of an electronic device operating according to one embodiment of the present disclosure.

[0022] FIG. 2 is a diagram showing examples of notification messages obtained from various external devices according to one embodiment of the present disclosure.

[0023] FIG. 3 is a diagram illustrating examples of various external devices capable of transmitting a notification message to an electronic device according to one embodiment of the present disclosure.

[0024] FIG. 4 is a drawing showing an example of a notification message according to one embodiment of the present disclosure.

[0025] FIG. 5 is a diagram illustrating an example of a method in which an electronic device according to one embodiment of the present disclosure determines the importance of a received notification message using artificial intelligence.

[0026] FIG. 6 is a diagram showing an example of a user notification message response history obtained by an electronic device according to one embodiment of the present disclosure.

[0027] FIG. 7 is a diagram illustrating an example of a method in which an electronic device according to one embodiment of the present disclosure predicts the level of immersion of a content user in content using artificial intelligence.

[0028] FIG. 8 is a flowchart of a method for an electronic device according to one embodiment of the present disclosure to identify information about content in order to predict the level of immersion of a content user in the content.

[0029] FIG. 9 is a flowchart of a method of operation of an electronic device according to one embodiment of the present disclosure.

[0030] FIG. 10 is a drawing illustrating an example of a method in which an electronic device according to one embodiment of the present disclosure notifies a notification message.

[0031] FIG. 11 is a diagram illustrating an example of a method for determining the importance of a received notification message by an electronic device according to one embodiment of the present disclosure.

[0032] FIG. 12 is a flowchart of a method for an electronic device to process a combined message according to one embodiment of the present disclosure.

[0033] FIG. 13 is a diagram illustrating a method for an electronic device according to one embodiment of the present disclosure to generate received notification messages into a combined message and a change in importance during the process.

[0034] FIG. 14 is a flowchart of an electronic device according to one embodiment of the present disclosure.

[0035] FIG. 15 is a block diagram of an electronic device according to one embodiment of the present disclosure.

[0036] FIG. 16 is a detailed block diagram of an electronic device according to one embodiment of the present disclosure.

[0037] FIG. 17 is a diagram illustrating an example of a structure in which an electronic device according to one embodiment of the present disclosure operates using a server.

[0038] FIG. 18 is a diagram illustrating an example of a structure in which an electronic device according to one embodiment of the present disclosure operates using an on-device AI.

[0039] Embodiments of the present disclosure are described below in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0040] The terms used in this disclosure are described in their current, general form considering the functions mentioned herein; however, they may refer to various other terms depending on the intent of those skilled in the art, case law, the emergence of new technologies, etc. Accordingly, the terms used in this disclosure should not be interpreted solely by their names, but should be interpreted based on the meaning of the terms and the overall content of this disclosure.

[0041] Furthermore, the terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit this disclosure.

[0042] In addition, when a component is described in the present disclosure as being “connected” or “connected” to another component, it should be understood that the component may be directly connected to or directly connected to the other component, but unless otherwise specifically stated, it may also be connected or connected through another component in between.

[0043] The terms “above” and similar designations used in this specification, particularly in the claims, may indicate both singular and plural forms. Furthermore, unless there is a description explicitly specifying the order of the steps describing the method according to this disclosure, the described steps may be performed in a suitable order. This disclosure is not limited by the order in which the described steps are described.

[0044] Phrases such as "in some embodiments" or "in one embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment.

[0045] Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations.

[0046] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps.

[0047] Furthermore, the connecting lines or connecting members between the components depicted in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In the actual device, connections between components may be represented by various alternative or added functional connections, physical connections, or circuit connections.

[0048] Additionally, terms such as "...part," "module," etc., as described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0049] The expression “configured to” as used in this disclosure may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware. Instead, in some situations, the expression “system configured to” may mean that the system is “capable of” together with other devices or components. For example, the phrase “a processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing said operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or an application processor) capable of performing said operations by executing one or more software programs stored in memory.

[0050] Additionally, in the specification, the term “user” may refer to a person who controls the function or operation of an electronic device using the electronic device.

[0051] In the specification, "at least one A or B" or "at least one A, or B" means that it may include any one of A, B, and A and B. Similarly, "at least one A, B, or C" or "at least one A, B, or C" means that it may include any one of A, B, C, A and B, A and C, B and C, or A, B, and C.

[0052] Below, embodiments of the present invention are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the invention. Additionally, parts of the drawings that are irrelevant to the description are omitted to clearly explain the invention.

[0053] FIG. 1 is a diagram illustrating an example of an electronic device operating according to one embodiment of the present disclosure, FIG. 2 is a diagram illustrating an example of a notification message obtained from various external devices according to one embodiment of the present disclosure, and FIG. 3 is a diagram illustrating an example of various external devices capable of transmitting a notification message to an electronic device according to one embodiment of the present disclosure.

[0054] In one embodiment, the electronic device (100) may receive at least one notification message from at least one external device (300) including external device #1 to external device #n or at least one application installed on the electronic device (100) while content is being played.

[0055] In the present disclosure, "notification message" may refer to any message transmitted to a user of the electronic device (100) through the electronic device (100). The notification message may be received from an external device (300) or from an internal application or hardware, etc.

[0056] For example, the electronic device (100) may receive a message notifying that washing is complete from an external device, such as (a) of FIG. 2, receive a connection-related notification message from an external device, such as (b) of FIG. 2, or receive messages notifying that washing and drying are complete from an external device, such as a washing machine and a dryer, respectively, as in (c) of FIG. 2.

[0057] In addition, the electronic device (100) can receive or obtain all messages that can be output by at least one external device (300) or at least one application as notification messages.

[0058] As illustrated in the embodiment of FIG. 1, the electronic device (100) may be a smart TV, but this is merely one embodiment and may be implemented in various forms.

[0059] The electronic device (100) can be easily implemented as an electronic device including a large video output unit, such as a TV, but is not limited thereto. Additionally, the electronic device (100) and the external device (300) may be fixed or mobile, and may be a digital broadcast receiver capable of receiving digital broadcasts.

[0060] In one embodiment, the electronic device (100) may be implemented in the form of a device that performs video output as part of its functions while performing other functions.

[0061] An electronic device (100) and an external device (300) according to one embodiment of the present disclosure may be implemented in various forms such as a tablet PC, a smartphone, a digital camera, a camcorder, a laptop computer, a smart TV, a netbook computer, a desktop, an e-book terminal, a video phone, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigation device, a wearable device, a smart refrigerator, and other home appliances.

[0062] The electronic device (100) may have a built-in display, but is not limited thereto and may be implemented in a form that operates by connecting to an external display even without having a built-in display.

[0063] For example, the electronic device (100) may be implemented in a form that outputs video to a separate external display through a video or audio output port, such as an STB, Apple TV, etc., without a display or having a simple display for notifications, etc.

[0064] In this case, the electronic device (100) may have an output port for outputting a video or audio signal to a display. The output port may be in a form that can transmit video and audio signals simultaneously, such as HDMI, DP, Thunderbolt, etc., or may be in a form where there are separate ports for transmitting video and audio signals separately.

[0065] In one embodiment, the electronic device (100) can transmit video or audio signals via wired communication or wireless communication, etc.

[0066] The electronic device (100) may be implemented not only as an electronic device with a flat display, but also as an electronic device with a curved display having curvature or as a flexible electronic device with adjustable curvature. The output resolution of the electronic device (100) may include, for example, HD (High Definition), Full HD, Ultra HD, or a resolution sharper than Ultra HD.

[0067] In one embodiment, at least one external device (300) capable of transmitting a notification message to an electronic device (100) may include the electronic devices shown in FIG. 3.

[0068] At least one external device (300) may include various electronic devices that can be used by the user of the electronic device (100).

[0069] At least one external device (300) may include a portable device such as a smartphone, smart pad, etc., that can be used by the user of the electronic device (100).

[0070] At least one external device (300) may include IoT devices around the electronic device (100), such as a camera, switch, motion sensor, door sensor, doorbell, light, humidity sensor, smoke detection sensor, washing machine, dishwasher, speaker, thermometer, various locking devices, refrigerator, cooker, power outlet, etc.

[0071] At least one external device (300) is not limited to those described in the present disclosure and may include any electronic device that can be connected to and communicate with the electronic device (100) via a wired or wireless network.

[0072] The electronic device (100) may receive a notification message from at least one external device (300) while playing content, and among the at least one notification message received by the electronic device (100), there may be important messages that must be checked immediately, but there may also be messages that are not.

[0073] In one embodiment, the electronic device (100) may classify at least one received notification message and collect or store at least one notification message to determine an appropriate notification time and notification method for each classification.

[0074] In one embodiment, the electronic device (100) can determine the importance of a received notification message. A method for the electronic device (100) to determine the importance of a received notification message will be described later in FIGS. 6 and FIGS. 7, etc.

[0075] In one embodiment, the electronic device (100) can infer context regarding content being played. In one embodiment, the electronic device (100) can predict the user's level of immersion regarding content being played by analyzing information regarding content being played. A method for the electronic device (100) to predict the user's level of immersion regarding content being played will be described later in FIGS. 8 and 9, etc.

[0076] In one embodiment, the electronic device (100) can determine the notification method of a notification message based on the importance of the received message and the predicted user immersion in the content being played.

[0077] In one embodiment, the electronic device (100) may classify the notification method of a notification message into three types based on the importance of the received message and the predicted level of user immersion regarding the content being played. The first type is an "Aggregation and Notice later" type that generates a combined message and notifies at a later time; the second type is an "Aggregation and Notice at Intermission" type that generates a combined message and notifies at an intermission; and the third type is a "Notice Now and Instruction" type that notifies immediately and conveys a requirement to the user.

[0078] In one embodiment, the first type may use a notification method that automatically accumulates and combines simple repetitive notification messages determined not to be important to the user, and then notifies the user all at once later. In one embodiment, if the electronic device (100) determines that notification messages have been continuously accumulated but their importance is lower than the importance of the viewing content, it may integrate them into a single notification and then notify the user all at once at the time the power of the electronic device (100) is turned off.

[0079] In one embodiment, the second type may be a notification method that accumulates and integrates notification messages determined to have higher importance than the first type and relatively lower importance than the third type, and displays them on the screen at an intermission point where the user's viewing immersion is predicted to have decreased. The intermission point may be, for example, an advertisement time, a remote control operation time, or a program or channel switching time.

[0080] In one embodiment, if the notification message is determined to be of a third type with high importance, the electronic device (100) immediately notifies the user of the notification message, and if it is identified that user feedback is required, it may generate a user instruction or notification to provide additional notification. In this case, the electronic device (100) may determine whether to remove the initial notification message based on the feedback received from the user. The feedback received from the user may be feedback regarding compliance with the additionally notified user instruction or notification. For example, if the user needs to immediately check the status of one of the external devices (300), the electronic device (100) may immediately display a message instructing this, and if feedback is obtained that the user has completed the instruction, the notification message may be deleted.

[0081] In one embodiment, the electronic device (100) may determine that the initial notification message cannot be removed if it detects that the user has not complied with additionally notified user instructions or notifications. In one embodiment, if the electronic device (100) determines that the initial notification message cannot be removed, it may generate the same user instructions or notifications to re-notify the user. In one embodiment, if the electronic device (100) determines that the initial notification message cannot be removed, it may draw the user's attention by re-notifying the notification message with added auditory or visual effects, etc.

[0082] In one embodiment, the electronic device (100) can notify a notification message according to a notification time and notification method determined based on the importance of the determined notification message and the predicted level of user engagement.

[0083] In the present disclosure, the notification method may include at least one of a method of combining a plurality of notification messages into a single message, a method of notifying in the form of a popup, a method of notifying in the form of a text bar, a method of notifying after saving, a method of notifying accompanied by voice output, and a method of requesting feedback from the user after notification. However, this is merely an example and the notification method is not limited to what is described.

[0084] In the present disclosure, integrating a plurality of notification messages to generate a combined message does not simply mean generating a message by combining all notification messages, but may mean generating a message by summarizing, omitting, or modifying messages based on context or relationships between messages.

[0085] In one embodiment, the electronic device (100) can use a server (200) to generate and train a first model used to determine the importance of a received message and a second model used to predict the user's immersion in content being played.

[0086] The electronic device (100) can improve the user's content viewing environment by automatically determining the timing and method of notification of notification messages based on the user's response history to notification messages and the user's level of immersion in the content being played, thereby allowing the user to skip the cumbersome task of manually setting whether to notify each application or external device (300) and, if necessary, turning off the notification sound or changing to vibration mode.

[0087] In one embodiment, the user can be identified based on an account logged into the electronic device (100).

[0088] In one embodiment, the user may be identified by person recognition through a camera of the electronic device (100), iris recognition, person recognition based on motion feature analysis, or person recognition through voice analysis.

[0089] FIG. 4 is a drawing showing an example of a notification message according to one embodiment of the present disclosure.

[0090] In one embodiment, the notification message received by the electronic device (100) may be an advertisement message.

[0091] In one embodiment, the notification message received by the electronic device (100) may include an advertisement message provided from at least one external device (300) or at least one application based on the content of the content being played or the context of the user.

[0092] In one embodiment, when the content being played is a travel entertainment program, the electronic device (100) may receive an advertising message about a travel destination or related travel destination introduced in the program from at least one external device (300) or an advertising application installed within the electronic device (100).

[0093] In one embodiment, the advertising message may be a travel advertising message such as “Special offer on Istanbul 7-night 8-day travel package!” as in FIG. 4 (a).

[0094] In one embodiment, the electronic device (100) may receive an advertising message for a product or a similar product from at least one external device (300) or an advertising application installed within the electronic device (100) regarding a product included in the content being played or a product used by a performer in the content being played.

[0095] In one embodiment, the advertising message may be a product advertising message such as "AA laptop promotion!!! Only for one week" as in FIG. 4 (b).

[0096] In one embodiment, the advertising message may be an image including video.

[0097] In one embodiment, when a specific scene is viewed repeatedly or a gesture of sudden concentration by the user is detected, the electronic device (100) may receive an advertising message related to the scene from at least one external device (300) or an advertising application installed within the electronic device (100).

[0098] An electronic device (100) according to one embodiment of the present disclosure can determine the importance of a received advertising message by using a first machine learning model learned to determine the importance of an advertising message based on a user's response history to an advertising message.

[0099] An electronic device (100) according to one embodiment of the present disclosure can predict the level of immersion of a user in content being viewed by acquiring and inputting information about content being viewed into a second machine learning model. The second machine learning model may be a model trained to predict the level of immersion of a user in said content being played based on information about the content being played.

[0100] An electronic device (100) according to one embodiment of the present disclosure can determine the timing and method of notification of a received advertising message based on the importance of the determined advertising message and the predicted level of user engagement.

[0101] An electronic device (100) according to one embodiment of the present disclosure can determine the timing of providing or displaying a received advertising message based on the importance of the determined advertising message and the predicted level of user engagement.

[0102] An electronic device (100) according to one embodiment of the present disclosure can determine a method for providing or displaying a received advertising message based on the importance of the determined advertising message and the predicted level of user engagement.

[0103] An electronic device (100) according to one embodiment of the present disclosure can notify a notification message according to the notification time and notification method of a determined advertisement message.

[0104] In one embodiment, the electronic device (100) may notify the user of an advertising message immediately after the viewing of the program has ended, based on the importance of the advertising message and the user's predicted level of immersion in the content being played. In this case, the electronic device (100) can expect improved advertising effectiveness without interfering with the user's viewing of the content.

[0105] In one embodiment, the electronic device (100) may notify the user of an advertisement message immediately during content playback or at the time the power of the electronic device (100) is turned off, based on the importance of the advertisement message and the user's immersion in the content being played.

[0106] In one embodiment, the electronic device (100) may not notify the advertising message based on the importance of the advertising message and the predicted level of user immersion regarding the content being played.

[0107] An electronic device (100) according to one embodiment of the present disclosure can update a first machine learning model and a second machine learning model based on user feedback regarding notification of a received advertisement message.

[0108] The contents described in this disclosure regarding general notification messages may be applied to advertising messages.

[0109] FIG. 5 is a diagram illustrating an example of a method in which an electronic device according to one embodiment of the present disclosure determines the importance of a received notification message using artificial intelligence.

[0110] An electronic device (100) according to one embodiment of the present disclosure can determine the importance of a received notification message through a neural network learned based on the user's response history to the notification message.

[0111] Artificial intelligence is a computer system that implements human-level intelligence, where machines learn and make judgments autonomously, and recognition accuracy improves with use. AI technology consists of machine learning (deep learning) technology, which utilizes algorithms to autonomously classify and learn the characteristics of input data, and component technologies that employ machine learning algorithms to mimic functions such as cognition and judgment of the human brain.

[0112] For example, the elemental technologies may include at least one of a linguistic understanding technology that recognizes human language / characters, a visual understanding technology that recognizes objects like human vision, an inference / prediction technology that judges information to logically infer and predict, a knowledge representation technology that processes human experience information into knowledge data, and a motion control technology that controls autonomous driving of a vehicle and the movement of a robot.

[0113] Artificial intelligence-related functions according to the present disclosure may be operated through the processor (110) and memory (120) of FIG. 16. The processor (110) may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs, VPUs (Vision Processing Units), or artificial intelligence-dedicated processors such as NPUs. The one or more processors (110) control input data to be processed according to predefined operation rules or artificial intelligence models stored in memory (120). Alternatively, if the one or more processors (110) are artificial intelligence-dedicated processors, the artificial intelligence-dedicated processors may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0114] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a predefined operation rules or artificial intelligence models are created by a basic artificial intelligence model being trained using multiple learning data by a learning algorithm to perform a desired characteristic (or purpose). Such learning may be performed within the electronic device (100) itself where the artificial intelligence according to the present disclosure is performed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0115] An artificial intelligence model can be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values ​​and performs neural network operations through calculations between the results of previous layers and the multiple weights. The multiple weights possessed by the multiple neural network layers can be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights can be updated so that the loss or cost values ​​obtained by the artificial intelligence model during the learning process are reduced or minimized.

[0116] In an embodiment using a deep learning algorithm, the processor (110) can determine the importance of a received notification message using a pre-trained deep neural network model (510).

[0117] The previously trained deep neural network model (510) may be an artificial intelligence model trained through learning that takes the user's response history to the notification message as input and outputs the importance information of the received notification message as output.

[0118] A deep neural network model may be, for example, a Convolutional Neural Network (CNN). However, it is not limited thereto, and the deep neural network model may be a known artificial intelligence model comprising at least one of a Recurrent Neural Network (RNN), a Restricted Boltzmann Machine (RBM), a Deep Belief Network (DBN), a Bidirectional Recurrent Deep Neural Network (BRDNN), and Deep Q-Networks.

[0119] The electronic device (100) can determine the importance of a received notification message using various machine learning algorithms, such as a regression model and multiple linear regression analysis.

[0120] In one embodiment, the previously trained deep neural network model (510) may be an artificial intelligence model trained through learning that takes as input the user's response history to the notification message and the importance information for each notification message determined by the notification message sender, and outputs the importance information of the received notification message as an output value.

[0121] In one embodiment, the previously trained deep neural network model (510) may be an artificial intelligence model trained through learning that takes as input the user's response history to the notification message, the characteristics of the notification message sender, and the importance information for each notification message determined by the notification message sender, and outputs the importance information of the received notification message.

[0122] In one embodiment, the previously trained deep neural network model (510) may be an artificial intelligence model trained through learning that acquires as inputs the user's response history to the notification message, environmental information around the electronic device (100) at the time the notification message is received, characteristics of the notification message sender, and importance information for each notification message determined by the notification message sender, and outputs the importance information of the received notification message. The characteristics of the notification message sender may include the functional characteristics of the notification message sender. Environmental information around the electronic device (100) at the time the notification message is received may include the time, date, season, location information, etc., at which the notification message is received.

[0123] FIG. 6 is a diagram showing an example of a user notification message response history obtained by an electronic device according to one embodiment of the present disclosure.

[0124] According to the embodiment of FIG. 6, the electronic device (100) can store a notification message response history for the user as follows.

[0125] The electronic device (100) can store user notification message response information that the notification message (610) received from the washing machine during content playback, "Starting washing with washing machine XX course," was not confirmed and was removed.

[0126] The electronic device (100) can store user notification message response information that the notification message (620) received from the washing machine during content playback, "Check the level of contamination and change the washing course to XX," was not confirmed and was removed.

[0127] The electronic device (100) can store user notification message response information that the notification message (630) received from the cooker while playing content has been confirmed as "cooker cooking is complete".

[0128] The electronic device (100) can store user notification message response information indicating that a subsequent action of opening the door of the cooker was performed after confirming the notification message (640) "Cooker cooking is complete" received from the cooker while playing content.

[0129] The electronic device (100) can store user notification message response information indicating that a subsequent action of operating the dryer was performed after confirming the notification message (650) received from the washing machine while the electronic device (100) is playing content, which says "Washing is complete. Automatically set the drying course according to the laundry."

[0130] The electronic device (100) can store user notification message response information when the notification message (660) "The robot vacuum cleaner has started cleaning" received from the robot vacuum cleaner while playing content is not acknowledged and is removed.

[0131] The electronic device (100) can store user notification message response information indicating that a subsequent action of operating the induction cooker was performed after confirming a notification message (670) received from the induction cooker during content playback, which is "Cooking is complete. The induction cooker is still at a high temperature."

[0132] The electronic device (100) can store user notification message response information indicating that the notification message (680) received from the dryer while playing content, "Drying is complete. The door has been opened automatically," has been confirmed.

[0133] In one embodiment, the electronic device (100) may store the elapsed time from the time each notification message is received until the time it is confirmed by the user and use this to determine the time of message notification. In one embodiment, the electronic device (100) may identify messages with shorter elapsed times as important messages.

[0134] The history of notification message responses accumulated over a long period for a user can serve as important reference information for determining how important a notification message received from an external device (300) is to the user.

[0135] In one embodiment, the user's notification message response history may include the time taken to check the received notification message and the frequency of checking, the method of checking the message after receiving the notification message and the method of removing it.

[0136] In one embodiment, the electronic device (100) can determine the importance of a received notification message by considering not only the notification message response history accumulated for the user, but also the importance information for each notification message set by the notification message sender.

[0137] For example, even if the notification messages are sent from the same refrigerator, a message indicating a simple refrigerator door opening or closing may be set as a low-importance notification by the refrigerator, whereas a notification that the refrigerator door has not been closed for more than a certain amount of time may be set as a relatively higher-importance notification.

[0138] In one embodiment, the electronic device (100) can determine the importance of a notification message received from the notification message sender based on the importance of the notification message determined by the notification message sender.

[0139] FIG. 7 is a diagram illustrating an example of a method in which an electronic device according to one embodiment of the present disclosure predicts the level of immersion of a content user in content using artificial intelligence.

[0140] Content regarding artificial intelligence that overlaps with what has already been explained in Fig. 5 may be omitted.

[0141] An electronic device (100) according to one embodiment of the present disclosure can predict the level of immersion in the content being played through a neural network learned based on information about the content being played.

[0142] In one embodiment, information about the content may include at least one of the type of content being played, genre, motion analysis information, color analysis information, running time information, whether it is a series, and the user's playback information.

[0143] For example, user engagement with content when the content type is 'advertisement' can be predicted to be lower than user engagement with content when the content type is 'drama'.

[0144] In one embodiment, information regarding the content may include time sequential information after the electronic device (100) has started to be used. The time sequential information after the electronic device (100) has started to be used may include information such as remote control operation information and operation sequence while viewing the content, and whether another external device is used while viewing the content.

[0145] For example, the electronic device (100) can predict that the level of immersion in the content being played is low if the user spends a long time using the smartphone while watching the content.

[0146] In an embodiment using a deep learning algorithm, the processor (110) can predict the level of immersion of the content being played by using a pre-trained deep neural network model (710).

[0147] The previously trained deep neural network model (710) may be an artificial intelligence model trained through learning that takes information about the content being played as input and outputs a prediction of the immersion level of the content being played.

[0148] The electronic device (100) can predict and output the level of immersion of the content being played using various machine learning algorithms.

[0149] In one embodiment, the previously trained deep neural network model (710) may be an artificial intelligence model trained through learning that takes as input information about the content being played and context information of the user watching the content, and outputs a prediction of the level of immersion for the content being played.

[0150] In one embodiment, the user's context information may include at least one of the usage status of at least one external device (300) connected to the electronic device (100), user operation information, and the user's response history to notification of a notification message.

[0151] For example, if the electronic device (100) uses at least one external device (300) connected to the electronic device (100) while the user is watching the content, the user's level of immersion in the content being watched can be predicted to be low.

[0152] In one embodiment, the electronic device (100) can predict that the user's immersion in the content being viewed is lower as the frequency of use or the usage time of at least one external device (300) connected to the electronic device (100) of the user who is viewing the content is higher.

[0153] In one embodiment, the deep neural network model (710) may be learned based on user context information obtained from the electronic device (100) or information about various content played on the electronic device (100).

[0154] In one embodiment, the previously trained deep neural network model (710) may be an artificial intelligence model trained through learning that takes as input information about the content being played, context information of the user watching the content, and surrounding environment information of the electronic device (100), and outputs a prediction of the immersion level of the content being played.

[0155] In one embodiment, learning may be performed separately on a server (200) or on an electronic device (100).

[0156] FIG. 8 is a flowchart of a method for an electronic device according to one embodiment of the present disclosure to identify information about content in order to predict the level of immersion of a content user in the content.

[0157] Referring to FIG. 8, an electronic device (100) according to one embodiment of the present disclosure can divide at least one scene of content being played to generate a plurality of image patches (S810).

[0158] An electronic device (100) according to one embodiment of the present disclosure can perform linear projection and position embeddings on each image patch (S820). In step S820, the electronic device (100) can extract features of each image patch through linear transformation and position embeddings of each image patch.

[0159] An electronic device (100) according to one embodiment of the present disclosure can perform feature embedding (S830).

[0160] Feature embedding refers to the process of converting acquired features into a vector form. Feature embedding is primarily used in thin learning or deep learning and can enable efficient processing by reducing high-dimensional data into low-dimensional data.

[0161] Feature embedding is performed for various reasons, but one of the biggest reasons for performing feature embedding is to reduce 'non-linear' data—which is difficult to analyze using general linear regression or classification algorithms because it is not expressed linearly—to an appropriate dimension and represent it linearly.

[0162] An electronic device (100) according to one embodiment of the present disclosure can obtain more accurate and effective results when performing tasks such as classifying or clustering data by placing data with similar characteristics in close locations and data with different characteristics far apart through feature embedding.

[0163] Feature embeddings are learnable parameters, and since the model automatically finds the optimal embedding space as training progresses, if a sufficient amount of training data is provided, the model can independently identify the characteristics of the data and construct an appropriate embedding space.

[0164] An electronic device (100) according to one embodiment of the present disclosure can resolve nonlinearity by converting high-dimensional data into low-dimensional vectors through feature embedding, reflect similarity and difference between data, and find an optimal embedding space through learnable parameters.

[0165] An electronic device (100) according to one embodiment can represent features for each frame of an image patch as a vector through feature embedding.

[0166] An electronic device (100) according to one embodiment of the present disclosure can identify information about content through a feed forward network (S840) (S850).

[0167] A feed-forward network refers to a neural network in which information flows in only one direction from the input layer to the output layer—that is, a network that operates only in the forward direction without feedback.

[0168] A feedforward network generally consists of three layers: an input layer, a hidden layer, and an output layer. The input layer receives data from the outside, the hidden layer processes the input data and performs necessary operations, and the output layer produces the final output based on the results of the hidden layer's execution.

[0169] Activation functions exist between each layer and can serve to limit input values ​​to a specific range or introduce non-linearity. Representative activation functions include ReLU, sigmoid, and tanh, and recently, modified forms of activation functions such as LeakyReLU are also being used.

[0170] Feed-forward networks have the advantage of being easy and fast to implement thanks to their simple structure, so they are widely used in various fields such as image classification and speech recognition.

[0171] In one embodiment, information about the content may include context information such as what kind of content is being played, whether an advertisement is being played, and whether the channel is being changed.

[0172] An electronic device (100) according to one embodiment of the present disclosure can output a vector obtained from a transformer block in a desired form by using a feed forward network (S840). For example, the electronic device (100) can identify information regarding whether a certain type of content is being played, whether an advertisement is being played, and whether a channel is being changed.

[0173] In one embodiment, the electronic device (100) can input the usage status of various external devices (300), notification messages received from these external devices (300) when viewing content, and other user habit data to the feed-forward network along with the output from the transformer block.

[0174] In one embodiment, the electronic device (100) can output a descriptor describing the relationship between the user's content viewing status and external devices (300) based on user feature information from a feed-forward network.

[0175] In one embodiment, the electronic device (100) can identify information regarding content, such as whether a certain type of content is being played, whether an advertisement is being played, and whether a channel is being changed, through a feed-forward network.

[0176] FIG. 9 is a flowchart of a method of operation of an electronic device according to one embodiment of the present disclosure.

[0177] Referring to FIG. 9, an electronic device (100) according to one embodiment of the present disclosure can receive a notification message while the first content is displayed (S910).

[0178] The first content may be any video that can be played on the electronic device (100). The notification message may be a message transmitted from at least one external device (300) or an application or hardware within the electronic device (100). The notification message may be in the form of text or an image. The length of the notification message may vary.

[0179] An electronic device (100) according to one embodiment of the present disclosure can determine the importance of a received notification message by using a first machine learning model learned to output an importance determined for an input notification message based on a user's response history for at least one notification message corresponding to an input message (S920).

[0180] In one embodiment, the first machine learning model may be a model trained to determine the importance of the notification message described in FIGS. 5 and FIGS. 6, etc. In this regard, details that overlap with previously described content are omitted.

[0181] An electronic device (100) according to one embodiment of the present disclosure can obtain the user's immersion in the first content by inputting information obtained for the first content to a second machine learning model trained to obtain the user's predicted immersion in the content being played based on input information for the content being played (S930).

[0182] In one embodiment, the second machine learning model may be a model trained to predict the level of immersion for the content being played as described in FIGS. 7 and 8, etc. In this regard, details that overlap with those previously described are omitted.

[0183] An electronic device (100) according to one embodiment of the present disclosure can determine the notification time and notification method of a received notification message based on the importance and predicted level of immersion of the determined notification message (S940).

[0184] An electronic device (100) according to one embodiment may adjust the weights of the importance of the determined notification message and the predicted user immersion level considered to determine the notification timing and notification method of the received notification message. For example, the electronic device (100) may reflect the weight of the importance of the determined notification message as 60% and the weight of the predicted user immersion level as 40%. That is, the electronic device (100) may determine the notification timing and notification method of the received notification message by considering the importance of the notification message determined in step S920 as more important than the user immersion level predicted in step S930.

[0185] An electronic device (100) according to one embodiment may determine the notification method of a notification message by distinguishing it into an "Aggregation and Notice later" type, an "Aggregation and Notice at Intermission" type, and a "Notice Now and Instruction" type, as described in the embodiment of FIG. 1. For each type, content that overlaps with what has already been described in FIG. 1 may be omitted.

[0186] An electronic device (100) according to one embodiment can classify the timing of notification of a notification message into immediate notification, notification during an advertisement, notification when using a remote control, notification when changing scenes, notification after watching content, etc.

[0187] An electronic device (100) according to one embodiment may notify a notification message by selecting at least one notification method among a method of combining a plurality of notification messages into a single message, a method of notifying in the form of a popup, a method of notifying in the form of a text bar, a method of notifying after saving, a method of notifying accompanied by voice output, and a method of requesting feedback from the user after notification. However, the notification method of the notification message is not limited to those described above. A method of combining a plurality of notification messages into a single message will be described later in FIGS. 10 to 13.

[0188] An electronic device (100) according to one embodiment may notify by varying the location, size, thickness, color, etc. of the message notification according to the importance of the message determined in step S920.

[0189] For example, high-importance notification messages can be displayed in large, bold letters in the center of the screen. In one embodiment, important parts of the notification message content may be highlighted in red.

[0190] An electronic device (100) according to one embodiment of the present disclosure may notify a notification message according to the notification time and notification method of the determined notification message (S950).

[0191] An electronic device (100) according to one embodiment can notify a notification message based on the importance of the notification message determined in step S920 and the user's immersion predicted in step S930, according to the notification timing and notification method of the notification message determined in step S940.

[0192] An electronic device (100) according to one embodiment of the present disclosure may re-notify a notification message when it is determined that there is a need for re-notification by considering the importance score of the notification message.

[0193] For example, in the case of a message sent from a high-risk sender such as an induction cooktop, the notification message may be re-notified even if the user has deleted the content, by considering the induction cooktop's current temperature (remaining heat output), the presence of dishes remaining on the cooktop, and whether the power is turned off.

[0194] In one embodiment, the notification message being re-notified may be notified in the form of a pop-up message. In one embodiment, the electronic device (100) may vary the form of the pop-up by considering the importance score of the notification message.

[0195] In one embodiment, the electronic device (100) may be configured to automatically delete the associated notification message when it is determined that there is no dangerous situation.

[0196] FIG. 10 is a drawing illustrating an example of a method in which an electronic device according to one embodiment of the present disclosure notifies a notification message.

[0197] In the embodiment of FIG. 10, the electronic device (100) may be a smart TV. After TV viewing begins, two notification messages may be received from external device #3, and then one notification message may be received from external device #2. The above notification messages may be messages of low importance. The importance of the above notification messages may be determined by a first model (1010), that is, a model that determines the importance of messages. The first model (1010) may be the same as the first machine learning model described in FIG. 9. In this regard, details that overlap with previously described content are omitted.

[0198] In the embodiment of FIG. 10, the TV does not notify the user immediately upon receiving two notification messages received from external device #3 and one notification message received from external device #2, respectively, but combines the three notification messages to create a first combined message, and then notifies the user of the first combined message at the time when an advertisement starts between the content being watched.

[0199] The second model (1020) is a model that predicts the user's immersion in content and may be the same as the second machine learning model described in FIG. 9. In this regard, any content that overlaps with what has been previously described is omitted. The second model (1020) can identify video types, such as whether specific content is being played, whether an advertisement is being played, whether the channel has been changed, or whether TV viewing has ended, by analyzing information about content being played.

[0200] In the embodiment of FIG. 10, after the first combined message is notified, two additional notification messages may be received from external device #2 while watching TV. The two received notification messages may be messages of low importance. The TV may not notify the user immediately upon receiving each of the two notification messages from external device #2, but may combine the two notification messages to create a second combined message, and then notify the user of the second combined message at the time when a change in the channel being watched is detected.

[0201] In the embodiment of FIG. 10, after the second combined message is notified, three additional notification messages may be received from external device #1 while watching TV. The three received notification messages may have different levels of importance. However, no message may have a higher level of importance than the content being watched. The level of importance of the content being watched may be determined based on the prediction results of the second model (1020). The TV may not notify the user immediately upon receiving each of the three notification messages received from external device #1, but may combine the three notification messages to create a third combined message, and then notify the user of the third combined message when TV viewing ends. In one embodiment, the time when TV viewing ends is detected may be when the TV power is turned off.

[0202] User response information regarding the notified first combination message, second combination message, and third combination message can be stored as history and used to update the first model.

[0203] FIG. 11 is a diagram illustrating an example of a method for determining the importance of a received notification message by an electronic device according to one embodiment of the present disclosure.

[0204] Referring to FIG. 11, an electronic device (100) according to one embodiment of the present disclosure can perform word embedding (S110).

[0205] Word embedding can refer to the representation of words in natural language processing (NLP). Generally, a representation can be a vector of real values ​​that encodes the meaning of a word in a vector space in a manner where the meanings of words closer to each other are expected to be similar. Word embedding can be performed using language modeling and feature learning techniques where words or phrases of a vocabulary are mapped to real vectors.

[0206] An electronic device (100) according to one embodiment can store notification messages received from at least one external device (300) or other application and perform word embedding.

[0207] For example, the electronic device (100) can store notification messages received from the washing machine, which is an external device (300), such as "Washing machine XX course starts," "Checks soiling level and changes washing course to XX," "Washing is complete. Automatically sets drying course according to laundry," and "Drying is complete. Door opened automatically," and perform word embedding.

[0208] An electronic device (100) according to one embodiment of the present disclosure can perform Long Short-Term Memory (LSTM) modeling on received notification messages after word embedding, through Convolutions (S1120) and Pooling (S1130) (S1140).

[0209] In one embodiment, the LSTM model can effectively combine various incoming notification messages over time to generate a combined message.

[0210] An electronic device (100) according to one embodiment can use an LSTM model to integrate messages accumulated in chronological order up to a given point in time with new messages to generate a combined message, determine the importance of the generated combined message, and use the determined importance of the combined message to determine the notification time and notification method of the combined message.

[0211] Specifically, the electronic device (100) can input messages accumulated up to that point and new messages into an LSTM model. At this time, each notification message may include an individually determined importance score. The LSTM model can process messages by considering the importance score of each notification message. The LSTM model can combine the notification messages to create a combined message, in which important information may be emphasized and other information may be omitted. The generated combined message may again be assigned an importance score according to its importance.

[0212] In one embodiment, the importance score may be the result calculated by the LSTM model itself.

[0213] In one embodiment, the LSTM model can perform not only the task of simply generating a combined message, but also the task of determining the importance of the generated combined message so that it is used to determine the notification timing and notification method of the combined message.

[0214] In the embodiment of FIG. 11, the LSTM model can combine sequentially received notification messages such as "Washing machine XX course starts," "Checks soiling level and changes washing course to XX," "Washing is complete. Automatically sets drying course according to laundry," and "Drying is complete. Door opened automatically" to generate a combined message such as "Washing / drying is complete and the door opened."

[0215] The LSTM model can identify the context in which the washing machine started washing, changed the washing course, started the drying course after washing was completed, and automatically opened the door after drying was completed, and in that context, it can identify that what is currently meaningful to the user is that the washing and drying are completed and the door is opened.

[0216] Therefore, based on the identified context, the LSTM model can omit low-importance notification messages and leave only notifications that are meaningful to the user in the current situation, thereby generating a combined message such as "The door has been opened as the washing / drying is complete."

[0217] LSTM models can remember the context of previous messages over the long term and reflect it in the processing of new messages.

[0218] An electronic device (100) according to one embodiment can determine the importance of a notification message by calculating an importance score.

[0219] In the embodiment of FIG. 11, the electronic device (100) can calculate an importance score for the generated combined message and obtain a score of 0.7.

[0220] The importance scores for each notification message prior to the generation of the combined message can be calculated as follows: "Starting washing on washing machine XX course" is 0.1, "Checking soil level and changing washing course to XX" is 0.2, "Washing is complete. Automatically setting drying course based on laundry" is 0.3, and "Drying is complete. Door opened automatically" is 0.7. The importance scores for each notification message may also be generated by an LSTM model.

[0221] In one embodiment, the electronic device (100) can determine that the importance score of the generated combined message is the same as that of a message whose main part of the content matches, namely, the message “Drying is complete. The door has been opened automatically”.

[0222] In one embodiment, the electronic device (100) may determine that the importance score of the generated combined message is higher than that of the message whose main part of the content matches, if the generated combined message includes more meaningful information other than the message whose main part of the content matches, namely, "Drying is complete. The door has been opened automatically."

[0223] An electronic device (100) according to one embodiment of the present disclosure can output a vector with a desired notification importance through a feed forward network (S1150).

[0224] Details regarding the feed-forward network that overlap with those already explained in Fig. 8 are omitted.

[0225] In one embodiment, the notification importance may be output in the form of an importance score.

[0226] In one embodiment, the notification importance can be output by distinguishing it into high, medium, and low importance scores.

[0227] In addition, the notification importance can be displayed in various forms.

[0228] An electronic device (100) according to one embodiment of the present disclosure can output individual notification messages and combined messages distinguished by importance using a feed forward network.

[0229] For example, the electronic device (100) can output notification messages distinguished as "Aggregation and Notice later" type, "Aggregation and Notice at Intermission" type, and "Notice Now and Instruction" type as described in FIG. 1.

[0230] FIG. 12 is a flowchart of a method for an electronic device to process a combined message according to one embodiment of the present disclosure.

[0231] An electronic device (100) according to one embodiment may store a first notification message received during content viewing without immediately outputting it, based on the importance of the notification message and the predicted level of user immersion.

[0232] An electronic device (100) according to one embodiment may store a second notification message received during content viewing without immediately outputting it, based on the importance of the notification message and the predicted level of user immersion.

[0233] An electronic device (100) according to one embodiment can identify that a first notification message that was not notified is stored before the storage of a second notification message.

[0234] An electronic device (100) according to one embodiment can combine a first notification message and a second notification message to generate a first combined message.

[0235] Referring to FIG. 12, an electronic device (100) according to one embodiment of the present disclosure can determine the importance of the first combined message using a first machine learning model when a first combined message is generated (S1210).

[0236] The first machine learning model is identical to the one described in Fig. 9, so redundant content may be omitted.

[0237] In one embodiment, the importance of the first combined message may be equal to or higher than the importance of one of the first notification message and the second notification message.

[0238] An electronic device (100) according to one embodiment of the present disclosure can determine the notification time and notification method of a first combined message based on the importance of the determined first combined message and the predicted user immersion (S1220).

[0239] The determination of the notification timing and notification method of the first combined message can be made in the same way as the determination of the notification timing and notification method for a general notification message.

[0240] An electronic device (100) according to one embodiment of the present disclosure may notify a first combined message according to a determined notification time and notification method (S1230).

[0241] Notification of the first combined message can be done in the same way as notification of a general notification message.

[0242] In one embodiment, the electronic device (100) may decide to store the first combined message without immediately notifying it.

[0243] An electronic device (100) according to one embodiment may store a third notification message received during content viewing without immediately outputting it, based on the importance of the notification message and the predicted level of user immersion.

[0244] An electronic device (100) according to one embodiment can identify that there is an unnotified first combined message when storing a third notification message.

[0245] An electronic device (100) according to one embodiment can combine a first combined message and a third notification message to generate a second combined message.

[0246] An electronic device (100) according to one embodiment can determine the importance of a second combined message using a first machine learning model when a second combined message is generated.

[0247] In one embodiment, the importance of the second combined message may be equal to or higher than the importance of one of the first combined message and the third notification message.

[0248] An electronic device (100) according to one embodiment can determine the notification time and notification method of a second combined message based on the importance of the determined second combined message and the predicted user immersion.

[0249] An electronic device (100) according to one embodiment can notify a second combined message according to a determined notification time and notification method.

[0250] FIG. 13 is a diagram illustrating a method for an electronic device according to one embodiment of the present disclosure to generate received notification messages into a combined message and a change in importance during the process.

[0251] An electronic device (100) according to one embodiment of the present disclosure may receive a message (1310) from a washing machine, which is an external device (300), during content playback, saying "Washing machine XX course is starting." Since the importance score of the 1310 message calculated by the electronic device (100) is 0.1, it may be stored without being notified immediately.

[0252] In one embodiment, the electronic device (100) may be set to immediately notify when the importance score of a notification message is greater than or equal to a threshold value in one embodiment. In the embodiment of FIG. 13, the electronic device (100) may be set to immediately notify when the importance score of a notification message is greater than or equal to 1.

[0253] An electronic device (100) according to one embodiment of the present disclosure can generate an unnotified 1310 message as a combined message (1311), such as "start washing."

[0254] An electronic device (100) according to one embodiment of the present disclosure may receive a message (1320) from a washing machine, which is an external device (300), during content playback, stating, "Check the level of contamination and change the washing course to XX." Since the importance score of the 1320 message calculated by the electronic device (100) is less than 1, the electronic device (100) may store the 1320 message without notifying.

[0255] An electronic device (100) according to one embodiment of the present disclosure can combine an unnotified 1320 message with a combined message (1311) to generate a new combined message (1321), such as "washing course changed".

[0256] An electronic device (100) according to one embodiment of the present disclosure may calculate the importance score of a new combination message (1321) and obtain 0.15. The importance score of the new combination message (1321) may be greater than or equal to the importance score of the existing combination message (1311). That is, as combination messages accumulate, the importance score may increase. However, since the importance score is still less than 1, the new combination message (1321) may also be stored without being immediately notified.

[0257] An electronic device (100) according to one embodiment of the present disclosure may receive a message (1330) stating "Cooker cooking is complete" from a cooker, which is an external device (300), during content playback. Since the importance score of the 1330 message calculated by the electronic device (100) is less than 1, the electronic device (100) may store the 1330 message without notifying.

[0258] An electronic device (100) according to one embodiment of the present disclosure can combine an unnotified 1330 message with a combined message (1321) to generate a new combined message (1331) such as "washing course changed, cooker cooking complete".

[0259] An electronic device (100) according to one embodiment of the present disclosure may calculate the importance score of a new combination message (1331) and obtain 0.25. However, since the importance score is still less than 1, the new combination message (1331) may also be stored without being immediately notified.

[0260] An electronic device (100) according to one embodiment of the present disclosure may receive a message (1340) stating "Cooker cooking is complete" from a cooker, which is an external device (300), during content playback. Since the importance score of the 1340 message calculated by the electronic device (100) is less than 1, the electronic device (100) may store the 1340 message without notifying.

[0261] An electronic device (100) according to one embodiment of the present disclosure can combine an unnotified 1340 message with a combined message (1331) to generate a new combined message (1341) such as "washing course changed, cooker cooking complete".

[0262] An electronic device (100) according to one embodiment of the present disclosure may calculate the importance score of a new combination message (1341) and obtain 0.35. However, since the importance score is still less than 1, the new combination message (1341) may also be stored without being immediately notified.

[0263] An electronic device (100) according to one embodiment of the present disclosure may receive a notification message (1350) stating "Washing is complete. Automatically set drying course according to laundry" in the same manner and generate a combined message (1351) stating "Drying course set after washing is complete".

[0264] An electronic device (100) according to one embodiment of the present disclosure may calculate the importance score of a new combination message (1351) and obtain 0.4. However, since the importance score is still less than 1, the new combination message (1351) may also be stored without being immediately notified.

[0265] An electronic device (100) according to one embodiment of the present disclosure can receive a notification message (1360) that says "Robot vacuum cleaner starts cleaning" in the same way and generate a combined message (1361) that says "Drying course set after washing complete, robot vacuum cleaner starts cleaning".

[0266] An electronic device (100) according to one embodiment of the present disclosure may calculate the importance score of a new combination message (1361) and obtain 0.4. However, since the importance score is still less than 1, the new combination message (1361) may also be stored without being immediately notified.

[0267] An electronic device (100) according to one embodiment of the present disclosure can receive a notification message (1370) stating "Cooking is complete. Induction is still at high temperature" in the same manner and generate a combined message (1371) stating "Drying course set after washing, robot vacuum cleaner started cleaning, induction at high temperature".

[0268] An electronic device (100) according to one embodiment of the present disclosure may calculate the importance score of a new combination message (1371) and obtain 0.6. However, since the importance score is still less than 1, the new combination message (1371) may also be stored without being immediately notified.

[0269] An electronic device (100) according to one embodiment of the present disclosure can receive a notification message (1380) stating "Drying is complete. Door opened automatically" in the same manner and generate a combined message (1381) stating "Washing and drying complete, dryer door open, robot vacuum in progress, induction high temperature state continued".

[0270] An electronic device (100) according to one embodiment of the present disclosure may calculate the importance score of a new combination message (1381) and obtain 0.7. However, since the importance score is still less than 1, the new combination message (1381) may also be stored without being immediately notified.

[0271] An electronic device (100) according to one embodiment of the present disclosure can detect the occurrence of discontinuity in user content after a 1380 notification message is received and a combined message (1381) is generated.

[0272] In one embodiment, discontinuity of user content may be detected when the channel is switched, the viewing content is changed, an advertisement content is started, or the power of the electronic device (100) is turned off.

[0273] An electronic device (100) according to one embodiment of the present disclosure may, when a discontinuity in user content is detected, generate and notify a final combined message (1891) based on the last combined message (1381) that "Washing and drying are complete and the door is open. The induction is at a high temperature and the robot vacuum is in operation."

[0274] In the embodiment of FIG. 13, the electronic device (100) receives eight notification messages while playing content, but the notification of the notification messages can be performed only once in the form of a combined message (1391).

[0275] Through this process, the electronic device (100) can ensure that the user is not interrupted from viewing content due to unimportant notification messages.

[0276] Additionally, the electronic device (100) can enable the user to effectively obtain necessary information by notifying only one message (1391) that summarizes the essential message at the time when a content discontinuity occurs.

[0277] The single message processing history illustrated in the embodiment of FIG. 13 may be the user's previous response history for each notification message.

[0278] FIG. 14 is a flowchart of an electronic device according to one embodiment of the present disclosure.

[0279] Referring to FIG. 14, an electronic device (100) according to one embodiment of the present disclosure can receive a notification message while the first content is displayed (S1410).

[0280] An electronic device (100) according to one embodiment of the present disclosure can determine the importance of a received notification message by using a first machine learning model learned to determine the importance of a notification message based on the user's response history to the notification message (S1420).

[0281] An electronic device (100) according to one embodiment of the present disclosure can predict the user's immersion in the first content by acquiring and inputting information about the first content into a second machine learning model trained to predict the user's immersion in the content being played based on information about the content being played (S1430).

[0282] An electronic device (100) according to one embodiment of the present disclosure can determine the notification time and notification method of a received notification message based on the importance of the determined notification message and the predicted user's immersion (S1440).

[0283] An electronic device (100) according to one embodiment of the present disclosure may notify a notification message according to the notification time and notification method of a determined notification message (S1450).

[0284] Steps S1410 through S1450 may correspond to steps S910 through S950 of FIG. 9. Therefore, any content that overlaps with what has already been explained in FIG. 9 is omitted.

[0285] An electronic device (100) according to one embodiment of the present disclosure can update a first machine learning model and a second machine learning model based on user feedback regarding notification of a received message (S1460).

[0286] An electronic device (100) according to one embodiment of the present disclosure can receive user feedback that removes a received message without acknowledging the notification.

[0287] An electronic device (100) according to one embodiment may receive negative feedback from a user indicating that notification of a message is unnecessary. In this case, the electronic device (100) may reflect the above feedback in determining the importance of similar notification messages and the timing of notification of the notification message. For example, the electronic device (100) may update the first model so that the importance score of similar notification messages is lowered.

[0288] An electronic device (100) according to one embodiment of the present disclosure may receive feedback from a user regarding the notification of a received message, indicating that the notification of the message was too late.

[0289] In this case, the electronic device (100) may incorporate the above feedback into the calculation of importance of similar notification messages and the determination of the timing of notification of the notification messages. For example, the electronic device (100) may update the first model so that the importance score of similar notification messages increases.

[0290] In one embodiment, the electronic device (100) may determine the timing and method of notification of a received notification message by giving more weight to the importance of the determined notification message when determining the timing and method of notification of a received notification message based on the importance of the determined notification message and the predicted user's immersion.

[0291] An electronic device (100) according to one embodiment of the present disclosure can update a first machine learning model and a second machine learning model based on user feedback regarding notification of a received message and user notification processing behavior information after notification.

[0292] An electronic device (100) according to one embodiment of the present disclosure can update a first machine learning model and a second machine learning model based on user feedback regarding notification of a received message, information on the user's notification processing behavior after notification, and changes in the usage patterns of the electronic device (100) and the external device (300).

[0293] In one embodiment, the electronic device (100) adjusts the parameters and rules of each model according to user feedback, changes in the viewing environment of the electronic device (100), and changes in the usage patterns of the electronic device (100) and the smart device, thereby continuously improving the user's TV viewing experience.

[0294] FIG. 15 is a block diagram of an electronic device according to one embodiment of the present disclosure.

[0295] Referring to FIG. 15, the electronic device (100) may include a processor (110) and a memory (120).

[0296] The memory (120) can store a program for processing and controlling the processor (110). Additionally, the memory (120) can store data that is input to or output from the electronic device (100).

[0297] The memory (120) may include at least one of internal memory (not shown) and external memory (not shown).

[0298] The memory (120) may include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., SD or XD memory, etc.), RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), magnetic memory, magnetic disk, and optical disk.

[0299] The built-in memory may include, for example, at least one of volatile memory (e.g., DRAM (Dynamic RAM), SRAM (Static RAM), SDRAM (Synchronous Dynamic RAM), etc.), non-volatile memory (e.g., OTPROM (One Time Programmable ROM), PROM (Programmable ROM), EPROM (Erasable and Programmable ROM), EEPROM (Electrically Erasable and Programmable ROM), Mask ROM, Flash ROM, etc.), a hard disk drive (HDD), or a solid-state drive (SSD).

[0300] According to one embodiment, the processor (110) can load instructions or data received from at least one of the non-volatile memory or other components into the volatile memory for processing. Additionally, the processor (110) can store data received from or generated from other components in the non-volatile memory.

[0301] The external memory may include, for example, at least one of CF (Compact Flash), SD (Secure Digital), Micro-SD (Micro Secure Digital), Mini-SD (Mini Secure Digital), xD (extreme Digital), and Memory Stick.

[0302] The memory (120) may store one or more instructions that can be executed by the processor (110).

[0303] In one embodiment, the memory (120) may store one or more instructions executable by the processor (110) separately in multiple memories (120).

[0304] In one embodiment, the memory (120) may store one or more instructions that can be executed individually or collectively by at least one processor (110).

[0305] In one embodiment, the memory (120) can store various information input through an input / output unit (not shown).

[0306] In one embodiment of the present disclosure, at least one of instructions, an algorithm, a data structure, program code, and an application program that can be read by a processor (110) may be stored in the memory (120). The instructions, algorithm, data structure, and program code stored in the memory (120) may be implemented in a programming or scripting language such as, for example, C, C++, Java, assembler, etc.

[0307] In one embodiment, the memory (120) may store instructions for controlling the processor (110) to notify the notification message according to the determined notification message time and method, and to determine the notification message based on the determined notification message time and method.

[0308] The processor (110) can execute an OS (Operation System) and various applications stored in memory (120) when there is user input or when conditions stored in a preset state are satisfied.

[0309] The processor (110) may include RAM (RAM) used as a storage area corresponding to various tasks performed in the electronic device (100) or for storing signals or data input from outside the electronic device (100), and ROM (ROM) stored as a control program for controlling the electronic device (100).

[0310] The processor (110) may include at least one processing circuit.

[0311] The processor (110) may include a single core, dual core, triple core, quad core, and multiples thereof. Additionally, the processor (110) may include multiple processors. For example, the processor (110) may be implemented as a main processor (not shown) and a sub processor (not shown) operating in sleep mode.

[0312] Additionally, the processor (110) may include at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and a VPU (Video Processing Unit). Alternatively, depending on the embodiment, it may be implemented in the form of a System On Chip (SOC) integrating at least one of a CPU, a GPU, and a VPU.

[0313] The processor (120) may include various processing circuits and / or multiple processors. For example, the term “processor” as used herein, including in the claims, may include at least one processor and various processing circuits. In at least one processor, one or more processors may be configured to perform the various functions described herein in a distributed manner, individually and / or collectively. As used herein, “processor,” “at least one processor,” and “one or more processors” may be configured to perform various functions. However, these terms cover, for example but without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor can perform all functions. Additionally, at least one processor may include a combination of processors performing various functions of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0314] The processor (110) can control components of various electronic devices (100) by executing one or more instructions stored in memory (120).

[0315] In one embodiment, the processor (110) can receive a notification message while the first content is displayed.

[0316] In one embodiment, the processor (110) can determine the importance of a received notification message by using a first machine learning model trained to determine the importance of a notification message based on the user's response history to the notification message.

[0317] In one embodiment, the processor (110) can predict the user's immersion in the first content by acquiring and inputting information about the first content into a second machine learning model trained to predict the user's immersion in the content being played based on information about the content being played.

[0318] In one embodiment, the processor (110) determines the notification time and method of notification of a received notification message based on the importance of the determined notification message and the predicted user engagement, and can notify the notification message according to the notification time and method of notification of the determined notification message.

[0319] In one embodiment, the processor (110) classifies the received notification message, and if the received notification message is classified into a first type, it stores the received notification message without notification, if the received notification message is classified into a second type, it notifies the received notification message at a time when the expected user's engagement is low, and if the received notification message is classified into a third type, it notifies the received notification message immediately.

[0320] In one embodiment, the processor (110) may, if the received notification message is classified as a first type, combine the received message with the previously stored message without notification to create and store a first combined message, and if the previously stored combined message is, combine the received message with the combined message to create and store a second combined message.

[0321] In one embodiment, when a first combined message is generated by executing one or more instructions, the processor (110) uses a first machine learning model to determine the importance of the first combined message, determines the notification time and method of the first combined message based on the determined importance of the first combined message and the predicted user engagement, and can notify the first combined message according to the determined notification time and method.

[0322] In one embodiment, the processor (110) can control the first machine learning model and the second machine learning model to be updated based on user feedback regarding notification of a received message by executing one or more instructions.

[0323] FIG. 16 is a detailed block diagram of an electronic device according to one embodiment of the present disclosure.

[0324] Referring to FIG. 16, the electronic device (100) may include a tuner unit (340), a processor (110), a display (320), a communication unit (350), a sensor unit (360), an input / output unit (370), a video processing unit (380), an audio processing unit (385), an audio output unit (390), a memory (120), and a power supply unit (395).

[0325] Hereinafter, the same reference numerals are assigned to configurations and steps identical to those described in FIG. 15, and redundant descriptions are omitted.

[0326] The processor (110) of FIG. 16 is configured to correspond to the processor (110) of FIG. 15, and the memory (120) of FIG. 16 is configured to correspond to the memory (120) of FIG. 15. Therefore, any content that overlaps with what was previously explained will be omitted.

[0327] A communication unit (350) according to one embodiment may include a Wi-Fi module, a Bluetooth module, an infrared communication module and a wireless communication module, a LAN module, an Ethernet module, a wired communication module, etc. At this time, each communication module may be implemented in the form of at least one hardware chip.

[0328] The Wi-Fi module and the Bluetooth module perform communication using the Wi-Fi method and the Bluetooth method, respectively. When using the Wi-Fi module or the Bluetooth module, various connection information such as SSID and session key is first transmitted and received, and then various information can be transmitted and received after establishing a communication connection using this information. The wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation).

[0329] A communication unit (350) according to one embodiment can receive user input from an external device.

[0330] A communication unit (350) according to one embodiment can communicate with an external device such as a server.

[0331] A communication unit (350) according to one embodiment may include a communication unit that performs wireless communication such as BT with a server, etc., and a communication unit that is connected to an external device such as an HDMI port, etc. At this time, the communication unit that performs wireless communication such as BT with a server, etc., can perform connection with other devices and transmission of video / audio data. The communication unit that is connected to an external device such as an HDMI port, etc., may include not only an input port for receiving input, but also an output port such as DP, HDMI, RGB, DVI, Thunderbolt, etc., for transmitting video or audio signals to an external display unit or speaker, etc.

[0332] A tuner unit (340) according to one embodiment can select only the frequency of the channel to be received by the electronic device (100) from among many radio wave components by tuning through amplification, mixing, resonance, etc. of a broadcast signal received via wired or wireless means. The broadcast signal includes audio, video, and additional information (e.g., EPG (Electronic Program Guide)).

[0333] The tuner unit (340) can receive broadcast signals from various sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, internet broadcasting, etc. The tuner unit (340) can also receive broadcast signals from sources such as analog broadcasting or digital broadcasting.

[0334] The sensor unit (360) detects voice around the electronic device (100), image around the electronic device (100), or interaction with the surroundings of the electronic device (100), and may include at least one of a microphone (331), a camera (332), and an optical receiver (333). The sensor unit (360) detects the state of the electronic device (100) or the state around the electronic device (100) and can transmit the detected information to the processor (110).

[0335] The microphone (331) receives the user's uttered voice and voice generated around the electronic device (100). The microphone (331) can convert the received voice into an electrical signal and output it to the processor (110). The microphone (331) can use various noise removal algorithms to remove noise generated during the process of receiving external acoustic signals.

[0336] The camera (332) can obtain image frames such as still images or video. Images captured through the image sensor can be processed through a processor (110) or a separate image processing unit (not shown).

[0337] Image frames processed by the camera (332) can be stored in memory (120) or transmitted externally through the communication unit (350). Two or more cameras (332) may be provided depending on the configuration of the electronic device (100).

[0338] The optical receiver (333) receives an optical signal (including a control signal) received from an external remote control device (not shown). The optical receiver (333) can receive an optical signal corresponding to user input (e.g., touch, press, touch gesture, voice, or motion) from the remote control device (not shown). A control signal can be extracted from the received optical signal under the control of the processor (110). For example, the optical receiver (333) can receive a control signal corresponding to a channel up / down button for channel switching from the remote control device (not shown).

[0339] The sensor unit (360) is illustrated as including a microphone (331), a camera (332), and an optical receiver (333), but is not limited thereto. It may include at least one of a magnetic sensor, an acceleration sensor, a temperature / humidity sensor, an infrared sensor, a gyroscope sensor, a position sensor (e.g., GPS), a barometric pressure sensor, a proximity sensor, an RGB sensor, an illumination sensor, and a Wi-Fi signal receiver, but is not limited thereto. Since the function of each sensor can be intuitively inferred by a person skilled in the art from its name, a detailed description is omitted.

[0340] The sensor unit (360) is shown as being provided in the electronic device (100) itself, but is not limited thereto and may be provided in a control unit, such as a remote control, which is located independently of the electronic device (100) and communicates with the electronic device (100). When a sensing unit (130) is provided in the control unit of the electronic device (100), the control unit may digitize information detected by the sensing unit (130) and transmit it to the electronic device (100). The control unit may communicate with the electronic device (100) using short-range communication including infrared, Wi-Fi, or Bluetooth.

[0341] For example, the microphone may be provided in the electronic device (100) itself, but may also be provided in a control device, such as a remote control, which is located independently of the electronic device (100) and communicates with the electronic device (100).

[0342] In one embodiment, when a microphone is provided in the remote control, an analog voice signal is received through the microphone, and the remote control can digitize it and transmit it to an electronic device (100) such as a TV. At this time, the remote control can communicate with the electronic device (100) using short-range communication including infrared, Wi-Fi, Bluetooth, and BT.

[0343] In one embodiment, the electronic device (100) may be equipped with a plurality of communication units (350) capable of various short-range communication including infrared, Wi-Fi, or Bluetooth.

[0344] In one embodiment, the electronic device (100) may have a plurality of communication units (350) that are different from each other, such as a communication unit that communicates with a server (200) and a communication unit that communicates with a remote control. For example, the communication unit that communicates with the server may be a communication unit that uses an Ethernet modem, a Wi-Fi module, etc., while the communication unit that communicates with the remote control may be a communication unit that uses a BT module.

[0345] In one embodiment, the electronic device (100) may have a communication unit (350) in which a communication unit communicating with a server and a communication unit communicating with a remote control are identical. For example, both the communication unit communicating with the server and the communication unit communicating with the remote control may be communication units that use a Wi-Fi module.

[0346] In one embodiment, a device such as a smartphone with a remote control application installed can perform the same role as the remote control described above. That is, a device with a remote control application installed can control an electronic device (100) and perform voice recognition functions.

[0347] Devices on which the remote control application can be installed may include, in addition to smartphones, any device capable of operating by installing an application, such as AI speakers.

[0348] In one embodiment, a device with a remote control application installed may be able to receive user voice.

[0349] In one embodiment, the electronic device (100) may include a plurality of communication units capable of implementing the communication method to transmit and receive data using Wi-Fi, BT, infrared, etc., with a device on which a remote control or a remote control application can be installed, and to control the device on which a remote control or a remote control application can be installed.

[0350] The input / output unit (370) receives video (e.g., video, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.) from outside the electronic device (100) under the control of the processor (110). The input / output unit (370) may include any one of HDMI (High-Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), DVI (Digital Visual Interface), component jack, and PC port.

[0351] The video processing unit (380) performs processing on video data received by the electronic device (100). The video processing unit (380) can perform various image processing on the video data, such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion.

[0352] The display (320) converts video signals, data signals, OSD signals, control signals, etc., processed by the processor (110) to generate driving signals. The display (320) can be implemented as a PDP, LCD, OLED, flexible display, etc., and can also be implemented as a 3D display. Additionally, the display (320) can be configured as a touch screen and used as an input device in addition to an output device.

[0353] The display (320) can output various content input through a communication unit (not shown) or an input / output unit (370), or output an image stored in memory (120). Additionally, the display (320) can output information input by a user through the input / output unit (370) to the screen.

[0354] The display (320) may include a display panel. The display panel may be a Liquid Crystal Display (LCD) panel or a panel containing various light-emitting elements such as a Light Emitting Diode (LED), Organic Light Emitting Diode (OLED), or Cold Cathode Fluorescent Lamp (CCFL). Additionally, the display panel may include not only a flat display device but also a curved display device having a curvature or a flexible display device with adjustable curvature. The display panel may also be a 3D display or an electrophoretic display.

[0355] The output resolution of the display panel may include, for example, HD (High Definition), Full HD, Ultra HD, or a resolution sharper than Ultra HD.

[0356] In the embodiment of FIG. 16, the electronic device (100) is shown to include a display, but is not limited thereto. The electronic device (100) may be configured to be connected via wired or wireless communication to a separate display device including a display, and to transmit video / audio signals to the display device.

[0357] In one embodiment, the electronic device (100) may be implemented in a form that operates by connecting to an external display even without having a built-in display.

[0358] For example, the electronic device (100) may be implemented in a form that outputs video to a separate external display through a video or audio output port, such as an STB, Apple TV, etc., without a display or having a simple display for notifications, etc.

[0359] In this case, the electronic device (100) may have an output port for outputting a video or audio signal to a display. The output port may be in a form that can transmit video and audio signals simultaneously, such as HDMI, DP, Thunderbolt, etc., or may be in a form where there are separate ports for transmitting video and audio signals separately.

[0360] In one embodiment, the electronic device (100) can transmit video or audio signals via wired communication or wireless communication, etc.

[0361] The audio processing unit (385) performs processing on audio data. Various processing such as decoding, amplification, and noise filtering on audio data can be performed in the audio processing unit (385). Meanwhile, the audio processing unit (385) may be equipped with multiple audio processing modules to process audio corresponding to multiple contents.

[0362] The audio output unit (390) outputs audio included in a broadcast signal received through the tuner unit (340) under the control of the processor (110). The audio output unit (390) can output audio (e.g., voice, sound) input through the communication unit (350) or the input / output unit (370). Additionally, the audio output unit (390) can output audio stored in the memory (120) under the control of the processor (110). The audio output unit (390) may include at least one of a speaker, a headphone output terminal, or an S / PDIF (Sony / Philips Digital Interface) output terminal.

[0363] The power supply unit (395) supplies power input from an external power source to the components inside the electronic device (100) under the control of the processor (110). Additionally, the power supply unit (395) can supply power output from one or more batteries (not shown) located inside the electronic device (100) to the internal components under the control of the processor (110).

[0364] The memory (120) may store various data, programs, or applications for driving and controlling the electronic device (100) under the control of the processor (110). The memory (120) may include a broadcast receiving module, a channel control module, a volume control module, a communication control module, a voice recognition module, a motion recognition module, an optical receiving module, a display control module, an audio control module, an external input control module, a power control module, a power control module for an external device connected wirelessly (e.g., Bluetooth), a voice database (DB), or a motion database (DB), which are not illustrated. The modules not illustrated and the database of the memory (120) may be implemented in software form to perform broadcast reception control functions, channel control functions, volume control functions, communication control functions, voice recognition functions, motion recognition functions, optical reception control functions, display control functions, audio control functions, external input control functions, power control functions, or power control functions for an external device connected wirelessly (e.g., Bluetooth) in the electronic device (100). The processor (110) can perform each of these functions using the software stored in memory (120).

[0365] In FIG. 16, the processor (110) is depicted as a single element, but is not limited thereto. In one embodiment, the processor (110) may be composed of one or more elements.

[0366] In one embodiment of the present disclosure, the processor (110) may be composed of a dedicated hardware chip that performs artificial intelligence (AI) learning.

[0367] A 'module' included in memory (120) refers to a unit that processes a function or operation performed by a processor (110), and can be implemented as software such as instructions, algorithms, data structures, or program code.

[0368] Meanwhile, the block diagram of the electronic device (100) illustrated in FIGS. 15 and 16 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted according to the specifications of the actual electronic device (100) being implemented. That is, as needed, two or more components may be combined into one component, or one component may be subdivided into two or more components. Furthermore, the functions performed in each block are intended to explain the embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

[0369] FIG. 17 is a diagram illustrating an example of a structure in which an electronic device according to one embodiment of the present disclosure operates using a server, and FIG. 18 is a diagram illustrating an example of a structure in which an electronic device according to one embodiment of the present disclosure operates using on-device AI.

[0370] An electronic device (100) according to the embodiment of FIG. 12 can obtain from a server (200) a first machine learning model learned to determine the importance of a notification message based on the user's response history to the notification message, and a second machine learning model learned to predict the user's immersion in the content being played based on information about the content being played.

[0371] The electronic device (100) according to the embodiment of FIG. 12 can determine the importance of a notification message received from at least one external device (300) using a first machine learning model.

[0372] The electronic device (100) according to the embodiment of FIG. 12 can predict the level of user immersion in the content being played using a second machine learning model.

[0373] The electronic device (100) according to the embodiment of FIG. 12 can determine the notification time and notification method of a notification message received from at least one external device (300) based on the importance of the notification message determined using a first machine learning model and the level of user immersion predicted using a second machine learning model.

[0374] In one embodiment, the server (200) can create and update a first machine learning model by acquiring and learning the user's response history to a notification message received during content playback from a plurality of electronic devices.

[0375] In one embodiment, the server (200) can update the first machine learning model by reflecting user feedback regarding the notification of a notification message received during content playback.

[0376] In one embodiment, the server (200) can generate and update a second machine learning model by obtaining information about content from a plurality of electronic devices and performing learning to predict the user's immersion level for each content.

[0377] In one embodiment, information about the content may include history information regarding how a user responded to a notification message while watching similar content. For example, if the response rate to a notification message by the user is high while watching news and the response rate to a similar notification message by the user is low while watching a drama, the electronic device (100) can predict that the user's immersion in the drama is higher than that in the news.

[0378] In one embodiment, the electronic device (100) may provide to the server all information regarding what content notification message was received from a device or application while content is being played, and how the user responded to the received notification message. In one embodiment, the electronic device (100) may identify the user based on a user account.

[0379] In one embodiment, when the electronic device (100) provides information to the server (200) for the protection of personal information, it may perform separate procedures such as encrypting or deleting content corresponding to personal information.

[0380] The server (200) can train a first machine learning model and a second machine learning model based on information received from the electronic device (100).

[0381] In one embodiment, the electronic device (100) can periodically or in real time receive a first machine learning model and a second machine learning model from a server (200) and update the existing first machine learning model and second machine learning model.

[0382] The electronic device (100) according to the embodiment of FIG. 18 can directly perform the role performed by the server (200) of FIG. 17.

[0383] An electronic device (100) according to one embodiment can periodically or in real time train and update a first machine learning model and a second machine learning model.

[0384] An electronic device (100) according to one embodiment can train and update a first machine learning model and a second machine learning model by reflecting in real time the user's response information to the received notification message and the feedback to the notified notification message.

[0385] In one embodiment, the electronic device (100) can train and update a first machine learning model and a second machine learning model by reflecting information in real time or periodically about what content notification message was received from a device or application while some content is being played and how the user responded to the received notification message.

[0386] The method of operation of an electronic device (100) according to one embodiment may also be implemented in the form of a computer-readable medium containing instructions executable by a computer, such as a program module executed by a computer. The computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, and both removable and non-removable media. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions may include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0387] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0388] The foregoing description is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical concept or essential features of the invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0389] An electronic device according to one embodiment includes at least one processor comprising a processing circuit and a memory storing one or more instructions, wherein the one or more instructions are executed individually or collectively by the at least one processor, so that the electronic device receives a notification message while a first content is displayed, determines the importance of the received notification message using a first machine learning model learned to determine the importance of the notification message based on the user's response history to the notification message, and predicts the user's immersion in the first content by acquiring and inputting information about the first content into a second machine learning model learned to predict the user's immersion in the content being played based on information about the content being played, and determines the notification time and method of the received notification message based on the determined importance of the notification message and the predicted user's immersion, and notifies the notification message according to the determined notification time and method of the notification message.

[0390] The above one or more instructions may be executed individually or collectively by the above one or more processors to classify the received notification message, and if the received notification message is classified into a first type, the received notification message is stored without notification, if the received notification message is classified into a second type, the received notification message is notified at a time when the predicted user's immersion is low, and if the received notification message is classified into a third type, the received notification message is notified immediately.

[0391] If the received notification message is classified as a first type, and there is a previously stored message without notification, the received message may be combined with the previously stored message without notification to create and store a first combined message, and if there is a previously stored combined message, the received message may be combined with the combined message to create and store a second combined message.

[0392] The importance of the second combined message is greater than or equal to the importance of the first combined message, and the importance of the first combined message may be greater than or equal to the importance of the received notification message.

[0393] When the first combined message is generated, the importance of the first combined message is determined using the first machine learning model, and the notification time and method of the first combined message are determined based on the determined importance of the first combined message and the predicted user immersion, and the first combined message is notified according to the determined notification time and method.

[0394] The above one or more instructions may be executed individually or collectively by the above one or more processors to update the first machine learning model and the second machine learning model based on the user's feedback regarding the notification of the received message.

[0395] The first machine learning model and the second machine learning model can be trained on a server.

[0396] The above notification method may include at least one of the following: a method of combining multiple notification messages into a single message, a method of notifying in the form of a popup, a method of notifying in the form of a text bar, a method of notifying after saving, a method of notifying accompanied by voice output, and a method of requesting feedback from the user after notification.

[0397] The second machine learning model is a model trained to predict the level of immersion of the content user in the content based on information about the content and context information of the content user, and by acquiring and inputting information about the first content and context information of the user into the second machine learning model, the level of immersion of the user in the first content can be predicted.

[0398] Information regarding the first content may include at least one of the type, genre, motion analysis information, color analysis information, running time information, and user playback information of the first content.

[0399] A method of operation of an electronic device according to one embodiment may include: receiving a notification message while a first content is displayed; determining the importance of the received notification message using a first machine learning model trained to determine the importance of the notification message based on the user's response history to the notification message; predicting the user's immersion in the first content by acquiring and inputting information about the first content into a second machine learning model trained to predict the user's immersion in the content being played based on information about the content being played; determining the notification time and notification method of the received notification message based on the determined importance of the notification message and the predicted user's immersion; and notifying the notification message according to the determined notification time and notification method of the notification message.

[0400] The step of determining the notification timing and method of the received notification message may include: classifying the received notification message; storing the received notification message without notification if the received notification message is classified into a first type; notifying the received notification message at a time when the predicted user's immersion is low if the received notification message is classified into a second type; and notifying the received notification message immediately if the received notification message is classified into a third type.

[0401] If the received notification message is classified as a first type, and if there is a previously stored message without notification, the received message is combined with the previously stored message without notification to create and store a first combined message, and if there is a previously stored combined message, the received message is combined with the combined message to create and store a second combined message.

[0402] The importance of the second combined message is greater than or equal to the importance of the first combined message, and the importance of the first combined message may be greater than or equal to the importance of the received notification message.

[0403] When the first combined message is generated, the method may include the steps of determining the importance of the first combined message using the first machine learning model, determining the notification time and method of the first combined message based on the determined importance of the first combined message and the predicted user immersion, and notifying the first combined message according to the determined notification time and method.

[0404] The method may further include the step of updating the first machine learning model and the second machine learning model based on the user's feedback regarding the notification of the received message.

[0405] The first machine learning model and the second machine learning model can be trained on a server.

[0406] The above notification method may include at least one of the following: a method of combining multiple notification messages into a single message, a method of notifying in the form of a popup, a method of notifying in the form of a text bar, a method of notifying after saving, a method of notifying accompanied by voice output, and a method of requesting feedback from the user after notification.

[0407] The second machine learning model is a model trained to predict the level of immersion of the content user in the content based on information about the content and context information of the content user, and may include a step of predicting the level of immersion of the user in the first content by acquiring and inputting information about the first content and context information of the user into the second machine learning model.

[0408] A computer-readable recording medium may be provided on which a program for performing the operation method of the above electronic device on a computer is recorded.

Claims

1. At least one processor (110); and An electronic device comprising a memory (120) for storing one or more instructions, wherein the one or more instructions are executed individually or collectively by the at least one processor (110): Receive a notification message while the first content is displayed, and The importance of the received notification message is determined using a first machine learning model trained to output a determined priority for the input notification message based on the user's response history to at least one notification message corresponding to the input notification message, and By inputting information obtained regarding the first content into a second machine learning model trained to obtain a predicted level of immersion regarding the content currently being played for the user based on input information regarding the content currently being played, the predicted level of immersion of the user regarding the first content is obtained, and Based on the importance of the above-determined notification message and the above-determined immersion, the notification timing and notification method of the above-determined notification message are determined, and An electronic device that notifies the notification message according to the notification time and notification method determined above.

2. In paragraph 1, the one or more instructions are executed individually or collectively by the one or more processors (110), thereby, Classify the above-mentioned received notification messages, and If the received notification message is classified as a first type, the received notification message is stored without notification, and If the received notification message is classified as a second type, the received notification message is notified at a time when the predicted user's immersion is low, and An electronic device that immediately notifies the received notification message when the received notification message is classified as a third type.

3. In paragraph 2, if the received notification message is classified as type 1, If there is a previously stored message without notification, the received message is combined with the previously stored message without notification to create and store a first combined message, and An electronic device that, if there is a previously stored combined message, combines the received message with the combined message to generate and store a second combined message.

4. In Paragraph 3, An electronic device characterized in that the importance of the second combined message is greater than or equal to the importance of the first combined message, and the importance of the first combined message is greater than or equal to the importance of the received notification message.

5. In Paragraph 4, When the first combined message is generated, the importance of the first combined message is determined using the first machine learning model, and Based on the importance of the first combined message determined above and the predicted user immersion, the notification timing and notification method of the first combined message are determined, and An electronic device that notifies the first combined message according to the above-determined notification time and notification method.

6. In any one of paragraphs 1 to 5, the one or more instructions are executed individually or collectively by the one or more processors (110), thereby, An electronic device that updates the first machine learning model and the second machine learning model based on the user's feedback regarding the notification of the received message.

7. In any one of paragraphs 1 through 6, The electronic device, wherein the first machine learning model and the second machine learning model are learned on a server.

8. In any one of paragraphs 1 through 7, The above notification method comprises at least one of the following: a method of combining multiple notification messages into a single message, a method of notifying in the form of a popup, a method of notifying in the form of a text bar, a method of notifying after saving, a method of notifying accompanied by voice output, and a method of requesting feedback from the user after notification.

9. In any one of paragraphs 1 through 8, The above second machine learning model is a model trained to predict the level of immersion of the content user in the content based on information about the content and context information of the content user, and An electronic device in which the above one or more instructions are executed individually or collectively by the above one or more processors (110) to predict the user's immersion in the first content based on input information for the first content and context information of the user to the second machine learning model.

10. In any one of paragraphs 1 through 9, An electronic device comprising at least one of the information regarding the first content, the type, genre, motion analysis information, color analysis information, running time information, and the user's playback information of the first content.

11. In a method of operating an electronic device, A step of receiving a notification message while the first content is displayed; A step of determining the priority of the received notification message using a first machine learning model trained to output a determined priority for the input notification message based on the user's response history to at least one notification message corresponding to the input notification message; A step of obtaining the user's predicted immersion for the first content by inputting information obtained for the first content into a second machine learning model trained to obtain the user's predicted immersion for the content being played based on input information for the content being played; A step of determining the notification time and notification method of the received notification message based on the importance of the notification message determined above and the predicted level of immersion; and A method of operation of an electronic device comprising the step of notifying the notification message according to the notification time and notification method of the notification message determined above.

12. In paragraph 11, the step of determining the notification time and notification method of the received notification message is: A step of classifying the received notification message; If the received notification message is classified as a first type, the received notification message is stored without notification, and If the received notification message is classified as a second type, the received notification message is notified at a time when the predicted user's immersion is low, and A method of operation of an electronic device comprising the step of immediately notifying the received notification message when the received notification message is classified into a third type.

13. In the case of Paragraph 12, if the received notification message is classified as Type 1, If there is a previously stored message without notification, the received message is combined with the previously stored message without notification to create and store a first combined message, and A method of operation of an electronic device comprising the step of, if there is a previously stored combined message, combining the received message with the combined message to generate and store a second combined message.

14. In Paragraph 13, A method of operation of an electronic device characterized in that the importance of the second combined message is greater than or equal to the importance of the first combined message, and the importance of the first combined message is greater than or equal to the importance of the received notification message.

15. A computer-readable recording medium having a program recorded thereon for performing the method of any one of paragraphs 11 through 14 on a computer.

Images