Display device for obtaining prompt and control method thereof

Abstract

A display device is disclosed. The display device comprises: a display; a sensor; and one or more processors configured to identify user context on the basis of first sensing data received from the sensor, identify a category corresponding to the user context among a plurality of categories on the basis of at least one of emotion information and situation information of a user included in the user context, obtain a prompt corresponding to the user context by inputting preference information corresponding to the identified category to a first neural network model, and control the display to display an image corresponding to the prompt.

Description

Display device for obtaining a prompt and method for controlling the same

[0001] The present disclosure relates to a display device and a control method thereof, and more particularly, to a display device for obtaining a prompt and an image corresponding to the prompt, and a control method thereof.

[0002] Technology related to display devices is rapidly developing, and various types of display devices for viewing content at home are becoming widespread.

[0003] In the past, users directly selected content through a display device and the content selected by the user was displayed, so there was a problem in that content could not be displayed according to the user's situation (or context).

[0004] For example, there was inconvenience and difficulty for users to search for and select content that was right for their situation.

[0005] There has been a demand for a way to provide content that is relevant to one's situation and context without requiring the user to manually search and select it, and even when the content that is necessary (or appropriate) for one's situation does not already exist.

[0006] According to one embodiment of the present disclosure, a display device includes a display, a sensor, and at least one processor for identifying a user context based on first sensing data received through the sensor, identifying a category corresponding to the user context among a plurality of categories based on at least one of the user's emotional information or situational information included in the user context, inputting preference information corresponding to the identified category into a first neural network model to obtain a prompt corresponding to the user context, and controlling the display to display an image corresponding to the prompt, wherein the preference information includes at least one of the user's preferred activity or preferred content corresponding to the identified category.

[0007] A method for controlling a display device according to an embodiment of the present disclosure includes a step of identifying a user context based on first sensing data, a step of identifying a category corresponding to the user context among a plurality of categories based on at least one of the user's emotional information or situational information included in the user context, a step of inputting preference information corresponding to the identified category into a first neural network model to obtain a prompt corresponding to the user context, and a step of displaying an image corresponding to the prompt, wherein the preference information includes at least one of the user's preferred activity or preferred content corresponding to the identified category.

[0008] According to one embodiment of the present disclosure for achieving the above-described object, a computer-readable recording medium including a program for executing a method for controlling a display device, the method for controlling the display device includes a step of identifying a user context based on first sensing data, a step of identifying a category corresponding to the user context among a plurality of categories based on at least one of the user's emotional information or situational information included in the user context, a step of inputting preference information corresponding to the identified category into a first neural network model to obtain a prompt corresponding to the user context, and a step of displaying an image corresponding to the prompt, wherein the preference information includes at least one of the user's preferred activity or preferred content corresponding to the identified category.

[0009] The above and other aspects and features of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0010] FIG. 1 is a drawing for explaining a display device according to an embodiment of the present disclosure.

[0011] FIG. 2 is a block diagram showing the configuration of a display device according to an embodiment of the present disclosure.

[0012] FIG. 3 is a diagram for explaining a category corresponding to a user context according to an embodiment of the present disclosure.

[0013] FIG. 4 is a diagram illustrating a prompt corresponding to a user context according to an embodiment of the present disclosure.

[0014] FIG. 5 is a drawing for explaining an image corresponding to a prompt according to an embodiment of the present disclosure.

[0015] FIG. 6 is a drawing for explaining a display device for identifying a user context according to an embodiment of the present disclosure.

[0016] FIG. 7 is a diagram illustrating a display device that identifies a user context by communicating with an external device according to an embodiment of the present disclosure.

[0017] FIG. 8 is a drawing for explaining status information and surrounding environment information of a display device according to an embodiment of the present disclosure.

[0018] FIG. 9 is a diagram illustrating a prompt corresponding to a user context according to an embodiment of the present disclosure.

[0019] FIG. 10 is a drawing for explaining a prompt corresponding to the surrounding environment information of a display device according to an embodiment of the present disclosure.

[0020] FIG. 11 is a flowchart for explaining a method for controlling a display device according to an embodiment of the present disclosure.

[0021] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0022] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0023] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0024] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0025] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0026] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be coupled via another component (e.g., a third component).

[0027] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0028] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware.

[0029] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0030] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0031] FIG. 1 is a drawing for explaining a display device according to an embodiment of the present disclosure.

[0032] Referring to FIG. 1, a display device (100) according to an embodiment of the present disclosure can display content. The display device (100) can be implemented as a TV, but is not limited thereto, and is applicable to any device having a display function, such as a video wall, a large format display (LFD), a digital signage, a digital information display (DID), a projector display, etc. In addition, the display device (100) can include various types of displays, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), a liquid crystal on silicon (LCoS), a digital light processing (DLP), a quantum dot (QD) display panel, a quantum dot light-emitting diodes (QLED), etc.

[0033] Referring to FIG. 1, the display device (100) can sense the user and identify the user context.

[0034] For example, the display device (100) may acquire first sensing data through a sensor and identify a user context based on the first sensing data. Depending on the embodiment, the user context may include the user's emotional information or situational information.

[0035] According to an embodiment, the display device (100) may obtain a prompt to provide an image corresponding to a user context.

[0036] For example, the display device (100) can input preference information according to the user context into the first neural network model to obtain a prompt. For example, the display device (100) can identify preference information about activities preferred by the user (hereinafter, preferred activities) or contents preferred by the user (hereinafter, preferred contents) according to the user context, and input the preference information into the first neural network model to obtain a prompt for generating (or obtaining) an image corresponding to the user context.

[0037] According to an embodiment, the display device (100) inputs a prompt to a second neural network model, and the second neural network model can output (or generate) an image corresponding to a user context based on the prompt.

[0038] In some embodiments, the prompt may include text that is input to the second neural network model to generate an image corresponding to the user context. In some embodiments, the prompt may include instructions (or commands) regarding a desired action for the second neural network model to perform, or conditions (or information) that may control the second neural network model.

[0039] According to an embodiment, the display device (100) can provide an image acquired through a second neural network model.

[0040] FIG. 2 is a block diagram showing the configuration of a display device according to an embodiment of the present disclosure.

[0041] Referring to FIG. 2, the display device (100) includes a display (110), a sensor (120), one or more processors (130), and a communication interface (140).

[0042] According to an embodiment, the display (110) may be implemented as various types of displays such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), a liquid crystal on silicon (LCoS), a digital light processing (DLP), a quantum dot (QD) display panel, a quantum dot light-emitting diodes (QLED), a micro light-emitting diodes (μLED), a mini LED, etc. Meanwhile, the display (110) may also be implemented as a touch screen combined with a touch sensor, a flexible display, a rollable display, a 3D display, a display in which a plurality of display modules are physically connected, etc.

[0043] According to an embodiment, the sensor (120) may sense a user located in the same space as the display device (100) and transmit first sensing data to one or more processors (130).

[0044] According to an embodiment, the sensor (120) may include a camera for acquiring image data including a user, a microphone for acquiring voice data of the user, etc. However, the present invention is not limited thereto, and the sensor (120) may also include a LiDAR sensor for sensing the distance between the display device (100) and the user, an infrared (IR) sensor for sensing the movement of the user, an ultrasonic sensor, a microwave sensor, etc.

[0045] According to an embodiment, the sensor (120) may obtain first sensing data including at least one of image data including a user or voice data of the user.

[0046] According to an embodiment, one or more processors (130) control the overall operation of the display device (100). Specifically, one or more processors (130) may be connected to each component of the display device (100) to control the overall operation of the display device (100).

[0047] One or more processors (130) may perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in memory.

[0048] According to an embodiment, one or more processors (130) may be implemented as a digital signal processor (DSP), a microprocessor, or a timing controller (TCON) that processes a digital signal. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a communication processor (CP), an ARM processor, or an artificial intelligence (AI) processor, or may be defined by the relevant terms. In addition, one or more processors (130) may be implemented as a system on chip (SoC) having a built-in processing algorithm, a large scale integration (LSI), or may be implemented in the form of a field programmable gate array (FPGA). One or more processors (130) may perform various functions by executing computer executable instructions stored in a memory.

[0049] The one or more processors (130) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (130) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors (130) may execute one or more programs or instructions stored in a memory. For example, the one or more processors (130) may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0050] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-dedicated processor).

[0051] One or more processors (130) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (130) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0052] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among a plurality of cores included in a multi-core processor, or may be performed by a plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0053] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0054] According to an embodiment, one or more processors (130) may identify a user context based on the first sensing data.

[0055] For example, one or more processors (130) can identify a user context based on at least one of image data or audio data included in the first sensing data.

[0056] According to an embodiment, the video data may include at least one of the user's pose, the user's facial expression, or the user's surroundings, and the audio data may include the user's voice.

[0057] According to an embodiment, one or more processors (130) may identify the user's emotional information based on at least one of the user's pose, the user's facial expression, the user's surroundings, or the user's voice.

[0058] For example, one or more processors (130) can identify the user's emotional information by assigning a score of happy, anger, contempt, disgust, fear, neutral, sadness, surprise, boredom, etc. based on at least one of the user's pose, the user's facial expression, the user's surroundings, or the user's voice.

[0059] According to an embodiment, one or more processors (130) may identify user context information based on at least one of the user's pose, the user's facial expression, the user's surroundings, or the user's voice.

[0060] For example, one or more processors (130) can identify the user's situation information such as preparing to go out, going out, just after returning home, exercising (home workout), resting, cooking, studying, etc. based on at least one of the user's pose, the user's facial expression, the user's surroundings, or the user's voice.

[0061] According to an embodiment, one or more processors (130) may identify one of a plurality of categories based on a user context that includes at least one of emotional information or situational information.

[0062] FIG. 3 is a diagram for explaining a category corresponding to a user context according to an embodiment of the present disclosure.

[0063] Referring to FIG. 3, one or more processors (130) can identify a category (20) corresponding to a user context (10) among a plurality of categories (20).

[0064] According to an embodiment, each of a plurality of categories (20) is classified according to the user's emotions, health, situation, behavior, etc., and one or more processors (130) can identify a category (20) corresponding to a user context (10) among the plurality of categories (20).

[0065] For example, one or more processors (130) can input first sensing data or a user context (10) identified based on the first sensing data into a neural network model to identify a category (20) corresponding to the user context (10).

[0066] Here, the neural network model can output a category (20) corresponding to the user context (10) based on at least one of the user's emotional information or situational information. Depending on the embodiment, the neural network model can output a category (20) corresponding to the user context (10) among a plurality of previously classified categories (20), or can create and output a new category (20).

[0067] For example, if the emotion information is 'sadness' and the situation information is 'right after returning home', one or more processors (130) can identify 'a state in need of rest' as a category (20) corresponding to the user context (10).

[0068] For example, if the emotional information is 'boredom' and the situation information is 'exercise', one or more processors (130) can identify 'a state requiring motivation to exercise' as a category (20) corresponding to the user context (10).

[0069] For example, if the emotion information is 'anger', one or more processors (130) can identify 'a state requiring anger management tips' as a category (20) corresponding to the user context (10).

[0070] For example, if the situation information is 'studying', one or more processors (130) can identify 'a state requiring a method to increase concentration' as a category (20) corresponding to the user context (10).

[0071] According to an embodiment, one or more processors (130) may obtain preference information (30) corresponding to a category (20).

[0072] For example, preference information (30) may include at least one of the user's preferred activities or preferred contents corresponding to the category (20).

[0073] For example, one or more processors (130) may obtain preference information (30) based on the user's usage history information. For example, one or more processors (130) may identify the user's preferred activities or the user's preferred content based on the usage history information.

[0074] For example, if one or more processors (130) see a video with calm music in a forest background while the user is resting according to usage history information, if the category (20) corresponding to the user context (10) is 'state in need of rest', the processors (130) can identify 'video playback' as a preferred activity and 'calm music' and 'video with a forest background' as preferred contents to obtain preference information (30).

[0075] For example, if a user listens to music of 120 to 140 bpm while exercising, based on usage history information, one or more processors (130) can identify 'music playback' as a preferred activity and 'music of 120 to 140 bpm' as preferred content if the category (20) corresponding to the user context (10) is 'a state requiring motivation for exercising', and obtain preference information (30).

[0076] However, the present invention is not limited thereto, and one or more processors (130) may input the first sensing data or the user context (10) identified based on the first sensing data into a neural network model to obtain preference information (30) corresponding to the user context (10).

[0077] Here, the neural network model can learn the user's usage history information and output preference information (30) corresponding to the user context (10) based on at least one of the user's emotional information or situational information.

[0078] However, the present invention is not limited thereto, and the neural network model may output preference information (30) when a category (20) corresponding to a user context (10) is input. For example, one or more processors (130) may input a 'state requiring a method to increase concentration' into the neural network model to output 'playing music' as a preferred activity and 'content including white noise' as a preferred content. Depending on the embodiment, the preference information (30) may include at least one of a preferred activity or preferred content.

[0079] According to an embodiment, the display device (100) includes a memory, and preference information (30) corresponding to each of a plurality of categories (20) classified in advance may be pre-stored in the memory.

[0080] FIG. 4 is a diagram illustrating a prompt corresponding to a user context according to an embodiment of the present disclosure.

[0081] Referring to FIG. 4, one or more processors (130) identify a category (20) corresponding to a user context (10), and input preference information (30) corresponding to the identified category (20) into a first neural network model (1) to obtain a prompt (A) corresponding to the user context (10).

[0082] For example, as described in FIG. 3, if the category (20) corresponding to the user context (10) is 'state requiring rest', one or more processors (130) can identify 'video playback' as a preferred activity (31) and 'calm music' and 'video with forest background' as preferred content (32) to obtain preference information (30).

[0083] However, this is an example for convenience of explanation, and it is obvious that preferred activities (31) and preferred contents (32) may change depending on the user's usage history information.

[0084] According to an embodiment, one or more processors (130) may input preference information (30) including at least one of a preferred activity (31) and a preferred content (32) into a first neural network model (1) to obtain a prompt (A).

[0085] In some embodiments, the prompt (A) may include text to be input into the second neural network model to obtain an image corresponding to the user context. For example, the prompt (A) may include instructions (or commands) that cause the second neural network model to generate an image corresponding to the user context, or conditions (or information) regarding the image.

[0086] For example, the prompt (A) may include text corresponding to at least one of the background composing the video, an object included in the video, a sound included in the video, or the length of the video. Referring to FIG. 3, when preference information (30) is input to the first neural network model (1), the first neural network model (1) may output as the prompt (A) “Generate a 10-minute meditation video with ‘forest background’ and ‘calm music’ as background music (BGM)” based on at least one of the preferred activity (31) and the preferred content (32).

[0087] According to an embodiment, the first neural network model (1) may be a large language model (LLM).

[0088] According to an embodiment, the prompt (A) is a text to be input to a second neural network model that generates an image corresponding to a user context, and may include an 'Instruction' that indicates a specific task or instruction that the second neural network model wants to perform, 'Context Information' that indicates external information or additional context that can adjust the second neural network model, 'Input Data' that indicates an input or question for which an answer is sought, and 'Output Data' that indicates a type or format of an output of the second neural network model.

[0089] According to an embodiment, the prompt (A) may be a story line of an image generated by the second neural network model.

[0090] FIG. 5 is a drawing for explaining an image corresponding to a prompt according to an embodiment of the present disclosure.

[0091] Referring to FIG. 5, when the first neural network model (1) outputs a prompt (A), one or more processors (130) can input the prompt (A) to the second neural network model (2) to obtain an image (B).

[0092] According to an embodiment, the second neural network model (2) may be a generative artificial intelligence.

[0093] For example, the second neural network model (2) may be a TTV (Text-to-Video) model trained to generate an image (B) corresponding to a prompt (A) when a prompt (A) containing text is input.

[0094] The artificial intelligence-related function according to the present disclosure is operated through the processor (130) and memory of the display device (100).

[0095] The processor (130) may be composed of one or more processors. In this case, the one or more processors may include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an NPU (Neural Processing Unit), but is not limited to the examples of the processors described above.

[0096] CPUs are general-purpose processors capable of performing not only general calculations but also artificial intelligence calculations. Their multi-layered cache structure allows for the efficient execution of complex programs. CPUs are advantageous for serial processing, enabling organic linking of previous and subsequent calculation results through sequential calculations. General-purpose processors are not limited to the examples described above, except where specifically identified as CPUs.

[0097] A GPU is a processor designed for large-scale computations, such as floating-point operations used in graphics processing. It integrates a large number of cores to perform large-scale computations in parallel. In particular, GPUs may be advantageous over CPUs in parallel processing methods, such as convolution operations. Furthermore, GPUs can be used as coprocessors to supplement the functions of CPUs. Processors for large-scale computations are not limited to the examples described above, except in cases where they are specifically referred to as GPUs.

[0098] An NPU is a processor specialized in artificial intelligence computation using artificial neural networks, and each layer of the artificial neural network can be implemented in hardware (e.g., silicon). Since an NPU is designed specifically according to the company's specifications, it has less freedom than a CPU or GPU, but can efficiently process the AI ​​computations requested by the company. Meanwhile, as a processor specialized in artificial intelligence computation, an NPU can be implemented in various forms, such as a Tensor Processing Unit (TPU), an Intelligence Processing Unit (IPU), or a Vision Processing Unit (VPU). Except as specifically stated as an NPU, an AI processor is not limited to the examples described above.

[0099] Additionally, one or more processors may be implemented as a System on Chip (SoC). In this case, in addition to one or more processors, the SoC may further include memory and a network interface, such as a bus, for data communication between the processor and the memory.

[0100] When a plurality of processors are included in a SoC (System on Chip) included in a display device (100), the display device (100) may perform operations related to artificial intelligence (e.g., operations related to learning or inference of an artificial intelligence model) by using some of the plurality of processors. For example, the display device (100) may perform operations related to artificial intelligence by using at least one of a GPU, an NPU, a VPU, a TPU, and a hardware accelerator specialized in artificial intelligence operations such as convolution operations and matrix multiplication operations among the plurality of processors. However, this is merely an example, and it is of course possible to process operations related to artificial intelligence by using a CPU or a general-purpose processor.

[0101] Additionally, the display device (100) can perform operations related to functions related to artificial intelligence by utilizing multiple cores (e.g., dual cores, quad cores, etc.) included in a single processor. In particular, the display device (100) can perform artificial intelligence operations, such as convolution operations and matrix multiplication operations, in parallel by utilizing multiple cores included in the processor.

[0102] One or more processors are controlled to process input data according to predefined operating rules or artificial intelligence models stored in memory. The predefined operating rules or artificial intelligence models are characterized by being created through learning.

[0103] Here, "created through learning" means that a predefined set of behavioral rules or an AI model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be performed on the device itself, where the AI ​​according to the present disclosure is implemented, or through a separate server / system.

[0104] An artificial intelligence model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its own operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0105] A learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0106] FIG. 6 is a drawing for explaining a display device for identifying a user context according to an embodiment of the present disclosure.

[0107] Referring to FIG. 6, the sensor (110) of the display device (100) can sense the user and obtain first sensing data. For example, the sensor (110) can obtain image data including the user or voice data of the user (e.g., "I'm tired").

[0108] According to an embodiment, one or more processors (130) may analyze the user's emotions or the user's situation based on at least one of image data or audio data.

[0109] According to an embodiment, one or more processors (130) may identify a user context (10) corresponding to the first sensing data from a memory in which emotional information or situational information corresponding to at least one of the user's pose, facial expression, or surrounding environment is previously stored, or may input the first sensing data into a neural network model to identify the user context (10).

[0110] For example, one or more processors (130) can identify the user's emotion or the user's situation based on the user's pose, the user's facial expression, or the user's surrounding environment included in the image data.

[0111] For example, one or more processors (130) may identify the user's situation information as 'resting' when the user's pose is sitting or lying on a sofa, and may analyze the user's facial expression to identify the user's emotional information as 'sadness'.

[0112] For example, one or more processors (130) may identify the user's context information as 'resting' if the user's surroundings are dark and the lights are on.

[0113] However, this is an example for the convenience of explanation, and the user's pose, expression, etc. can be changed in various ways, and the user's situation information and emotional information can also be changed in various ways.

[0114] One or more processors (130) can identify the user's emotions or situations based on the user's voice included in the voice data, the user's surrounding noise, etc.

[0115] For example, one or more processors (130) may identify the user's emotional information as 'tiredness' based on the user's voice mentioning 'I'm tired', and may identify the user's situation information as 'resting' if the user's surrounding noise is below a threshold decibel.

[0116] This is an example for the convenience of explanation, and the user's voice, the user's surrounding noise, etc. can be changed in various ways, and the user's situational information and emotional information can also be changed in various ways.

[0117] For example, one or more processors (130) may identify the user's context information as 'exercise' based on the user's voice counting the number of times they exercise, or may identify the user's context information as 'cooking' based on ambient noise including the sound of boiling water.

[0118] According to an embodiment, one or more processors (130) may identify a category (20) corresponding to a user context (10) among a plurality of categories based on a user context (10) including at least one of emotional information and situational information, and input preference information (30) corresponding to the identified category (20) into a first neural network model (1) to obtain a prompt (A).

[0119] FIG. 7 is a diagram illustrating a display device that identifies a user context by communicating with an external device according to an embodiment of the present disclosure.

[0120] According to an embodiment, the display device (100) may further include a communication interface (140).

[0121] According to an embodiment, the communication interface (140) may include a wired or wireless input / output interface (or input / output terminal) according to various standards. For example, the communication interface (11) may include various interfaces such as HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), DVI (Digital Visual Interface), AP-based Wi-Fi (Wireless LAN network), Bluetooth, Zigbee, wired / wireless LAN (Local Area Network), WAN (Wide Area Network), Ethernet, IEEE 1394, AES / EBU (Audio Engineering Society / European Broadcasting Union), optical, coaxial, etc.

[0122] According to an embodiment, one or more processors (130) may communicate with an external device (200) via a communication interface (140).

[0123] According to an embodiment, the external device (200) may include a wearable device. The wearable device may be implemented as at least one of an accessory type (e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD)), a fabric or clothing-integrated type (e.g., an electronic garment), a body-attached type (e.g., a skin pad or tattoo), or a bio-implantable circuit.

[0124] However, it is not limited thereto, and the external device (200) may be a set-top box, a cloud server, an over-the-top media service (OTT) server, a streaming service (e.g., Samsung Gaming Hub), a home automation control panel, a security control panel, a media box (e.g., Samsung HomeSync). TM , Apple TV TM , or Google TV TM ), game consoles (e.g. Xbox TM , PlayStation TM , Switch TM ), electronic dictionary, electronic key, camcorder, or electronic picture frame. However, the present invention is not limited thereto, and the external device (200) may include at least one of a TV, a user terminal device, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a PDA, a portable multimedia player (PMP), an MP3 player, a medical device, a camera, a virtual reality (VR) implementation device, or a spatial computing device.

[0125] According to an embodiment, an external device (200) may obtain first sensing data including at least one of image data including a user or voice data of the user and transmit the data to a display device (200).

[0126] According to an embodiment, the external device (200) may obtain second sensing data and transmit it to the display device (200). For example, the second sensing data may include at least one of the user's activity level (e.g., number of steps, amount of exercise, etc.), heart rate, blood pressure, body temperature, ECG (electrocardiogram), body composition, sleep pattern, or stress index.

[0127] According to an embodiment, when second sensing data is received from an external device (200), one or more processors (130) may identify a user context (10) based on the second sensing data.

[0128] For example, an external device (200) can identify a stress index when the user's heart rate increases or changes irregularly through a heart rate sensor (e.g., HRM sensor).

[0129] According to an embodiment, when second sensing data including a stress index is received from an external device (200), one or more processors (130) can identify 'anger', 'depression', 'fatigue', etc. as user emotion information and obtain a user context (10) including the emotion information.

[0130] According to an embodiment, if the emotion information is 'anger', one or more processors (130) can identify a 'state requiring anger management tips' as a category (20) corresponding to the user context (10).

[0131] According to an embodiment, one or more processors (130) may input a category (20) corresponding to a user context (10), 'state requiring anger management tips', into a neural network model, or may use usage history information to identify 'video playback' as a preferred activity (31) and 'calm music' and 'video with a forest background' as preferred content (32) to obtain preference information (30).

[0132] According to an embodiment, one or more processors (130) may input preference information (30) into a first neural network model (1) to obtain a prompt corresponding to a user context. According to an embodiment, the first neural network model may generate a prompt including text corresponding to at least one of a background constituting the image (B), an object included in the image (B), music included in the image (B) (e.g., background music, etc.), or the length of the image, based on at least one of a preferred activity (31) or a preferred content (32).

[0133] However, this is not limited thereto. For example, if the user's situation information is 'sleep deprivation' based on the sleep pattern included in the second sensing data, one or more processors (130) may identify 'a state requiring sleep-inducing images' as a category (20) corresponding to the user context (10).

[0134] According to an embodiment, one or more processors (130) may input a category (20) corresponding to a user context (10), 'state requiring sleep-inducing images', into a neural network model, or may use usage history information to identify 'image playback' as a preferred activity (31) and 'natural background images including bird sounds, wave sounds, wind sounds, etc.' as preferred content (32) to obtain preference information (30).

[0135] According to an embodiment, one or more processors (130) may input preference information (30) into a first neural network model (1) to obtain a prompt corresponding to a user context. According to an embodiment, the first neural network model may output 'a 10-minute nature video including bird sounds, wave sounds, wind sounds, etc.' as a prompt (A).

[0136] FIG. 8 is a drawing for explaining status information and surrounding environment information of a display device according to an embodiment of the present disclosure.

[0137] According to an embodiment of the present disclosure, the user context (10) may include at least one of status information of the display device (100) or surrounding environment information of the display device (100).

[0138] For example, when one or more processors (130) identify receipt of a notification based on status information of the display device (100), they can identify a user context (10) indicating receipt of the notification.

[0139] For example, one or more processors (130) may identify a user context (10) indicating a change in the location of the display device (100) when the location of the display device (100) changes within a preset space (e.g., a home) based on the status information of the display device (100).

[0140] According to an embodiment, the user context (10) may include at least one of status information of the external device (200) or surrounding environment information of the external device (200).

[0141] For example, when the reception of a notification is identified based on the status information of the external device (200), one or more processors (130) can identify a user context (10) indicating the reception of a notification, and when the location of the external device (200) is changed within a preset space based on the status information of the external device (200), one or more processors (130) can identify a user context (10) indicating a change in the location of the external device (200).

[0142] Referring to FIG. 8, when a user context (10) indicating receipt of a notification from an external device (200) is identified, one or more processors (130) can identify a category (20) corresponding to the user context (10) among a plurality of categories (20).

[0143] For example, one or more processors (130) can identify a 'state of receiving a notification' as a category (20) corresponding to a user context (10) and obtain preference information (30) corresponding to the category (20). For example, one or more processors (130) can obtain preference information (30) by identifying a user's preferred activity or content based on usage history information.

[0144] For example, one or more processors (130) may identify ‘notification confirmation’ as a preferred activity (31) if the category (20) corresponding to the user context (10) is ‘notification received’.

[0145] According to an embodiment, one or more processors (130) may input preference information (30) into a first neural network model (1) to obtain a prompt (A). One or more processors (130) may input the prompt (A) into a second neural network model (2) to obtain an image (B) corresponding to the prompt (A).

[0146] FIG. 9 is a diagram illustrating a prompt corresponding to a user context according to an embodiment of the present disclosure.

[0147] Referring to FIG. 9, one or more processors (130) input preference information (30) including 'check notification' as a preferred activity (31) into a first neural network model (1), and the first neural network model (1) can output 'Create a video that conveys the notification content by displaying the notification content in the background and displaying the sender in some area' as a prompt (A).

[0148] One or more processors (130) input a prompt (A) to a second neural network model (2) and can obtain an image (B) corresponding to the prompt (A).

[0149] For example, if the notification content is 'I will go to the Taj Mahal next week', the second neural network model (2) generates an image (B) with the Taj Mahal as the background, the sender displayed in a certain area (e.g., the lower right), and including a voice corresponding to 'I will go to the Taj Mahal next week', and one or more processors (130) can display the image (B).

[0150] However, this is an example and is not limited thereto. For example, when the location of the external device (200) (e.g., a user terminal device) changes within a preset space (e.g., a home) based on status information of the external device (200), one or more processors (130) may identify a user context (10) indicating a change in the location of the external device (200).

[0151] According to an embodiment, one or more processors (130) may identify a 'state in which the user has returned home' as a category corresponding to the user context (10) based on the user context (10) in which the external device (200) has changed into a preset space according to the state information of the external device (200).

[0152] If one or more processors (130) check for notifications that the user has not checked after returning home based on usage history information, and if the category (20) corresponding to the user context (10) is 'the user has returned home', they can identify 'checking unchecked notifications' as a preferred activity and obtain preference information (30).

[0153] According to an embodiment, one or more processors (130) input preference information (30) including 'confirm unconfirmed notification' as a preferred activity (31) into a first neural network model (1), and the first neural network model (1) can output 'generate a video that conveys the notification content by displaying the notification content in the background and displaying the sender in some area' as a prompt (A).

[0154] As another example, if the user watches a video with calm music in a natural background after returning home, one or more processors (130) may, based on usage history information, identify 'video playback' as a preferred activity and 'calm music' and 'video with a natural background' as preferred contents if the category (20) corresponding to the user context (10) is 'the user is back home', and obtain preference information (30). According to an embodiment, the first neural network model (1) may output 'Generate a 10-minute video with 'natural background' and 'calm music' as background music (BGM)' as a prompt (A) based on at least one of the preferred activity (31) and the preferred contents (32).

[0155] FIG. 10 is a drawing for explaining a prompt corresponding to the surrounding environment information of a display device according to an embodiment of the present disclosure.

[0156] According to an embodiment, the user context (10) may include at least one of the surrounding environment information of the display device (100) or the surrounding environment information of the external device (200).

[0157] For example, when a pet is identified based on the surrounding environment information received from an external device (200), one or more processors (130) can identify the 'pet identified state' as a category (20) corresponding to the user context (10).

[0158] According to an embodiment, if one or more processors (130) play 'videos that dogs like' according to usage history information, if the category (20) corresponding to the user context (10) is 'a pet is identified', 'videos that dogs like' can be identified as preferred content (32) and 'video playback' can be identified as preferred activity (31) to obtain preference information (30).

[0159] According to an embodiment, one or more processors (130) may input preference information (30) into the first neural network model (1) and output 'Generate a 20-minute video containing the dog's favorite colors and sounds' as a prompt (A).

[0160] However, this is an example for convenience of explanation and is not limited thereto.

[0161] For example, if the external device (200) includes a motion detection sensor attached to a pet, one or more processors (130) can receive environmental information received from the external device (200) to identify the user context (10). Depending on the embodiment, the environmental information may include the pet's activity level (e.g., number of steps, etc.).

[0162] According to an embodiment, one or more processors (130) may identify the activity level of a pet based on the surrounding environment information received from an external device (200), and if the activity level is less than a preset activity level, identify the 'state in which the pet needs exercise' as a category (20) corresponding to the user context (10).

[0163] According to an embodiment, if one or more processors (130) play a 'video of a dog walking' based on usage history information, if the category (20) corresponding to the user context (10) is 'a state in which the pet needs exercise', the 'video of a dog running around' can be identified as a preferred content (32) and 'video playback' can be identified as a preferred activity (31) to obtain preference information (30).

[0164] According to an embodiment, one or more processors (130) may input preference information (30) into the first neural network model (1) and output a prompt (A) such as 'Generate a video that guides the dog's movement to achieve a preset amount of activity or a video of the dog running around.'

[0165] According to an embodiment, one or more processors (130) may obtain preference information (30) using user usage history information, and input user context (10) into a third neural network model to obtain at least one of a category (20) corresponding to the user context (10) among a plurality of categories (20), a guide activity, or guide content corresponding to the category.

[0166] One or more processors (130) can input at least one of the guided activities or guided activities into the first neural network model (1) to obtain a prompt (A).

[0167] When a user context (10) is input, the third neural network model can output each of the activities preferred by an unspecified number of users and the contents preferred by an unspecified number of users as guide activities or guide contents.

[0168] Returning to FIG. 2, according to an embodiment, the display device (100) further includes memory, which can store data required for various embodiments of the present disclosure. Depending on the data storage purpose, the memory may be implemented in the form of memory embedded in the display device (100) or in the form of memory that can be attached or detached to the display device (100).

[0169] For example, data for driving the display device (100) may be stored in a memory embedded in the display device (100), and data for the extended functions of the display device (100) may be stored in a memory that can be attached or detached to the display device (100). Meanwhile, in the case of memory embedded in the display device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)). In addition, in the case of memory that can be detachably attached to the display device (100), it may be implemented as at least one of memory cards (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc. It can be implemented.

[0170] In one example, the memory may store a computer program including at least one instruction or instructions for controlling the display device (100).

[0171] According to an embodiment, the display device (100) can generate a prompt (A) in an ambient mode that displays preset content, and generate and play an image (B) corresponding to the prompt (A).

[0172] For example, the display device (100) can operate in an ambient mode (or standby mode) that displays schedules, weather, stocks, etc., and can generate and play a video (B) corresponding to a prompt (A) in the ambient mode rather than in a general mode that displays OTT videos, streaming videos, etc.

[0173] According to an embodiment, the display device (100) can input at least one of the first sensing data, the second sensing data, the status information of the display device (100), the surrounding environment information of the display device (100), the status information of the external device (200), or the surrounding environment information of the external device (200) into a neural network model to obtain a prompt (A), and can input the prompt (A) into a second neural network model (2) to obtain an image (B).

[0174] According to an embodiment, the display device (100) can obtain an image (B) by inputting at least one of the first sensing data, the second sensing data, the status information of the display device (100), the surrounding environment information of the display device (100), the status information of the external device (200), or the surrounding environment information of the external device (200) into a neural network model.

[0175] According to an embodiment, the display device (100) may transmit at least one of the first sensing data, the second sensing data, the status information of the display device (100), the surrounding environment information of the display device (100), the status information of the external device (200), or the surrounding environment information of the external device (200) to an external server, and may receive a prompt (A) or an image (B) corresponding to the prompt (A) from the external server.

[0176] For example, the display device (100) may identify a category (20) corresponding to a user context (10) and transmit at least one of the preferred activities or preferred contents corresponding to the identified category (20) to an external server. According to an embodiment, when the display device (100) receives an image (B) from an external server, the display device (100) may display the image (B).

[0177] FIG. 11 is a flowchart for explaining a method for controlling a display device according to an embodiment of the present disclosure.

[0178] A method for controlling a display device according to an embodiment of the present disclosure identifies a user context based on first sensing data (S1110).

[0179] The control method identifies a category corresponding to the user context among a plurality of categories based on at least one of the user's emotional information or situational information included in the user context (S1120).

[0180] The control method inputs preference information corresponding to the identified category into a first neural network model to obtain a prompt corresponding to the user context (S1130).

[0181] The control method displays an image corresponding to the prompt (S1140).

[0182] Preference information according to an embodiment includes at least one of the user's preferred activities or preferred contents corresponding to the identified category.

[0183] According to an embodiment, the first sensing data includes at least one of image data including a user or voice data of the user, and the step S1110 of identifying a user context may include a step of obtaining emotional information or situational information based on the user's pose included in the image data, the user's facial expression, the user's surrounding environment, or the user's voice included in the voice data.

[0184] A control method according to an embodiment further includes a step of receiving second sensing data from an external device, and the step S1110 of identifying a user context includes a step of identifying a user context based on the second sensing data, and the second sensing data may include at least one of a user's activity level, heart rate, blood pressure, body temperature, ECG, sleep pattern, or stress index.

[0185] The prompt includes text corresponding to at least one of a background, an object, a sound, or a video length that constitutes a video, and the displaying step S1140 includes a step of inputting the prompt to a second neural network model to obtain a video corresponding to the prompt, and the second neural network model may be a TTV (Text-to-video) model trained to generate a video corresponding to the prompt when a prompt including text is input.

[0186] A first neural network model according to an embodiment may generate a prompt including text corresponding to at least one of a background, an object, a sound, or a video length based on at least one of a preferred activity or a preferred content.

[0187] The user context according to the embodiment further includes at least one of status information of the display device or surrounding environment information of the display device, and the step S1120 of identifying a category may include a step of identifying a category corresponding to the user context among a plurality of categories based on at least one of the status information or surrounding environment information.

[0188] The step S1120 of identifying a user context according to an embodiment includes a step of identifying a user context indicating receipt of a notification when receipt of a notification is identified based on status information of a display device, and the step S1130 of obtaining a prompt includes a step of obtaining a prompt by inputting a preferred activity of a user who confirms a notification based on preference information into a first neural network model, and an image corresponding to the prompt may be an image that provides a notification to the user based on a sender of the notification and the content of the notification.

[0189] The control method according to the embodiment further includes a step of inputting a user context into a third neural network model to obtain at least one of a category corresponding to the user context among a plurality of categories and a guide activity or guide content corresponding to the category, and the step S1120 of obtaining a prompt may include a step of inputting at least one of the guide activity or guide content into a first neural network model to obtain a prompt.

[0190] The displaying step S1140 according to the embodiment may include a step of displaying an image corresponding to a prompt in an ambient mode in which the display device displays preset content.

[0191] The control method according to the embodiment further includes a step of transmitting at least one of the preferred activities or preferred contents corresponding to the identified category to an external server, and the displaying step S1140 may include a step of receiving an image corresponding to the prompt from the external server and displaying the image.

[0192] However, it goes without saying that the various embodiments of the present disclosure can be applied not only to display devices but also to various types of electronic devices including display functions.

[0193] Meanwhile, the various embodiments described above may be implemented in a computer-readable recording medium or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments, such as the procedures and functions described herein, may be implemented as separate software modules. Each of the software modules may perform at least one function and operation described herein.

[0194] Meanwhile, computer instructions for performing processing operations of the display device (100) according to the various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When the computer instructions stored in the non-transitory computer-readable medium are executed by a processor of a specific device, they cause the specific device to perform processing operations in the display device (100) according to the various embodiments described above.

[0195] A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0196] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. Display; sensor; and Identifying a user context based on the first sensing data received through the above sensor, Identifying a category corresponding to the user context among a plurality of categories based on at least one of the user's emotional information or situational information included in the user context, Preference information corresponding to the above-identified category is input into the first neural network model to obtain a prompt corresponding to the user context, one or more processors controlling the display to display an image corresponding to the prompt; The above preference information is, A display device comprising at least one of the user's preferred activities or preferred contents corresponding to the identified category.

2. In paragraph 1, The above first sensing data is, Contains at least one of image data including the user or voice data of the user, One or more of the above processors, A display device that obtains the emotional information or the situation information based on the user's pose included in the image data, the user's facial expression, the user's surrounding environment, or the user's voice included in the voice data.

3. In paragraph 1, further comprising a communication interface; One or more of the above processors, Receive second sensing data from an external device through the above communication interface, Identifying the user context based on the second sensing data, The above second sensing data is, A display device comprising at least one of the user's activity level, heart rate, blood pressure, body temperature, ECG, sleep pattern, or stress index.

4. In paragraph 1, The above prompt is, Contains text corresponding to at least one of the background, object, sound or video length that constitutes the above video, One or more of the above processors, By inputting the above prompt into the second neural network model, an image corresponding to the above prompt is obtained, The above second neural network model is, A display device, which is a text-to-video (TTV) model trained to generate the image corresponding to the prompt when the prompt including the text is input.

5. In paragraph 4, The above first neural network model is, A display device that generates the prompt including the text corresponding to at least one of the background, the object, the sound, or the video length based on at least one of the preferred activity or the preferred content.

6. In paragraph 1, The above user context is, further comprising at least one of status information of the display device or surrounding environment information of the display device, One or more of the above processors, A display device that identifies a category corresponding to the user context among the plurality of categories based on at least one of the status information or the surrounding environment information.

7. In paragraph 6, One or more of the above processors, When the reception of a notification is identified according to the status information of the display device, the user context indicating the reception of the notification is identified, The user's preferred activity for checking the notification based on the preference information is input into the first neural network model to obtain the prompt, Display the image corresponding to the above prompt, The above video corresponding to the above prompt is, A display device that provides the notification to the user based on the sender of the notification and the content of the notification.

8. In paragraph 1, One or more of the above processors, By inputting the user context into a third neural network model, at least one of the categories corresponding to the user context among the plurality of categories and the guide activity or guide content corresponding to the category is obtained, A display device that obtains the prompt by inputting at least one of the above guide activities or the above guide contents into the first neural network model.

9. In paragraph 1, One or more of the above processors, A display device that displays the image corresponding to the prompt in an ambient mode that displays preset content.

10. In paragraph 1, further comprising a communication interface; One or more of the above processors, Transmitting at least one of the preferred activities or the preferred contents corresponding to the identified category to an external server through the communication interface; A display device that receives and displays an image corresponding to the prompt from the external server.

11. In a method for controlling a display device, A step of identifying a user context based on first sensing data; A step of identifying a category corresponding to the user context among a plurality of categories based on at least one of the user's emotional information or situational information included in the user context; A step of inputting preference information corresponding to the identified category into a first neural network model to obtain a prompt corresponding to the user context; and a step of displaying an image corresponding to the above prompt; The above preference information is, A control method comprising at least one of the user's preferred activities or preferred contents corresponding to the identified category.

12. In paragraph 11, The above first sensing data is, Contains at least one of image data including the user or voice data of the user, The step of identifying the above user context is: A control method comprising: a step of obtaining the emotional information or the situation information based on the user's pose included in the image data, the user's facial expression, the user's surrounding environment, or the user's voice included in the voice data.

13. In paragraph 11, The above control method is, further comprising a step of receiving second sensing data from an external device; The step of identifying the above user context is: A step of identifying the user context based on the second sensing data; The above second sensing data is, A control method comprising at least one of the user's activity level, heart rate, blood pressure, body temperature, ECG, sleep pattern, or stress index.

14. In paragraph 11, The above prompt is, Contains text corresponding to at least one of the background, object, sound or video length that constitutes the above video, The above displaying step is, A step of inputting the above prompt into a second neural network model to obtain an image corresponding to the prompt; The above second neural network model is, A control method, wherein the TTV (Text-to-video) model is trained to generate the video corresponding to the prompt when the prompt including the text is input.

15. A non-transitory computer-readable medium storing computer instructions that, when executed by a processor of a display device, cause the display device to perform an operation, The above action is, A step of identifying a user context based on first sensing data; A step of identifying a category corresponding to the user context among a plurality of categories based on at least one of the user's emotional information or situational information included in the user context; A step of inputting preference information corresponding to the identified category into a first neural network model to obtain a prompt corresponding to the user context; and a step of displaying an image corresponding to the above prompt; The above preference information is, A non-transitory computer-readable medium comprising at least one of the user's preferred activities or preferred contents corresponding to the identified category.

Images